LARRUZ tested top AI models playing Super Mario Bros in real time. Claude 3.7 crushed Gemini and GPT4O with higher scores and faster response times. The test shows that speed matters as much as reasoning ability when an AI has to make split-second decisions.
March 10, 2025.
Cloud 3.7, Google’s Gemini, and OpenAI’s GPT4O.
An emulator powered by Gaming Agent, using Python commands and live screenshots to control the game in real time.
If you thought Pokemon was the ultimate AI challenge, it turns out Super Mario Brothers is even tougher. Researchers recently tested top AI models by having them play Super Mario Bros and the results were surprisingly clear. Cloud 3.7 dominated, achieving higher scores and smoother gameplay, leaving Google's Gemini and OpenAI's GPT4O significantly behind in both response time and successful maneuver execution. The AIs weren't playing the classic Mario from your childhood. They used an emulator powered by Gaming Agent, a system letting AI control Mario in real time through Python commands based on live screenshots. But here's why that's so hard. Real time gameplay demands instant decisions. Usually strong reasoning based models like GPT4O were surprisingly slow because thinking for just a second too long means Mario falls straight off a cliff. Therefore, speed was just as important as intelligence. This new benchmark is reshaping our understanding of AI's true capability by testing real time decision making and response speeds. Unlike turn based benchmarks like Pokemon, watching AI play Mario is more than entertaining. It's a fascinating preview of how AI can handle complex split second decisions in the real world.
