Grok 3: “Smartest AI on Earth” Takes Down o3 mini, DeepSeek in Record time.

So last night, guys, Elon Musk unveiled Grok3 with his XAI team of researchers. Grok3 is a next generation level chatbot, according to Musk, and he touts it as the quote unquote, smartest AI on earth. If you saw my last video, you'll know that Musk has been hyping this release up as a major leap forward in the AI space. He even called it scary, spooky smart. And if you know anything about the AI community, all of this hyping, all of this talk means that both excitement and skepticism are sky high through the roof right now surrounding Grok 3. So that's what the goal of today's video is. We're going to get to the bottom of everything, look at a few tests, community reactions, and we're going to suss out what's what. If you will. We can see early Grok 3. This I don't believe is the model that we have access to today. This score right here is a little bit further behind in the training apparently for Groq3, but it actually did outperform rivals like OpenAI or Google or Deep Seek. In early testing we see a number on the Elemsys arena with the first model to achieve an ELO score of 1400, 1402 to be exact. Right behind it we have Gemini 2.0 flash thinking experimental at 1385, then 2.0 pro experimental at 1379. Then we see ChatGPT4O latest at 1377. Finally Deepseek R1 at 1361. And keep in mind, even further down the list here we can see the original 01 and 01 preview again further down the line. Shockingly, 03 mini is all the way down here to 1305. We do have to keep in mind that this is an Elemsys arena score. It's a bunch of different votes over time. There can be certain user preferences depending on the task at hand. You might want to choose a different model even. That's why we look at diversified benchmarks. But yes, in LMSIS arena it has the number one spot. And this is again an earlier version of Grok 3, not the version that was released and showed off yesterday and today. Really, I think Elon is signaling a direct challenge to industry AI leaders. Remember he helped co found OpenAI back in the day, even naming it, and now he's building a competitor from scratch. The AI community is buzzing about this because it's not an incremental update. It's a pretty large leap forward past Grok 2. And they did so in unprecedented time. But will Musk be able to swing the positioning that this is a potential game changer in an already crowded field, especially when the ELO scores here are so darn close. So what exactly is new here surrounding Grok 3? Well, we're starting off with a 10 power boost over Grok 2. This AI was trained on a massive cluster of GPUs. 200,000 to be exact. They had a first phase that was 122 days long, a hundred thousand GPUs, and then a second phase of 92 days expanded up to 200,000 GPUs. Obviously the theory of scaling here is, well, we're training it on more compute, meaning more complex reasoning and fewer dumb mistakes. So Elon essentially put Grok on steroids to really push and drive up its capabilities. But here's the thing. XAI didn't just build one model, they actually built multiple models. So we're going to talk about the reasoning model next. Obviously there's the base GRO3 model that the reasoning model is fine tuned and based off of. But the reasoning model as suggested, is designed to mimic human esque critical thinking. A lot of people thought that XAI wasn't going to have a reasoning model ready. They thought it was just going to be, you know, Grok 3 trained on more compute, a scaled up model. But they really did go off and fine tune a reasoning model as well. And they've also got some other tricks up their sleeve which we'll talk about later in this video. This really shocked me. This was a big surprise. Not only did they train a massive model, but they also fine tuned it for reasoning and unlocked the potential capabilities that Test Time Compute provides. You can see on their testing here, they enabled the Think option in the interface generate code for an animated 3D plot of a launch from Earth landing on Mars and then back to Earth at the next launch wind. So a difficult request, but a very simple prompt, literally like a sentence long. But watch, I'll play the video here you can see it starts the thinking and the tokens just fly out. It does all kinds of math. It's trying to obviously calculate trajectory from Earth to Mars and then back. And then it actually has to, you know, animate this 3D plot. So tall order. But as you can see, it did eventually finish the prompt and write out all the code. And you know what, dang, that's pretty crazy because it actually kind of did it. It did a pretty good job animating this. I mean It's a little bit crude, obviously, but we've got an axis here. We've got the sun, Earth, and then Mars and then the actual spacecraft. And the animation here actually shows it moving from Earth to Mars and then back around at the most optimal points in space. There's. There's a lot of complexity to this task. It's something that I definitely wouldn't be able to replicate. So, you know, dare I say, I'm impressed. They also tried another prompt where they tried to combine Tetris and Bejeweled and it sort of worked. Now, for this problem, they enabled something called Big Brain mode, which apparently just uses more compute. In the live stream, they did mention the idea of running multiple chain of thoughts at once and then combining the efforts of all of those chain of thoughts together to potentially get better outputs. So potentially, maybe that's what Big Brain mode is. It's using more compute, running more chains of thoughts at once to get you a better result. Don't quote me on that, that's just a theory. So the result with this game though, Bejeweled and Tetris, it was pretty good. It definitely had elements of both Bejeweled and Tetris. You can see it has the classic Tetris falling blocks and then when three blocks are aligned, it will Bejeweled style remove those blocks. So it's a pretty primitive combination of both games, but still, nevertheless, it's impressive for an AI to be able to kind of just one shot something like this out. Definitely not an easy task. Again, I wouldn't be able to replicate. So consider me impressed. At the end of the day. The question remains though, can other AI models, Deepseek, R1 03, mini, etc. Actually also complete this task? Now, they also mentioned agents as well, which they call the next frontier. And I think we can all kind of agree that autonomous action and autonomous work really looking like the future of large language models at the very least. But to show off agents, they also built a Deep Research competitor. Of course. I mean, that's crazy. I wasn't expecting them to come out of the gate with a reasoning model, let alone also a Big Brain mode, whatever that means, and a Deep Research competitor all at once. Again, that's why I'm saying that they're really trying to position this as competition. A wake up call to other AI industry leaders saying, hey, we can do it too and we can do it fast. So obviously, guys, this is super, super familiar to OpenAI's deep research that was just released and I was really impressed with Deep Research. I used it in the past and I was very, very impressed with it. Actually just did a video a couple of days ago on it. But this, yeah, like I said, looks super similar. It's browsing and comparing and contrasting all these different web pages, X posts and results. And it's able to do pretty deep analysis tasks, research paper esque with, you know, in text, citations and long, long spouts of writing. So what's the deal actually surrounding these new features and getting access to these things ourselves? Well, you can access all of these features with Premium plus on the X app and then there's also a separate Grok as well. So if you have the Premium plus subscription on X, you get access to all these features out of the box. They're Deep Search. It's not Deep Research, guys. It's Deep Search, Advanced Reasoning and increased usage limits. Also, I believe Big Brain mode. Right, but if you don't care about Twitter and using those other X features, you can just pay for Super Grok. You know, at least it's not called Grok Pro or Grok Pro Max, right? But this appears to be pretty much the same thing as premium plus guaranteed access to Grok 3 deep search and Think modes, early access to potential new features, whatever that means, and I guess higher image gen limits. So again, direct competitor here to chat GPT and some of their recent features. I also believe that this is positioned at 30 bucks a month or $300 a year. So again, that's, that's competitive because if you want to access deep research from OpenAI, for example, you have to shell out for the $200 a month pro plan. And yeah, it comes with other stuff like Sora. But if you don't care about Sora and you just want Deep Research, like, why would you buy that? Especially now that a competitor such as Grok3 is here with their own Deep Search. All right guys, let's get to the hard data. How does GROK actually stack up against the reigning champs? So we've got math, AIM24, science, GPQA and coding LCB. You'll find here that we've got two models from Grok, Grok 3 Reasoning Beta and Grok 3 Mini Reasoning. You'll see both of these Grok 3 models are actually significantly closer to each other. And in the livestream, the Researchers mentioned that Grok 3 Mini was a little bit further, further along its training than the Grok 3 reasoning beta we see here in this graph. But that is a sign that training is going very well, encoding, we can see that both Grok3 Reasoning Beta and Grok3 Mini Reasoning handedly take out O3 Mini High Original 01 and Deepseek R1 plus Gemini Flash 2 thinking by quite a lot here. So that's 79 and 80% respectively, with O3 Mini High only achieving a 74. They're close benchmarks for short, but man, I mean it does look like the XAI team cooked here. It's competitive with the likes of OpenAI's latest models that we have access to. You're gonna notice that this graph does include O3, which we do not have access to full O3. So while we don't have access to these models, it's still nice to have them on this graph to compare with with science. We can see again, O3 mini gets stomped on. Oh one gets stomped on. Deepseek R1 gets stomped on 85 and 84 respectively for these models compared to an 80, 78, 71 and 74. But hey, the unreleased full O3 according to OpenAI's benchmarks score an 88 here. So just a tad above Grok3's reasoning beta in Mini Reasoning and here in Math we're reaching near perfect scores. Grok 3 mini reasoning 96, Grok 3 reasoning beta 93 followed by 03 mini high at 87, then 01 at 83, Deepseek R1 at 80 and Gemini Flash thinking all the way down here at 73. O3 still takes the cake. Again, we don't have access to this model, it's not public, but according to OpenAI's benchmarks, it does perform a little bit better. But we're nearing the top of these benchmarks for sure. So impressive showing all the way around. And yeah, in terms of models you can actually access today, it appears that Grok3 with reasoning enabled is going to be the highest benchmarking model you can get your hands on. Pretty wild. It appears they might have actually cooked here, guys. Now it is also very important to point out when we're looking at these benchmarks, models such as DeepSeek R1 for example, are open source, where Grok 3 it's not being released open source. Certain particular use cases you might still want to choose DeepSeek R1 cost is also a factor here and also I've heard the XAPI isn't great. So for developers you might still want to go with an OpenAI model. Maybe you don't need the highest scoring benchmark model out there for whatever you're trying to build. Benchmark scores and numbers aren't the end all be all of these models. Important to keep that in mind. Now I will say though I do think this is essentially Musk rolling up to the party and saying hey, our model is better. OpenAI, anthropic, Google, whoever else can suck it. We did this in record time and we are topping the benchmarks. Now it's up to the others to respond. I think that it will put pressure on the other AI companies. I think OpenAI is going to sweat a little bit when they see this, especially especially because it happened so quickly. This is all in less than a year's time and it's scary, I think for OpenAI to think where XAI could be in just a year's time, right? If they keep accelerating at this rate, who knows, OpenAI might not get the chance to lap XAI and I. And again, I think the same also can be said for Google and Anthropic. Everyone's really competing here and something like this, it's impressive. As David Shapiro points out here, the TLDR here is that multiple competitors are now all in the same class and it only took a few months to close the gap after 01 was released, including shipping open source models like Deep Seq. From a macro perspective, Shapiro points out that there is no real moat for any of these companies except for compute. There's no special sauce that makes OpenAI way better than any other company except for having a lot of COMPUTE to train these models on. David Shapiro also points out that AI is intrinsically democratic here and what he means by that is that the the concept of the technology itself at hand here does not SW itself to one particular company or another. Again, no moat. All right, now I want to dive into some community testing, some community reactions. You guys already know that people in the AI space like to do rigorous testing and put the rubber to the road. In this video. I'm not going to be doing any personal testing. I want to save that for a separate video because I know this video is probably already going to be too long. So let me know if you want to see some GROK testing down in the comments below. But there are some really impressive and fun examples to take a look at. First up, we're starting off with some notable praise from big people, people like Lex Friedman. He said he got to use GROK extensively early and his mind was blown by it. Good point. In the comments here, Mackie says can't tell if praising Grock 3 on X is honest or if it's sucking up to Elon. Fair enough. So let's take a look at some other reactions next up, Technium saying that in his testing it was at least as good in thinking mode as O3 full deep research was despite O3 with deep research not being listed in any of the benchmarks. An interesting note is that Grok3 mini seems genuinely better than full in some cases. Again, Grok3 mini reasoning was a little bit farther along in its training, suggesting that the training was going very well. I think Technium's perspective is valid here though. Co founder and head of post training at Noose Research previously worked at Stability AI in an account I typically follow for his takes in the AI space. Also notably, Technium is a supporter of open source models like Deepseek R1, so I think this is some solid praise and a solid look that notable figures in the community do seem to enjoy using Grok 3 some other testing here from Sharia making Portal 2 or like a Doom Portal 2 clone, very rudimentary. Using Grok 3 was actually able to code the portals which is pretty incredible, even providing a little mini map up in the corner there. Sharia here claims that it's the only model that's been able to get this far in such a project. I love that AI is getting increasingly and increasingly better at coding and doing more complex and more complex tasks. This is a pretty abstract idea. Portals being placed on the sides of walls and then moving you to that position. Being able to move the portals as well through a maze. It's not easy to recreate something like this. I mean this would be a good demo to include on your resume, I'd imagine so. Impressive stuff. We've also got AK a fantastic account here on X for keeping up with AI news, especially open source stuff. Tried a prompt with Grok3 make a P5JS simulation of a sphere made up of assy numbers. It should be rotating, the closest numbers should be pure white and the furthest ones should fade to gray on a black background. And it absolutely did that. It made a sphere of these characters rotating around with the white fading to gray. Really cool example. But also O3 Mini High was able to create a very similar example which Javi Lopez shows off down here. Some of the comments here. Thorpe also changed it to a cube instead, which looks really cool and destroys the bitrate here on X Tom Caban also had it one shot a visual simulation of a supernova collapsing. Again, love all these little coding based demos. It's so cool what AI is able to create these days. Questions remain though, again, like you know, could O3 mini high replicate something like this as well? Overall with this though, fun benchmarks, fun ways to mess around with code and use AI to produce it. Also, we've got our friend Pliny the Liberator here. Jailbreaking it is essentially immediately according to Pliny, the guardrails folded to an old prompt on the first query, so not really a lot of safety built directly into Grok 3. That's something to keep in mind. Pliny notes that we could see some very wild outputs from this model, but some examples so far Ricin essentially poison recipe instructions for make a thermite guide to hacking Starlink satellites oh man, the lyrics to Grok's favorite dirty song incredible. Now admittedly Grok 3 already out of the box is a little bit less censored than models coming from anthropic OpenAI or Google, but it still does have some guardrails which are easily removed with a plany prompt. Pliny was also able to leak the system prompt for Grok3 which appears to be not very large at all. Obviously just some context around the tools it's able to use. Current date and then some notes about information and knowledge cutoff as I agree with many experts want to verify claims claims in results for sure. Bindu ready here notes that it would be great to get access to an API so we can verify the results. I would also love to see more benchmarks. I'd love to see more comparisons to other models. I think the common sentiment around most experts and people trying Grok3 out for the first time is Crock3 is very good. But let's not just claim it's the absolute best until we see more proof and fair enough, absolutely agree. Alright guys, so where does this put us? Well, first of all, the AI arms race is absolutely intensifying. No matter how you cut it, the release of Grok 3 is pretty impressive. This ramps up and puts pressure on competitors. We're likely to see everyone come to the table and step up their game here. I'm looking at you anthropic. Where are your models? In fact, we literally just talked about yesterday how Sam Altman is mentioning the OpenAI roadmap. We're gonna see GPT 4.5 and 5. Don't forget Google's continuously improving Gemini. The sort of competition is fantastic for innovation folks and it's also good for the consumer because at the end of the day we decide whether or not these are commerc commercial successes or commercial failures. Clearly here we as users are benefiting from more choices and faster improvements. And also don't forget that open source is a factor in this equation as well. Not only are open source models competing with each other, but they're also competing with the big dogs such as OpenAI or XAI or Google. And speaking of open source, it appears that XAI is going to be open sourcing the previous Grok 2, which would be pretty cool. We already have models that are better such as DeepSeek R1, but nevertheless I guess I appreciate the open source consideration that they are going to be providing. Grok 3 surprisingly strong showing here despite being closed source might encourage the other big players to also continuously keep their models closed source. However, Sam Altman just held a Twitter poll with two options for OpenAI to potentially open source in the future. So if OpenAI starts to open source models, I don't think XAI will have many good excuses. I think at the end of the day, with so many options to choose from, whichever AI that actually helps people at the best price is going to win the hearts and the minds of the community and not just bragging rights in benchmarks. One thing is clear though, 2025 is shaping up to be one hell of a year in AI so far, and it's only February. The competitive fire for the rest of this year is lit, man. Better AI systems sooner. We're at a potential takeoff event for AI technology with all this leapfrogging. At the end of the day, whether Grok3 is truly the smartest AI or not, it's absolutely pushing the entire industry and that is something to be exc about. Thanks so much for watching folks. Check out my Discord server where we're talking about the latest and greatest at all times, including last night when this announcement was going on. Also check out my X Twitter for my recent thoughts on the AI space and stay tuned for more. Let me know if you want to see me hands on with Garok 3 really putting it through the wringer. See you guys in the next video. Goodbye.