Grok 3 vs GPT-o1: Which Is Actually Better? (We Tested It)

Hey everyone, we are talking the brand new Grock 3 or what some people are calling the smartest AI model on the Internet. It dropped last night. Recording this on Tuesday morning, dropped Monday night. And in the core arena testing it is the first model ever to score over 1400 on the chatbot arena and outperforms the best reasoning models from OpenAI and Google. So the very baseline is Grok 3 is a very smart model. And Kieran, one of the things that blew me away about Grok being a smart model and everything, that's cool. But you're like, well how is it so much smarter? And one of the reasons is that they have put together 200,000 GPUs in Memphis, Tennessee to run Grok 3. And what's crazy is they went from nothing to 200,000 GPUs in 214 days.

It's a very Elon Musk.

Yeah. I saw a separate tweet, Karen, that they have leased 25% of all temporary cooling in America to run this center.

Like one of the things you and I are actually using deep research for is to learn call option trading. And I was actually thinking I should trade Nvidia of this being like you basically you gamble. If it was bad news, they get tanked. If it was good news, it shows.

That Nvidia is already up above 140 in pre market. You lost the trade already. You were right. What's crazy though is that yes, this is a very pro Nvidia 200,000 GPUs in Memphis, Tennessee using a ton amount of cooling. That's not really what people care about. It's interesting, but it's not the core thing. The core thing people are caring about is like how is the model? What's it doing? One of the things I did really like though, before we get into that, is this post from nikita. Game over. Grok1, if you're an investor opening I, you could report your tax loss to your accountant this year.

The thing that I would have thought originally would Grok is they were predominantly going to go and just try to beat OpenAI and consumer use cases because they have Twitter. But this really caught my eye. Right.

Oh yes.

Palantir has brought Gro to the enterprise now officially available. So I, I do think that this is problematic for OpenAI because you've got a pretty incredible competitor in Elon. You've got a team that has been there for just a year. And one of the things that I thought was one of the people I really listened to. So like if Something happens in AI, there's a ton of noise. The person I go to like figure out what they say because that's my gospel is Andres Karpathy. But he basically gave his summary and he said, you know, vibe check. Have you seen this new thing all over Twitter? Vibe code in everything is a vibe. I do like this.

I think this is just really technical people trying to be cool.

Yeah. So he basically thinks it's somewhere around the state of art territory of OpenAI strongest model. So that's O1 Pro, $200 a month, which you and I use.

Yeah.

Slightly better than Deep seqs. Only slightly better. Is interesting. Slightly better than this is, I think is amazing. Slightly better than Gemini 2.0. Flash thinking, which considering this team started a year ago, is not good for Google, not great for OpenAI. And I do think they have a new competitor who's not only shown that he can compete on anything. That really is a paradigm shift. I do think in these kind of things these benchmarks kind of get lost to me because the proof is in the use cases that you want to use it for. And so when I over when I see these benchmarks, it's starting to just be like this model is better in this benchmark. I know they're really important because I shared this. I think this is like pretty wild actually. But this is a pretty great way to think about AI. So AI is like rapidly approaching expert human level expertise. Right. So 01 models were the first time they started to breach that. I think this is 01 Pro. But all of these models are pulling each other up. You have Google's Gemini 2.0, which I think is down here. You have these newer models kind of pushing themselves up here. Other could be Deep seq. But what's happening is this year the human intelligence is a solid problem in that GROK is another evolution here. So it's probably again in the. What we're showing for our listeners is a chart that shows the progress of AI models against expert human level education or expertise. And since July 2023 to Jan 2025, we have rapidly been approaching not just expert level, expert human expertise, but actually we've surpassed it with some of the O models. And Grok is there as well. So what's actually incredible to think about is this year we've solved the problem of being able to give everyone human intelligence for pennies, if not for free, because Grok will have a free version. OpenAI has a free version. Deepseek is free. All of these different models are Free and I think that's a wild thing.

Should we want to look at some software finally?

Let's do it. Let's look at some use case.

Let me just pull up Grok. So what's interesting first thing we talked about, how smart the models are, all that kind of stuff. The first thing that strikes me, Kieran, is that Elon and Grok have done what Sam said he is going to do in the future at OpenAI is that there's just one. So you have, you have these two models right here. You have the old one Grok 2 or the brand new one Grok 3 and you don't have to pick other than that. And deep research and reasoning are built in. So it's not like you have a reasoning version of Grok 3 and a non reasoning version and a deep research version. You can just toggle everything pretty easily here. Am I doing deep research or do I need it to think critically? Right, so it's a, the user experience is a little simpler and more consumer friendly here than I think on some of the other models today.

It's, it looks a little like deep seq.

Look, if you've been using AI for your marketing but still haven't fully figured it out, we've got the playbook that is going to change the way you market this Playbook cuts through all the generic noise, gets down to the nitty gritty proven tactics that actually drive results. These strategies come straight from us over at HubSpot and the leaders over at A16Z and Asana. You can steal our playbook at the link in the description below. Grab it right now. Now let's get back to today's show and so now let's, let's look at a couple examples. So first I wanted to do, I wanted to do a side by side on deep research because that's a use case you and I use a lot. So I just wanted to do something that like I thought would be a random consumer use case because that's kind of what we're talking about and I just wanted to do deep research on the health benefits of red light therapy. So here is Grok3 with deep research and what's interesting is it didn't need to ask me a ton of clarifying questions like I have to do often with ChatGPT. It shows me here, Kieran, this entire chain of thought around like the health benefits of red light therapy. I wanted to do something like very basic, right? I didn't give it this like very detailed academic prompt to start and it shows you, shows you what you're researching. It then looks at different aspects, exploring cellular effects, for example, gives you pretty good breakdown of what it's doing. And then after that, you know, it tells you it's safe, blah, blah, blah. Then it gives you this full summary of benefits, safety, usage, citations. It's good for wound healing, hair growth. What I liked is that it put a table together, Kieran, of the condition you might have the benefit and whether there's like real evidence or not, right? Like I found this incredibly helpful. I thought this was really good. And then it also then gives you citations if you want to go deep dive. And then obviously these citations are throughout the research, but also here at the end. As somebody who is just a, I think a top 1% ChatGPT deep research user, what are your, what are your initial thoughts?

I like that it's showing you exactly what it's doing.

You have researching red light therapy, exploring cellular effects, investigating side effects, evaluating health benefits, assessing safety concerns. And then it's done.

Like, I think it looks cool. I think what this is kind of my point with some of this until you really get into trying to use it for your own stuff, it's really hard to like differentiate between some of these models, which is like part of the problem is becoming part of the problem, right? Because you have so many options now. Who had known we would have so many options to have incredible AI intelligence?

So one of the things, Kieran, that we've done previously with 01 and O3 is we've used it as like a YouTube growth coach, right? And so if folks didn't see the show, we took basically all just the raw data, just copy and paste it out of a CSV and asked it to give us basically strategy and growth advice to get to 100,000 subscribers, right? And it did a really good job, right? It said identify high performing video topics, optimize your hook, your thumbnails, scale distribution and promotion, developed ongoing series, improve your retention, smart things that you'd want to do, right? And so the question is, how is GROK compared to those results? So I did two versions of this in grok, Karen. I took that same exact prompt, same exact data, and I put it into grok. And this version I'm showing first is the non thinking version. So I did not click the thinking icon. Okay, so less reasoning. And it's like, and it's not that great. It's like understand your content performance, which is fine and does a good job mathematically of breaking down like where our Videos stand, define a growth strategy, detailed action plan. Like this is very like basic, you know, kind of Claude 3.5 or worse results. And so then I was like, okay, well what happens if I do the reasoning version? So it thought about it for 57 seconds and then it basically gave me the same version. Not the same. I'm interested to see what you thought. It's like focus on high, high performing content themes, optimize titles and and thumbnails. Leverage short form content for viral potential which I thought was interesting. Picked up on like this one outlier, this one viral short we had from Cody Sanchez. Subscriber rate, publishing schedule, collaborate with influencers and creators. Called out Cody Sanchez and a few folks. Maximize watch time. Like it was good. Like it's on par with 01, right?

I think so. Okay. I actually don't think it is. This is, this would be one place that I would like to dig into. Hopefully I'll get access to Grok 3 tonight because I actually realized that I'm on premium, not premium plus and so I'll upgrade after the show. But the, the thing I would say is the O1 Pro model is really great at strategy.

It is.

And so we have a episode that will be live after this from Meyer Kraken where we did, we walked him through some things live and on just first look because I'm in, you know, in Europe so I'm having to like get access through a bunch of like Secret Ways to Grok 3 so I'll have access for the next show. But just on first looks in terms of what you showed me, that is a very vanilla strategy. And what I found one models to be a little bit better at is like really thinking through the problem and really thinking through the strategy. But I think what all of this proves is without context they're all like fine. But when you actually start to give them internal context, they're amazing. So let me, let me walk you through a real use case that I was working on this morning is I have four to five years of data and there's a, there's a problem that I'm working on in HubSpot. You know this problem, it's a low end of the market go to market problem. I have five years of data, research, numbers, strategic docs, results, goals, hypothesis, what we proved, what we didn't prove. And I can give that all to an A1 model in your case a Grok model. And it is incredible to be as a thought partner, incredible. Not not asking it to say, not to give me Solutions. But I'm going back and saying actually pull that line back out that you told like, I'm like, What are the 10 most interesting trends? What are the five most important problems and why do you think they're problems? Go back and give me that problem again. Actually give me like three reasons you believe that's a problem. It's like a really smart thought partner. And it was just another unlock for me. I was like, shit. I spent three hours this morning working with it like a colleague. I was just jamming, going back and forth, riffing and its ability to pull out the right line from a multitude of different docs and like us to go riff on that line. And so my point here is like all these models are going to be much more incredible when you give it context and so much more. The enterprise use case is really important. And so I do want. That's why I was really interested in the fact that Grok3 has enterprise ambitions. Because I think that's a place that those models really need to. They will make real impact.

Oh, 100%. That's what's going to happen. Right? Like this is. We're talking about GROK today. But like what you just described as the single biggest underused AI use case in the world right now, I think is to use advanced reasoning models to work with you to think through hard problems much faster and in different ways. Right.

You can basically ask it to steel man an argument. You can ask it to give you the Bullen bull bear case of that argument. What I ask it to do is like poke holes in my thinking here, which I need to do a better job of at unmasked. Like one of my things is I run ahead to solve things and don't spend time trying to like figure out. Try to poke holes in my own argument. It's great for that, right? It's great for those things. And so I think it's. If you're not having, if you're in a strategic role and you're not having a multitude of different back and forths with these recent models day in, day out, you are behind the curve of where you need to be in AI usage.

If anybody asks me for feedback on something, I have a format now. I give them my feedback and I tell them, I also give them the feedback from the model I think is most appropriate for that problem. And I tell them what I agree with or disagree with that the model said, that's pretty cool.

That's actually Right.

And so that way it's not me because I Need to expose that. The fact that, like, they should have used the model and not asked me.

I love that.

Right. And so, like, that is how I give feedback on everything now.

Yeah, I agree. There's no. No, no, no right or wrong way. Oh, I've just seen Grok 3's and cursor coming to cursor as well. Cursor is my new thing that I'm trying to learn.

I was going to say, are you going to be doing a cursor show for us?

I'm doing it. Well, I'm doing. I'm going to start a cursor course with my. One of the things I love about AI is it just. Again, I come back to it. It's just unlocking people's ambition. And so I have three brothers and one of them is a consultant. I won't give away the company he works at. He's very technical, actually. He's doing dope shit in AI, so we can actually do dope shit together, like take courses together. So there's just. There's this like, unlocking of not ambition, but like ideas. Right. Because the barrier to ideas have been broken away at and it's unlocking this in people. He's not the only example. I have people on my WhatsApp pinging me about things they're doing. I'm like, geez, that is so cool. All right. Yeah, cool. Did you. In the market, you. You ran the marketing prompt that I gave you?

Yeah, you. So tell people about the prompt. And then did it.

Did it give you a. Did you. Just one thing, Kip. Did you add the image? Because you kind of have to add the image.

I uploaded the. I uploaded the prompt and text and I uploaded the whole Google Doc with the image as a PDF.

So I have a bunch of Marklin Reason and docs, and so they're all usually based upon synthetic data. Now internally at HubSpot, I can use my internal data, which when it gets a lot better. So this one here is. It takes a bunch of synthetic data about previous marketing campaigns and then it digs in, like about a hypothetical company that I give it, and then it digs into what it can learn about those campaigns, what you should do in the future, and, you know, builds a whole campaign plan. And so when I did this in Deep Seq, it was incredible, right. It actually built me at a marketing annual plan for the channels based upon this data in a way that I think it would have taken a real marketer probably a good month to crunch that data. Distill all of that out and come up with this annual plan. And Deep Seek did it in a small amount of time. And so I have some reasoning prompts or like tasks that are specific to marketing that I run through these models and so that, that's what this one is.

Here's what he gave you. So it came up with a lead generation campaign to private clinics. This is a doctor's software company.

It sells doctor software and software to doctors.

It gave gives you a performance analysis, external features. So it got the seasonality side of it.

It got the seasonality yet. So I specifically use doctors because of seasonality. And so it got, it got that, it got the different Personas. It looks pretty good actually.

I like the structure. Actually I will say one of the things I have found in my early usage of GROK is it's very good at structuring structuring without having to ask you a bunch of questions about how you want it structured. The two, the two things I would say I like most about it are that and that it is freaking fast.

Yeah, I heard that you can tell.

There'S 200,000 GPUs because it is fast, man.

It's very similar to Deep seq which make me, does make me think that these reasonable models are very similar. That they're very similar capabilities and they're actually given a very. Yeah, so it's got the here, see here the external factors held in May potentially coinciding with major healthcare events. It got all the right trends because I actually injected things that would make it need to pick up on that blog. All the same things. Yeah, so it's very similar to Deep se. I put. Ooh, what's this part?

Patterns and trends.

Yeah, yeah, it's got, it's going to pull out the actual, the insights from the data I give it to tell us how to do better. This is the thing that you should really do. And when you do research products for your internal data, you and I talk about this. Everything I pair with external trends. And so this one is a good example where it's picked out conferences, it's picked out seasonal factors, it's picked out industry shifts. Actually this is a good one. Changes in healthcare policies. Oh, it's going to keep going on actionable recommendations. This is pretty cool. Actually gives you pretty good advice. Like if you're a marketer it gives you like again it's great thought partner. And So I think Grok3 is going to be very similar to how I feel about 01 Pro.

The use cases straight out of the way for me with Groq, if I was an individual AI user, would be faster. Repetitive reasoning tasks. Right. Like sometimes O3 high takes like 10, 12 minutes to reason something.

Right.

And does a nice job. But it's not fast where, like, this is like a very good quality output for. Let's go back up here. It reasoned for seconds, if I remember correctly, thought for 29 seconds to do this. That's lightning fast compared to other models I've been working with. Right.

The other thing I'm going to try to. When I get access tonight is I want to try it out with writing use cases because the other thing it has access to, which I don't really know, I haven't. I haven't really listened enough of. I'm going to listen into the webinar they give. I don't know if they're using their proprietary data because the thing that.

The Twitter data.

Yeah. The thing that Elon has that Sam doesn't is he has that in Telsa, he has it in Twitter, he has it in SpaceX. Like he actually has data sources that you can't get access to.

He has built the most vertically integrated system ever where he just controls everything and doesn't have to depend on anybody to be successful.

I really want to try out my writing use cases because Twitter has incredible writers and they have access to. It's like a lot of creativity in that platform. I wonder they've injected that any. Any of that into the model.

Yeah. Today's show was an overview of our initial reactions, what was happening with Grok. But as we use it more, we're going to probably come back with a few other shows of like, very specific use cases we find to be very good. And what we're saying now is that Grok is really good. If you are not a detailed prompter, GROK is pretty good because it's good at providing structure. And if you're in a hurry, Grok is fast. Right? Right. Like out of the box for any use case. Those would be two things that if I cared a lot about, I would be probably leaning towards GROK compared to some of the other models right now. Right.

It's cheaper.

It's cheaper, cheaper, faster. It's always a good start for product to disrupt. Right. And so, Kieran, before we close out today's show, anything else you want to add on Grok 3?

No. It's a model that I need to go start playing around with, get my European access through some sneaky way and have a good time.

Eu, baby.

Yeah, eu. We'll get there. I've got like most of the things I want now actually.

Okay, that's pretty impressive. Okay, so that was Grok 3. I hope you all enjoyed it. We encourage you to go and try it. I do think Kieran to close out. One of the things is it seems like they're going to go to the enterprise through API and not through like these enterprise seats. So it is going to be a more consumer kind of seat. End user product versus API product. And to be fair, OpenAI their reasoning models are really only through the API now too. So reasoning in companies is still very limited to the API. One of the things that are holding us back as kind of the business use cases are trying to come about.

And it's a huge unlock for enterprise use cases. The difference is orders of magnitude. Difference in me being able to work through problems as a thought partner with AI.

All right, that is Grok 3. We hope everybody has a great week. We'll see you real soon. On marketing against the grain. This data is wrong every freaking time. Have you heard of HubSpot? HubSpot is a CRM platform where everything is fully integrated. Whoa. I can see the client's whole history. Calls, support tickets, emails. And here's a task from three days ago. I totally missed HubSpot grow better.