AI Predictions That Will Completely Change Life In 2025

I think 2025 is going to be wild with what we see with AI video.

For anyone who's saying that these models do not reason and they're just like parents or whatever. No.

So the average IQ range for adults is between 90 and 109, with 100 being the theoretical average.

So estimated for 03157. This is like top of the top smartest people in the world level.

Hey, welcome to the Next Wave podcast. I'm Matt Wolf. I'm here with Nathan Lanz. And there has been a lot happening in the AI world, including an announcement that a lot of people think is just this monumental, huge, massive announcement in the fact that we got this O3 model from OpenAI. Well, I guess I can't really say we got it. We got to see it, we got to see the O3 model from OpenAI. But a lot of people have been touting it as this massive breakthrough. I've even seen YouTube videos give it the title of like, we finally have AGI and everybody's saying that O3 is AGI. So in this episode we're going to talk about that a little bit. We're going to talk about our predictions for 2025 and Nathan's going to show off something pretty cool that he's been working on using some of these models. So let's get into some of the, like 2025 predictions.

Being a know it all used to be considered a bad thing, but in business, it's everything. Because right now most businesses only use 20% of their data. Unless you have HubSpot, where data that's buried in emails, call logs and meeting notes become insights that help you grow your business. Because when you know more, you grow more. Visit HubSpot.com to learn more.

You and I obviously talk a lot, so chances are our predictions are going to be fairly close to each other's. But let's go ahead and dive in.

Yeah, let's start on the models, I guess. I think by the end of the year a few things are going to have happened. I think Xai is going to roll out Grok 3 and it's probably in terms of like a fast LLM that you can just chat with for daily use. It's going to be really, really, really good. The fact that they're just, they're trying to training on more data than almost anyone else and they, they've, you know, they're buying more chips from, from Nvidia than almost anyone else. I think it's really good for like daily just like, just Chatting with an AI, it's gonna, I think it's gonna surprise people.

But in terms of replace advanced voice mode for Chat GPT for you?

I don't know, I don't think so. I, I, I, I don't, I'm not, I'm not sure. Like, I, I think for power users, I, I'm pretty convinced ChatGPT and OpenAI are going to be even further ahead by the end of 2025. Like, I think there's a decent chance that because, like O3, if it's just O1 with more compute thrown at it. Well, they've definitely been working on other stuff. They're not just sitting there like, hey, let's take what we've created and throw more compute at it. And like, that's all, that's all we're doing for the year. Like, right. Like, it makes sense to me that like something like Orion, like a larger model underneath as the baseline, will be part of some new model by the end of 2025. And I think there's a decent chance that they actually release it in 2025. Like by the end of 2025. Yeah.

Well, Sam Altman hinted that he believes AGI is coming in 2025. Right.

So, yeah, yeah, I mean, like, like, like I said, like, I, I think 01 Pro is like some kind of basic AGI already. That, that's my definition. People, like, some people think that's like, stupid. Everyone had different definitions of this stuff. But I think, yeah, by the time you get to 04, when not only it can like, respond quickly to like, basic questions and have like, incredibly good answers, but also if you ask it something complicated, it'll give you something back in a minute or two that's amazing. Most people are going to call that AGI, and I think that could be coming by the end of 2025. And that will make all the stuff we talked about, like, in terms of agents possible. Like agents that go off and do market research for you, come back and bring you some charts and show you what, how you should be approaching a market and what's the best strategy that stuff will be agents, how you interact with, do email, like your assistant, that'll, that'll probably be an agent by the end of 2025. And that's going to be a dramatic change that's going to be like, where we're going, we're going to start to have some real questions, societal questions by the end of 2025. And it's going to be interesting to see how that plays out because you know, I have said in the past, like, well, Donald Trump's administration's going to be very pro AI. But also there was that recent thing with the, the port, what was it? The port unions. Unions for the port workers where he kind of backed down from them saying that we need to protect their jobs and we shouldn't automate the ports. And we might see more of that because like there may be such a cultural backlash to some of the job loss that you know, it may become a pretty tense conversation and by the end of 2025, because I do think we'll start to actually see dramatic changes in the job market. Like maybe everyone's not going to fire people, but you're going to see like, oh, I'm not hiring as many engineers anymore. Like there's less job openings, less assistants hired, things like this.

Yeah, I agree. I think, I do think like the big buzzword of 2025 is going to be agents though. I think like every company is going to be talking about AI agents. I think you're be hearing Google, OpenAI, Claude X, you know, you name it, they're all going to be talking about agentic workflows and tool use and being able to like do things on your computer on your behalf. And like I think that's going to be the conversation of 2025. I think if there's any one prediction that I'm fairly certain on, it's that I also think the video models, I don't think we've seen anything yet. I think, you know, eight second videos that look pretty decent. I don't think we've seen anything yet. I think 2025 is going to be wild with what we see with AI video.

I mean, Matt, like let's try to step back for a second. Like, like realize like, you know, we've, we've been talking about this stuff for like over two years now. Like you know, in 2023 the videos looked horrible. I, I, I was sharing the, the, I was sharing threads about them just because they, they looked cool because the technology underneath them. I was like this is amazing that you've asked to AI to dream up what a, a video of this scene. That's incredible. My, my co founder for one of my previous startups, JR Bedard, unfortunately he passed away but he used to be fascinated with this stuff. He would like, he would like read books about like how AI was gonna be able to generate art and he was showing me all this stuff. This was maybe like 10 years ago. I look, oh, that's cool. It can make like a little pattern and stuff.

Yeah, yeah, yeah.

It's like that's where it was 10 years ago. And it's like that was like the seeds of what we have now, but it was like, oh, we can make like a little a cool pattern. And I was like, oh, that's cool. And I was like, do you think that's gonna be like a real thing, like in the next, like five or 10 years? And he was like, you know, maybe in like 10 years. And it's about where it is now. And the videos looked hor. Like they, they were cool if you understood the technology, but they looked horrible two years ago. And now are they perfect? No, but like, man, they already look like like 90, 95% of what you might see in a, in a film, like at least like a short scene or something. Yeah, and. And it feels like too that, you know, I think I may have said this offhand the other day, but I, I believe that where the video models are right now, they're kind of like NGPT4 level in terms of. They're just trained on massive amounts of data, but they're not doing a whole lot in terms of reasoning about that data, about, about that output. And so, so once you start to. And that's why I think like, probably SORA is probably still going to be very good, because if OpenAI is that good with their internal models, as soon as they apply that to video as well, like reasoning on top of it, you're going to get so much more control of the output of these videos. You'll be able to like, yeah, I wanted the character to turn that way. I wanted to do this. And also make sure the characters are consistent because then you can reason upon the output and make sure that it's consistent that that quality is going to go up dramatically.

Well, I mean, we've got AI large language models now that it will basically generate an output, double check its output before sending it to you. Right. You know, when you look at like AI art generation, right? The, the big way of doing AI art early on was GANs, right? Generative adversarial networks, right? Where it would essentially you had a discriminator and a generator, right? And the generator would generate a piece of art and then the discriminator would go, you know, you would give it a prompt like, draw me a cat, right? And the generator would draw a cat and the discriminator would go, that didn't look like a cat. Try again. It would draw another cat. And the discriminator would go, that doesn't.

Look like a cat.

And they would go back and forth and until the discriminator couldn't tell the difference between a real photo of a cat and an image of a cat. Right. We haven't seen that kind of tech inside of video yet. So I think we're going to have something equivalent to like a gan, Right. Or like what we're seeing with like O3, which is kind of the same idea, Right. It does a generation double checks itself, goes, all right, there's things that needs to be fixed. Double checks itself, things that need to be fixed, and. And then finally presents the output once it's confident in the output that it gave you. Right. Very, very similar, conceptually. Well, I think we're gonna get to a point where we do that with video models. You give it a prompt of like a unicorn flying in space with a cat riding on its back, it will generate that. It will watch the video back itself and go, that doesn't look right. Generate it again. That doesn't look right. Generate it again. And eventually it spits out a video that looks exactly what you had in your mind of what a. A unicorn flying through space with a cat riding on its back would look like. Right. Like, I think we are still at the like mid journey v2 level of AI video. Right?

Yeah.

And if you remember, if you followed along to like AI art mid journey from V, I think it was V3 to V4 was this massive leap. Right. Once V4 came out, that's when people started fooling people on Facebook with like the Pope in like a big old puffer jacket and stuff like that. And. And it was this giant leap where all of a sudden people no longer could tell what was AI generated and what wasn't. I think we have not seen that leap with video yet. And I think in 2025 we will see that leap where it gets so good that, you know, people didn't even realize it coming. And the pace at what this happened is so mind blowing to me because, like, you were mentioning very beginning of 2023, we were looking at tools like Model scope and Xeroscope and. And they were generating like two second clips and they were like two ATP resolution, right?

Like, yeah, they were like jittering around and just. Yeah.

And like you can barely tell what they were actually happening on the video. And that was like 23 months ago. And now we're getting what we get out of vo. Like, holy crap. Like, that was a massive leap in two years.

Yeah. And then. And the crazy thing is though, if you look at like public perception, people are like, yeah, Sora sucks.

Like for me it's just like, oh my God. I, I was, you know, one of the like the first people like tweeting a lot about AI video on X and I remember the, the comments of oh, but It'll be like 10 years before you get anything that's even close to like a cinematic shot.

No, no, we're there. We're like you're there. Yeah. If you just need like a two or three second like B roll shot for something, it might take a few RE rolls, but you can get it right now, you know?

Yeah, yeah. And the fact that there's no like kind of reasoning model on top of that, like, like really judging the output and like as soon as you apply that, like these are like you said, it's gonna get way better. Now one thing with all of this is gonna be interesting is the cost. Like okay, it can get way better when you reason about the output. But like how much does that cost?

It's true.

And we've already seen like with like, you know, okay, with, with AGI, they eventually wanna make it where everyone has access to it, right? But it seems, it seems likely that like people who have money are going to get access to that way sooner just because of the cost to run all this. Like, like pro is $200. I think the, the chief financial officer of OpenAI, she said we're open to experimenting with things like $2,000 a month. And I think you probably will see that, right? You'll see like, okay, there'll always be a model that everyone can use. It's like around like 20 bucks a month or whatever. You might have something slightly better, you know, or like, you know, significantly better for like 200 bucks a month. And then if you're professional, you might be like paying two grand to use these.

I mean most of them you'd have free models, right? You can use Claude for free, I believe.

Right?

You can use ChatGPT for free. You can use Grok for free. Now that's brand new. Grok only became free like two weeks ago or something. Like Gemini, I'm pretty sure, yeah, they have Gemini and Gemini Advanced. The Gemini not Advanced is free, right? I think you could even test Gemini 2.0 right now for free inside of their AI studio, right? So there's always going to be like that free plan that's like anybody can access it, but it's not going to quite be the state of the art model. And the more state of the art, the further you go up that sort of tier of how state of the art it is, the more expensive it's going to be.

Yeah, yeah. I mean, so from using 01 Pro, which in my opinion right now is the best by far, it takes a long time to give you answers back. And so something I've been realizing is while you're waiting, like, what do you do? What do you do? Because sometimes it's like taking like three to five minutes and it's like, okay, you know, I go, what do I do? It's like, it's like back in the day when you would code it take something, would take a long time to compile and you're like, okay, what do I do while it's compiling? I go, yeah.

I mean, I still run into that when I'm trying to render my videos, right? It takes 10 minutes to render out a video that I make for YouTube or something. And I'm like, well, right, my GPUs in use right now, so like, I can't really do a whole lot with my computer, so I go walk away and do something else, you know.

Right, well, but in this case you're using their GPU so you can still use your computer.

That's true.

And so, you know, it slightly reminds me of like, so when I was younger, I think I told you this before, like probably 15, I made money playing EverQuest and there was a lot of people who made money playing EverQuest back in the day. I was, as far as I know, I was like a top 10 player on the game. I was like a, I was, I was a well known Necromancer, Dark elf, Necromancer on the game. And one of the strategies we would employ is, you know, like if you went with a party in the game and you were with a bunch of other people, if something good dropped, you had to roll on it. And that kind of, if you were doing it to make money, that sucked. To have to share it with those people and you might get unlucky and you might not get the stuff. So what became very common was to multibox, which was basically where you would be running multiple computers, multiple accounts, and then you get a huge financial benefit from doing so. And, and you would also make alliances with other people who are multi boxing. Right? So like, maybe if it was like, okay, it's hard to run six computers, but like the, if the, you know, if we could put five, five people, maybe one person runs three, one runs two and then that's a better split than six random people that I'm sharing with. And so we would do that and, and you would make a lot more money from doing that. And so I do wonder, like, okay, in this new world, we're. If you can spend more money, you get way better outputs, but you're still tied to like, the one account. Will we see, like, some really clever people start multi accounting AI?

Yeah, yeah. Well, you know, what you're describing here too, is also, and I don't want to get too in the weeds on this, but it also feels like a fairly decent use for like, blockchain tech, right, because you can distribute the compute, right. You can have other people, like, you ask O3 a question and it goes off and it uses somebody else's account and they get sort of tokens but back for allowing to use the account kind of thing. I can see something like that where it's almost like this distributed network of you're using other people's O3 accounts to generate what you want to generate because they're not currently using their account. And whenever their account is inactive, you use it and, you know, you pay some sort of cryptocurrency to use it. And that's their form of mining, you know.

Yeah, that sounds good, conceptually. You know, I've worked with blockchain stuff in the past. It's often harder than it sounds. But, you know, I guess, I guess I'll. I'll ask O3 or O4 for advice on if that's the best way to.

Go, and we'll, yeah, maybe AI will develop that software for us instead of a human actually developing it. Do you think O3 is AGI?

Like, from my definition of AGI, which I guess is like, can do most things humans could do if, if, if you didn't put all this stuff into the models where they tell you that they're AI. I'm not a human. If you didn't, if you didn't have any of that in there, I think you can trick most people into believing that this was a human doing stuff for you. I like, I really do. And I already thought like, 01 Pro was close to that. So, like, based on the benchmarks they're showing for OH three where it's, it's past what the, the, the ARC AGI benchmark, where it's got like a. I think it got like a 87.5 and the average human taking it is gets like an 85. That's for me, pretty close to AGI. Like, I could Call that the beginning, like a first version of AGI to me.

Yeah. Well, I looked it up on a live stream earlier today where we were talking about OH3 and that Arc AGI benchmark. The average human gets about a 76% on it, right?

Oh, really?

And so if we look at the O3 model that just came out, the O3 low model, what do they call it? Like, the. There's like a smaller model that used less training, and then like a larger model, the smaller model got a 76%. So the smaller model hit the average that a normal human would hit. The larger model, the one that used insane amounts of compute, got what, like an 86% or something like that? So smarter than the average human.

Yeah, which is wild. This is it. Looking at complicated logical puzzles and things like that and solving them. For anyone who's saying that these models do not reason and they're just like parrots or whatever. No, there is some re. There is some reasoning going on here. And this is the breakthrough. Like, this is the fact that you now have, like, the large language model, but you've got this computing, you know, test time, compute or whatever they're calling it. Like, that's. That's the big unlock where, like, you combine the two. It's like the two sides of a brain.

Well, I just pulled up this chart. This is from the ArcPrize website here about OpenAI's O3 model. So their low model scored about what an average human would score. But one thing that I find super interesting about this is the cost per task down here. Right. So it's like this low model, you know, this is a logarithmic scale, right? So it goes from $1 to $10 to $100 to a thousand. So if you were to, like, chart this, it would have, like a sort of curving up slope, right?

Yeah.

So this isn't like $11 for 76%. This is like. This is probably something like $30 per task right here, if I had to guess. Right. And if you look at the, oh, three high model, this is 1,000. If there was another line drawn right here on the chart, that would be at 10,000, following the, you know, the same sort of logarithmic scale. So this 88% is actually halfway between a thousand and ten thousand. So in the range of somewhere around $5,000 per task to be able to get that O3 high model actually working. Yeah, very, very, very expensive right now to be able to use this model.

I mean, as. As of right now. I mean, like, my Understanding, like, the amazing thing about this is at least from like, what people are like, kind of, you know, inferring from like tweets from people from OpenAI, is that O3, which they skipped ot O2 because of some trademark stuff, apparently.

Well, there's O2 arena in London, right? That's. There's a big company called O2 that owns that arena licensing. And.

Yeah, so it seems that O3 is literally just O1 with more compute thrown at it. And if that's the case, I mean, yeah, that's what you would expect. The cost would go up dramatically and you get. But it's amazing that you can just get more performance from it, though. Like, you do get more intelligence the more compute you throw at it. Which means that we are now in a world where, like, all the stuff we've talked about in the past, we're like, okay, race for more compute, race for more energy.

Those things are majorly in play now because we've now proven that you probably can get to beyond AGI just with more compute. That's been proven now. And so that's. That's a big revelation of like, okay, we thought that might be the case. It appears to be the case now. It is impressive. Like in this, it shows on. So here's a benchmark in competition math, which is like the kind of math that, like, we have no idea how to do. This is like people who are like PhDs in math who saw incredibly challenged. You know, this is not just like basic algebra and stuff like that. It's. It's, you know, it's frontier math. That GPT4O scored like a 13.4, but like, oh, 3 got a 96.7. So, I mean, this is an incredibly high score. This is like top of the top smartest people in the world level. And I found this fascinating, which this is. I think this is. I'm not sure if this is like an official, like, who's came up with these IQ rankings for the AI, but some people ran some numbers and it was showing that like, you know, GPT4O scored like an estimated 115 IQ, which is considered like a somewhat smart person, not like a super smart, but like a relatively smart person. 1:30 or over is considered a, you know, smart border. You know, maybe 140 is like, plus is genius. Einstein's estimated to be like around 160 IQ.

So the average IQ range for adults is between 90 and 109, with a hundred being the theoretical average. So a hundred is about what you'd expect average to be.

Yeah, yeah, yeah. In America average is about a hundred. So it's, and then so estimated for oh3 157. And like I said, you know, it's estimated. It's you know, iq. It's, it's, it's a highly debated like how useful it is, but you know, Einstein was estimated to be 160. And so this is like pulling in like close to 157. So that, that means we're possibly you know, in the ballpark of where like this will actually be helping us solve frontier math problems, like problems in physics and just basic understanding of the universe that we live in. This may be approaching that like by the time we get to 04, maybe that is already able to do that. Also, this appears to still be not a larger model that they've trained. This still seems to. Oh 1 and oh 3 are not dramatically larger models they've trained. They've just came up with this new paradigm of like applying like test time compute on top of the existing models. So if that's the case, well then maybe what was called Orion or GPT5 or whatever, maybe that's still in the works. And so maybe by the end of next year, not only will we possibly get O3 or maybe it'll be like a slightly less compute intensive O3. So maybe it's like three times better than O1, you know, pro or something.

Just to clarify, what you're saying is like they're still using like a GPT4 level model, but they're just throwing more compute during the sort of inference phase. Right? It's already, it's the same sort of trained model that they trained with maybe GPT4. Maybe it's some sort of like 4.5 or something. Right. Like maybe it's a slightly newer model, but the inference, when you actually give it a prompt and ask a question, it's throwing a ton of compute at it to like process the question and then double check itself and then double check itself and then double check itself until it's fairly certain that it gave like the proper output that you're looking for. Right. But it's still essentially using like a GPT4, maybe a slightly better model.

Yeah, yeah, that's, that's what people believe. And that kind of, that makes sense to me. Honestly, like if they have discovered this new paradigm of like, yeah, not only do you get the result from the LLM, but then you'd like reason on top of that. You could see, you know, it appears you can get like major improvements just from that. And if that's the case, it could be that yes, they do have a, the, the model that they've been training for a while that's on a, you know, has massively more amounts of data that's been trained upon that could still be in the works. And so when Sam Altman's saying that we're going to see huge increases in 2025, that could be what it is that they, they still have the new model coming and then they're just going to slap this new reasoning model on top of that.

It's like GPT5 combined with like the O3 reasoning model. And like all of a sudden O3 gets even further up the benchmarks and does even better and the only thing that changed was the underlying model before the reasoning portion happened.

Yeah, because then its baseline is way higher than it's a reasoning upon.

Right, right, right, right.

So you could in theory see dramatic increases beyond what we're seeing in these benchmarks. I mean, so you, you might see like, okay, yeah, we get like a 200 IQ plus model by the end of next year which obviously dramatically changes things. Because like, you know, I've been testing 01 Pro and people are kind of now saying that maybe 01 Pro is maybe. It's basically in between 01,03. It's like basically like when they were, when they were testing oh one that.

We'Re not allowed to talk about.

Yeah, yeah, yeah, it's. Yeah, it's basically like when you take O1 and put more compute towards it, what happens? It's some first version of that test. Right, right, right. And it seems like that like I think they showed, I forgot where the benchmark was, but I think somebody did show that like there was a major difference between 01 and 01 Pro on that ARC AGI test. And I've been seeing that too. Like when you throw real like if you throw a lot of code at 01Pro and make and tell it to reason about that code and to help re architect that code, it can do it where 01 fails.

Because one of the things that I keep on hearing come up is like it's a huge advancement and it's really cool. But what are the actual practical use cases? And I think the most obvious practical use case is probably coding. Right. I think that's going to be like as far as like a general consumer, the most actionable use case. But I do think there's a lot of other use cases for something like O3 beyond that. They're just not necessarily use cases that the average general consumer is going to care that much about. Right. That they're not going to try to tackle themselves using O3. Things like, you know, trying to solve climate problems or you know, trying to cure diseases or things like that. But I just wanted to hear like from your perspective, like outside of coding, let's, let's just set coding aside. Outside of coding, what do you see like O3 like really helping the world accomplish.

I think that when you start having agents powered by O3 that's when you'll see like real like regular people getting benefits from these reasoning models. I do, I do think that like, okay, if you know, if Orion comes out and it is what's powering like an O4 or whatever, maybe there, maybe there is like a, a standalone non reasoning model too or they have some kind of smart router that knows. Okay, I can respond super fast to this basic query and if it's a hard question, let's spend some time thinking about it. I hope that's coming because right now like the O3 like OH1 Pro, like the average person, if they used it and they're asking it like how to bake something, you know, or whatever, they're asking it like they're going to have a bad experience. Like it's going to be like, yeah.

Oh this is, you can get as good of a response out of GPT4 basically.

Yeah. Or like maybe, maybe O1 Pro would be like slightly better but like it would take so much longer that like the perception would be that it was not good for sure. And so I think that the magic's going to be when it start powering the agents, you know, because then you can like okay, I hate dealing with email. Well, let's just have the agent handle all my email. Like I don't, I don't actually want to do email anymore.

Well that thought just makes me laugh because it's like what happens when everybody's doing that? It's just like nobody's writing or checking their emails anymore. It's just agents doing it all for them.

I think that's where we're heading though. Like you knew it'll be super rare that you get like a handwritten email from somebody. You know, it's like not a handwritten letter but like a handwritten email.

Right, right. Email.

A hand typed. Yeah, hand typed email will be a, a rare thing probably in like two or three years. I think that's where we're headed because like everybody hates email. There's all these things that we do in our daily lives that like, almost everybody hates and there haven't, there haven't been major improvements to these systems. So I think the, the systems will continue to exist, but we'll start to have agents actually do those things for us that we don't want to do. And so I think that'll probably, I think in 2025 we will see O3 come out and it will be what makes agents really work. And that's when people see a change in their, their daily lives once we have robots. And like, reasoning models are going to be fundamental to having robots that can actually do things in your house that you feel safe about.

Yeah. And, and where I think like the, the sort of like, OH1 and O3 are really powerful for agents is the fact that they'll essentially double check themselves. Right. Like going back to the coding thing. Right. It can write code for you and then look at your code and look to see if it made any mistakes and then fix the code and then double check to see if it made any mistakes and then go, okay, I'm pretty sure the code's right now and give it back to you.

Right.

So the fact that it's sort of like constantly double checking its work is one of the biggest sort of jumps that I think this O1 and O3 are creating is that, you know, the reason it's taking so long is it's almost like kind of double checking its work and then triple checking and then quadruple checking and the ones that are using insane amounts of compute. It's like, all right, we just double checked this a hundred times. We're sure this is the right answer, you know?

Yeah. It's like multiple AIs talking to one another and then like finding the best, you know, and double checking everything, make sure it's all accurate. Like it hallucinates way less. Like, I, I've been playing around. I'll show you some later. Like a very basic game. Devo, don't expect this is not a Unreal Engine 5 game or anything like that. It's very, very basic. But I've been playing around with, creating a game demo just because I got so excited when I, when I tested 01 Pro and realized how good it was at like, actually architecting systems with code. But you find that, like, it's amazing at some things, but there's still some things that Claude's good at and there's some things that even, like Gemini 2.0 is good at.

Right.

But I Do I do find the O1 Pro is the best at architecting hard systems, like, actually, like, architecting the code, making major changes or refactors to the code. And in terms of, like, not hallucinating about helping you solve something. Um, I. The. The more and more I tested Claude, like, it'd be very good at some things. Like, some things like, in terms of like, modifying my UI or stuff like, that cloud was the best for some reason, but also it would just hallucinate really weird things. Like, I would, like, show it a bunch of code and say, hey, what do you think's the problem with this? I've got a bug. Here's the bug. And it was like, oh, you need to change this. And I, like, I look at my code and that's already with my code.

It's like, you need to change this to this. And I look at my code and it's like, what you're telling me to change it to is my code.

Yeah, I've seen that in cursor before. I've seen that. Like, because I have my cursor set up for Claude. Was it Sonnet? Right. I have it set up for Sonnet and I've seen that before where it's like, oh, I spotted the problem. You need to change this to this. And I'm like, that's the same thing. You didn't change anything.

I know. And it's like, so that's a crazy hallucination where it's literally like what you just pasted to it. You'd imagine that what you pasted was something else and the way you pasted was the solution.

Yeah, yeah.

And I have not had 01 Pro ever do that. So that's a major change where, like.

You can, like, likely catch it. Like, it might still hallucinate, but then it catches its hallucination before getting back to you. Right. Like, yeah, because there's a lot of stuff that's going on with the O1 model behind the scenes. Right. That they're not showing you the entire thinking. So it's very possible that it is hallucinating, then double checking what it just responded with, noticed it hallucinated, and then fixed itself before responding to you.

Yeah, yeah. So that's gonna make these models way more reliable. Like, anything that needs to be reliable, they're gonna get reliable. Like, because probably the more compute you throw at it, the more reliable it will get. The more times it thinks about the answer before it gives it to you, probably that, you know, maybe it's not gonna be 100%. But we'll get, like, we'll approach, like, 99.9% accuracy with these models probably in the next year with reasoning. So I guess the big question is, like, okay, for 2025, like, with, like, models, like, like, all the models are getting really good. And, like, now Google's like, teasing. They've got, like, a reasoning model and they're. They're kind of making it sound like it's not a big deal. And they're say, even saying that they're probably going to beat OpenAI in terms of coding next year. I do wonder if that's true or if, like, Google's overestimating, like, or underestimating what open. OpenAI has accomplished. So, like, OpenAI is saying, like, it's not as simple as you think. Like, we've discovered some things.

Yeah, yeah. The thing about OpenAI, though, is that I do feel like a lot of the other companies could potentially catch up fairly quickly just due to the amount of brain drain they have at that company. Right. Like, how. How many people are helping develop this stuff and then walking away from OpenAI, like, pretty close to the time they actually announce it. Right? We're seeing it constantly.

Yeah, maybe. But, I mean, I don't know. It's like, that could just be a narrative too, though, because, like, that's always happened in Silicon Valley. Like, talent moves around because, like, if you're at a company for a while, what happens is you get some stock there and the people at other companies just offer you ridiculous amounts of money, and so then it makes sense just to, like, keep your stock and then go get more stock. This has always happened in Silicon Valley. Like, people usually move around every two years. Like, people rarely stay at a company more than two years in Silicon Valley. I do wonder, like, that could be the case. You could be right. But it also feels to me like maybe it's something that's kind of been exaggerated just because, like, it's great for tweets and things like that. And it's just like, more of what's always happened in Silicon Valley. And there's constantly, like, who's to say they're not getting higher quality talent coming in as other talent leaves? Because now they can attract the best of the best. Whereas when they started the company, you know, maybe they, you know, maybe carpathy is amazing, but maybe he's not the best at creating this, despite having a huge Persona online.

Yeah.

And so in Silicon Valley, that's always, like, resulted in, like, issues with founders and people leaving and things like that.

I really think it's going to come down to like three major players. I think really it's going to be Google, Anthropic and OpenAI. And the reason I think Anthropic is in the mix is because they're just, they're getting infinite amounts of cash from Amazon. Right. So it's like you've got, you've got Anthropic, who's essentially owned by Amazon at this point, OpenAI, who's essentially owned by Microsoft at this point, and then Google and DeepMind, which, you know, all under the same umbrella.

I think you're really missing xai.

Oh, yeah, you're right. You're right. I am. I do think they are going to. They are going to. You're right. I am missing one as well. I do think they're going to do a lot, especially with, you know, Elon building like the largest server farm on the planet and whatnot. So.

Yeah, yeah. And it's going to be interesting because it sounds like Elon is thinking he's going to be able to create the largest model. But if OpenAI has discovered that, like, okay, having a bigger model is good and does improve things, but reasoning is actually the more important part. You know, is Elon going to be able to like, kind of slightly, is he going to be able to catch up and figure out how they did it, or does OpenAI really have some kind of secret sauce there that they'll take people time to, to figure out?

I do often underestimate and take for granted X AI because I feel like they're still trying to catch up. Right. Like the current models that we have available to us, which are, you know, grok inside of X and then they have their new, like, Aurora image model.

Yeah.

Neither of, like their large language model or their image model feel as good as what else is available. But also I'm not counting them out because I know how much. Obviously Elon, I believe, is still the richest person on the planet, if last time I checked. Right. He's got $6 billion in investment. He's building like the largest GPU cluster on the planet. Right. We're going to see them make some waves and, and forgot to mention, he's best friends with Donald Trump at this point. So, you know, he's got government inroads as well. So I think, yeah, I do think we are going to see a lot more out of xai. It's just I always forget about them because I don't personally use them on a daily basis, you know, Right, Yeah.

They don't have the consumer adoption yet. But, like, people have shown that, like, that has been going up. I think I saw a chart showing that it was going up a lot in Japan, which makes sense because then Twitter X is huge in Japan. But, like, I saw the, like, searches for Grok as well were, like, going up dramatically in Japan. I think Elon Musk shared that is like. Like a. Like, almost like it was like a global trend.

Yeah, yeah.

You know, he. He showed that chart. People were like, that's Japan. But, you know, but Japan's very important still, like a top three market. And so I am curious to see how that plays out. But, yeah, I think we have at least four major players going after it, and I think all of them are going to make major progress in 2025, so. Which will make all of them make progress faster. They're all, you know, people have said, like, oh, OpenAI had to show O3 on the final day of the 12 days of OpenAI because Google has been so freaking impressive with their announcements over the last month.

Do you think that they had something else intended for the 12th day and they pivoted to show off O3 because of Google, like, sort of matching them day for day with something equally as big, if not bigger?

I think it's. I think it's possible because, you know, it was also. You started seeing, you know, we've had Logan on the podcast before, and he was starting to, like, tweet out some stuff, you know.

Yeah.

Kind of. Kind of hinting that, like, Google's not behind. Maybe they're. Maybe they're ahead. And. And then, like, when O3 was. The benchmarks were shown for OM3, like, all these OpenAI people started sharing memes. Like, you know, OpenAI was never behind. You thought they were behind. Everybody's saying they're behind. What are we behind? Yeah, yeah. You know, like the. The most used model and with, like, the stuff they have internally still the best.

Well, they needed something like that to really sort of get people excited again. Because I feel like during the 12 days, there was a couple cool announcements, but most of them were kind of meh. Like most people. It doesn't impact them too much. Like, I thought the 1-800-chat-GPT was sort of like a gimmicky novelty sort of thing.

We got Santa voice.

Yeah, the Santa voice. I mean, I think the big moves they made were putting out Sora, but then Veo came out, like, two days later and sort of knocked Sora off the top of like being the best AI video tool at the moment. Sora Turbo. Right. The other Sora might still be a lot better, I don't know. But then they also announced the Vision inside of the advanced voice mode. And I think that was probably the most impactful thing that they did was the advanced voice mode. With Vision, we also got the 200amonth plan, which gave us access to, you know, the GPT or the O1 Pro.

Yeah, I mean, for me, O1 Pro is the biggest one. But like, like I said, like most people, number one, they're not gonna pay $200 a month. And number two, even if they did, you can see that everybody's getting very different results. Like it depends on what you're trying to use this for. Hard coding stuff or science people are blown away if they're trying to use it. Like they use just chatgpt on a regular daily basis. They're like, this sucks. Yeah, yeah, yeah, yeah, yeah.

But I'm curious, I want to see this, this game that you've been talking about.

So even super, it's super simple, so don't expect much. But I'm not a game designer at all.

I'm expecting Baldur's Gate 4 meets.

Yeah, yeah, yeah.

Elden Ring. Baldur's Gate meets Elden Ring. Let's see.

No, it's not that.

But You've been using 01 to develop this or 01 Pro.

I've been using the 01 Pro. I've been using Claude. I've been using Gemini 2.0. I've been using Sora.

Are you using an IDE, like a cursor or visual?

I'm using cursor, but I, I, I'm, I feel like I'm kind of a noob at using cursors, to be quite honest. Now that they added, you can put 01 into cursor. I do want to try it again because that'd probably be better than the outputs I was getting from Claude. I feel like right now 01's like slightly better than Sonnet. O1 Pro is significantly better, but way slower. But I've been using kind of like everything. That's why I was saying about the whole multi boxing thing is like. Yeah, yeah, multi accounting. Like, I think like the, like the wizards of AI, they are going to be like putting together all these different tools and maybe multiple LLMs based on what they're good at.

It's happening already. I think you're going to see like, you know, Chat GPT, give you an output and then it spits it over to like Gemini to double check it and then it spits it to Claude to double check it. And it sort of goes round robin through all these various LLMs and then spits back out like a sort of aggregated response based on what all the LLMs said, you know.

So this is very basic, so don't judge me. Okay, I was reading now this is all super ugly, but like you train a character, you go into like a little town which is like all Mid Journey generated.

It looks sick, dude. It does look really good.

And dude, I've had Mid Journey. I use Mid Journey for this. The special effects I asked Open 001 Pro what to. It's just like, okay, let's put some snow here. Okay, cool, let's do that.

So it's like a static image, but then like some coated in snow over the course.

You would make it look better. Like it was kind of like. There's this one game called Deepest Dungeon where they do something like this where it's like you have like a static image of a town and like when you highlight over them, like they kind of like they light up and stuff like that. So I'll eventually do some kind of special effects where when you go over it, it lights up. You know, here I go build my party. I have not worked on the ui, so all this is just like super ugly. But like, okay, I recruit my team, you know, random, you know.

So how many hours do you have into this right now?

Not. I mean, like I've been. It's a decent amount, but not, not a crazy amount. Like I have a hard time estimating because it's been the kind of thing where I'll do it while I'm doing other stuff. Yeah, like I'll. I'll have Owen Pro running and I'll just be totally doing other stuff. And I'll be like, hey, improve this how? Or not only like improve this, but sometimes it's like, what should we improve? So like, yeah, so you, you know, here'd be some kind of like world map. You know, it's all not that great for now, but like you go in kind of Mario style. We have different nodes that you go through to get to the final boss. And then what I've kind of done gameplay wise is I was like, well, what's something simple I could actually implement without like making, you know, Baldur's Gate 3, which is super complicated.

Yeah, yeah.

And I was like, well, I used to play this game called Puzzle Dragon. Actually Taizo san's game where it's like, basically like a Match three game with an RPG element. But I always felt like, oh, but it doesn't have, like, like real RPG elements in it. Like, they're very, very light. I was like, what if you made it just like. Like in between Baldur's Gate, like, more RPG elements and like a basic puzzle Match three that everyone understands and some people love, people hate. And so I just created something like that. And you have, like. You basically have, like, a party here. Like your party with, like, health bars, actions.

Really good. All this is Mid Journey art.

This is all Mid Journey. Yeah, I used. I've started to use scenario a little bit, but this is mostly Mid Journey and, you know, and it's just like very basic match game where you charge stuff up. You got different abilities. I haven't worked on the special effects at all. I literally just asked 01 Pro, like, put some special effects in there.

It looks really good. It's like a.

Like a special effects. It just came up with. I didn't. I didn't do any of that yet.

It's almost like a candy crush kind of concept in the actual, like, puzzle game almost. But then.

Yeah, yeah, you got a whole bunch.

Of, like, fantasy RPG elements also.

Yeah, yeah. I'm not gonna display the entire game logic.

You're like, I'm playing now. I'm doing good.

Yeah. This is my work now. Just. Just play games, you know, and then you, like, have different choices for going to different rooms. I probably would have the AI. I probably, like, actually, like, generate the. The dungeon, so it's not the same every time.

Yeah, yeah.

And then you have different, you know, different stuff you buy. And so to be honest with you, I've been kind of surprised how good it is.

It looks really good. I mean, everything I just saw looked pretty solid.

Yeah, dude, like, considering, like, I've literally. That's like 90% 01 Pro with little bits of Gemini 2.0 and Claude for different uses.

Well, you lowered my expectations where you were like, yeah, don't expect too much. And then what you showed me is better than I expected.

So I think it's been kind of good for me to do this too, because, you know, I think I told you off camera, like, I've been kind of decide, like, what else should I work on beyond the podcast, like, should I be building another startup or should I be doing a YouTube? And I kind of decided, like, well, I think I just want to be playing more with these AI tools. I think it'll Be more useful for the podcast for me to actually have hands on experience. Maybe, maybe these things turn into real projects, maybe they don't, but they're like.

I can see it spinning off into a real like game studio and you actually selling them.

It's been really useful for me though, like to actually understand how good, like what, what are these things good at currently? What are the limitations?

What would be really cool is if a tool like a cursor sort of knew which, what each model was good at and they would send whatever your prompt is to the best model for that specific thing.

Or you could create your own logic. That'd be kind of cool too. Like, okay, yeah, yeah, there's suggestions but like you can kind of like find. You can like, yeah, when I'm doing ui, I want to use this. I want to.

Yeah, I, I can see cursor building in something like that where it, it kind of, it's already model agnostic anyway. You just pick which model you want to use. It doesn't seem like too hard of an extra step to basically say, if the question is related to this, send it to this model. If it's related to this, send it to this model.

Yeah, so this, you know, doing this like little test, like demo project, like made me realize like, you know, maybe a year or two ago I was saying, oh, you're going to see like one or two people. Startups that hasn't super happened yet. But I feel like after like actually trying the newest stuff and like realizing how much better it's about to get, I think we probably are going to get there where you're going to see like one or two people, maybe five, you know, like a small group of people creating massive projects. Stuff that used to would have taken, you know, a hundred people, thousand people. You're going to see small teams of people with the concentrated focus being able to crank out amazing projects. And the fascinating thing is too is they may be able to do multiple projects as well. Like I've been finding, like, I can totally work on other stuff while I'm doing this because it's more like, it's almost more like I'm like the director.

Yeah, yeah, yeah, yeah. You're basically going and telling your team members, all right, we need to create this next. And they go and do it and you're just steering the ship. You're not doing the things, you're not fun.

And I think most people don't know how to tie this stuff together as well as I do. So I think that's going to get easier and more people will know how to do that.

I totally agree. I mean we were talking about this before hitting record. I think a lot of SaaS companies are going to be in trouble. I think it's, you know, a lot of companies are going to be able to just like build tailor made tools for their needs without needing to go and you know, pay for a SaaS company to do the thing for them. Right. I'm already finding myself doing that when I have like a need. Like I was talking on a previous episode of like, I want something that no matter what file type I put into, converts it to a jpeg. So if I throw in a png, converts to a jpeg, a webp jpeg, a file jpeg, I don't care what file type it is. If I throw it, even if I throw in a jpeg, I want it to output it as a JPEG in the selected folder so that I can bulk throw images in there and convert them to JPEGs. I guarantee there's already tools out there that do it. But it took me three minutes to build that using cursor and Claude. So I just built it real quick. And you know, I don't have to go and use some either insanely ad ridden website for it or pay for some SaaS to do it. I just have a tool that does it on my desktop now and it works really, really well. And I think you're going to see more and more and more of that where people are just like, I need this problem solved. I'm going to go and just code up a solution real quick because all it is is a prompt and the prompt will code up a solution, you know.

Yeah, it'll get faster and faster too. Right? Like eventually this will be almost instant where within a second you've got that output of what you wanted, a tool that does what you want. I think in the O3 mini demonstration they showed where like it basically set up its own server and it set up all this different stuff and it's like it set up its own server and created this, came up with a benchmark for itself and it set up a server to run that.

I built a benchmark. Look, I'm really good because I beat my own benchmark that I just built.

Fantastic. But still it is fascinating that it's able to do these kind of things. Right. It's mind blowing.

Yeah, for sure.

Yeah.

I mean, I think it's exciting times right now. I think 2025 is going to be even more exciting. So a lot of good stuff. I think this is probably a good spot to wrap this one up, our first episode of the year. Thank you so much for tuning in for our show in 2024, and hopefully you'll be with us for 2025. Make sure you subscribe wherever you listen to podcasts. We prefer YouTube because we try to make this visual and show stuff on our screen and show off what we're talking about a little bit. But if you prefer audio, you can also find us wherever you listen to podcast. So thanks so much. Happy New Year to everybody, and we'll see you in the next one.

Yeah, Happy New Year's.