Industry Roundup #3: The Rise of Reasoning LLMs, OpenAI Operator, Project Stargate, and Gemini’s Struggle for Recognition

Hi, I'm Maggie, VP of Curriculum at datacamp. And I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com.

All right, all right, all right. Richie Cotton, how are you?

Hey, Dale. Life is good. I'm feeling excited. So much stuff to talk about. It feels like it's been a while since we did one of these industry roundups. We could probably talk for hours on what's happened in the last month.

Yeah, indeed. AI goes in light year speed, so month and a half not covering news means we have like six hours of content for you. But we're going to try to condense it to our favorite story. And we have three stories for you today. We're gonna cover them. I'll go first. Right. The first one is gonna be on the rise of reasoning LLMs and the reasoning LLM wars that we've been seeing over the past couple of weeks. So, Richie, have you seen Deep Seek? Have you. Did you manage to catch up on that?

It's been impossible to miss all the talk of Deep Seek's R1 model. It's just been absolutely everywhere. But it's also making you YouTube famous. I saw your video went viral.

Yeah, we actually had a YouTube video. Everyone do check it out on our YouTube channel. I fine tune Deepseek on a medical data set. And yeah, that blew up. I was surprised. But that said, let's maybe go a bit give you a bit of a background on Deepseek R1. What's been going on in the ecosystem there? So deep seq r1 has been wildly hyped. If you've been on any social media in our space, you probably have already seen what Deepseek is. But in a nutshell, it's a reasoning model developed by the Chinese company called Deepseek, which rivals 01 in terms of performance. But it's much cheaper, apparently. Much, much cheaper. Deepsea claims that it costs around 5.5 to $6 million to train. And it's so much cheaper to use the deep seq R1 API. I'm talking like fractions of the price of the O1 API. So we're talking $0.55 per 1 million non cashed input tokens and 2.19 dollars per 1 million output tokens. Compare that to A1 which is in the $15 and $60 range for output tokens. So really changes the game and the economics of the reasoning LLM space. So a lot of people have had takes, you know, is Deep SEQ lying? Is it actually legit? What does this mean for LLMs? Actually we don't know. So I'm not gonna pretend like I have a take. But what this means is like more competition. So OpenAI reacted brought forward the release of the O3 mini model. It's accessible on ChatGPT today. They have the O3 mini and the O3 mini high model which is actually better. The O3 mini high model was actually quite impressive. It understands for example physical simulations better. It created a code I saw someone run a tutorial on it created code for a Snake game. Then that person asked prompt it to create a reinforcement learning algorithm that plays the Snake game and was able to do it in one shot. So that's actually pretty impressive. Right. And then OpenAI also released deep Research. So Sam Altman on X mentioned that they're gonna pull up a few releases, push them forward. Right. So Deep Research seems to be one of those releases is essentially a tool for multi step reasoning, research and report creation. Also steals the name of the Google Deep Research product. Like that's also. I don't assume it was intentional, but it is a Deep Research product. Right. And then here use cases include stuff like shopping for houses, comparing cars, analyzing business trends, so on and so forth and then under the radar. It's unfortunate that it's under the radar because I think Google actually has really great models announced the Gemini 2.0 flash thinking experimental model as well as the Flash Thinking experimental with apps model. Really short names for these Google models and it definitely is contributing to I think why everyone sleeps on Google in this space. So maybe to kind of set the stage here, I'll first ask you a couple questions, Richie. My first one is actually I'm frustrated because Google really has great models. Their Deep Research model, I've been using it for a long time now. But if you look at the degree of interest online, you look at Google trends, right? Why does Google not get the hype it deserves?

Yeah, I mean you're right that a load of the innovations around large language models, they've come from DeepMind and they're being incorporated into all the other models. I've got to say it's a product thing and it's a marketing thing. Like something like ChatGPT just works. The Google models. It's like, well, how do you even go about getting them? It's like it's stuff that's thrown at you in a Google query or you actually have to go and look, look for it. So I think Google's product strategy's not really come together, but also the marketing strategy, I mean like the naming, was it like Gemini 2.0 flash, blah blah, blah. I can't even remember the full name and you just told me it.

Flash thinking experimental with apps.

Exactly. Yeah. It doesn't roll off the tongue very nicely. Now I've got to say, OpenAI's naming strategy is much better. Like O3 mini high. It's like the MIDI one means it's small and cheap and not very good, but high means it is good. And I don't know what to think about that. So yeah, naming things is really, really hard. Like this is a long known problem in software development and products. Like just getting good name things is hard. But yeah, the Google marketing team really need to think a little bit better about like corner things.

Yeah. What's unfortunate is that they're amazing models. Like I have to say, like Gemini 2.0 flash thinking experimental with apps or experimental is like blazing fast. Really good performance as well. I've tried them, like played around with some of the use cases. Probably faster than Deep Seq and maybe equally good if not better. And I haven't seen the benchmarks yet, but really, really great models. And I really think this goes back to something that we've discussed on the podcast before, like this Vibes based kind of approach to what is the favorite model. Right. And it seems like the Vibes have not caught up with Google yet. And one thing that I think hurts Google here as well, that their discoverability is pretty hard. Right. Like it was pretty hard to go on Gemini. Understand which of the models are new, which of the models are not. Right. And I think there is maybe room for simplification, maybe the inclusion of Gemini and Google Search. Right. Like you have the most visited website on the planet, make use of it if you want adoption on Gemini.

There's a certain irony there that Google is designed to help you find things and if they're having a discoverability issue. Yeah, it's not ideal.

Indeed, indeed. And I do have to say though, Deep research from Google is an incredible product as well. I use it almost all of the time and yeah, which goes to deep research from OpenAI as well. So did you manage to try it, Richie? Did you experiment with the deep research model?

I haven't used the OpenAI deep research myself, but I've sort of read a lot about it from other people trying it and yeah, it seems the results are fairly mixed. It's not yet as good as a human doing research, particularly around shopping tasks. I feel like everyone spent decades now shopping online. They know how to do research products and having AI that does that is not as good as a sophisticated human yet.

Yeah, so I tried. It also makes reviews and when it comes to like asking it about, you know, reports of already really established topics and kind of themes that have tons of content on them on the online, it does a fairly standard job. Like if you ask it to explain the difference between getting an electric car that is used or getting a new ev, you know, a hybrid car, so on and so forth and it will give you like, selection of models, considerations, tables, so on and so forth. That was actually pretty nifty, but for new topics I found it to be quite disappointing. I'll give you an example. We asked it give us an overview of the LLM ecosystem from OpenAI, Google I think as well, Deep Seq. Right. And Meta. And the models that I listed for Deep seq were deep seq v2 and the one from meta were llama2 as the most recent models. And I found that pretty interesting because the query took 11 minutes to write. If you contrast this with Google's deep research tool, it also took like three minutes, four minutes maybe. And it had access to the Internet, it was looking at websites, but it didn't seem to pick up the newer models, which I found to be interesting. So I think there could be a gap in deep research that is applicable to like newer topics. But that said, this is a product that is included in the $200 bundle, right. So if I'm paying $200, I expect it to be pretty great. So it's, it's a bit of a mixed bag.

Yeah, that's tricky. And certainly once you start waiting for minutes at a time for a response, if it's not good, then that's a terrible product experience. Certainly 11 minutes exceeds my attention span for a lot of things, certainly for shopping desks. So yeah, if I'm wanting a sort of more scientific report on something, I can probably wait that long. But if it's just for casual usage, then that sounds like there's some work to be done there. Actually it does sound like These Deep Research products are kind of invading on the Perplexity space as well, where it's just a lot of search and a little bit of generative stuff on top there.

I do think that there's. Yeah, there's a lot of truth to that and it's going to be interesting to see how perplexity differentiates itself over time. For example, they released, I think in their mobile app, like a general assistant, they've been able to incorporate now reasoning models in their search. I've tried that recently. It was actually not that bad, the Deep SEQ experience and perplexity. And yeah, it's going to be interesting to see what that means for the industry. I mean, going back to the deep state conversation as well, if you remember the conventional wisdom around nine months ago is that OpenAI's next update is going to kill your startup, right? And that there's no point in building a wrapper around a model because the intelligence gains in the model will actually eliminate the need for many of the wrappers that are being made today. But actually that wisdom seems to be flipping on its head with the Deep SEQ release because now intelligence has been commodified. Reasoning models are becoming more and more commodified. I'm sure there's going to be an open source O3 mini competitor in a few months now, if maybe more, maybe less, but that's generally the time horizon and lots of folks now are changing their minds. They're like, okay, the way you differentiate yourself in the LLM space is by having a great application and a use case and an interface. Right. So I, Dewey, love to hear your thoughts here. What does competition look like in the foundational model space versus kind of the app layer? How do you see this? Do you see kind of any similarity with any other industries, for example?

Yeah, I mean, I think the fact that there's just been so much money being thrown at all levels of the AI stack, from like chip infrastructure right through to applications, it just means that there is so many companies competing for things now. And so at every sort of level of this, I think we're going to see competition intensifying because those venture dollars that were thrown in the last year or two, it's like, you know, you start to have to consider your Runway. Like if that money dries up now, then you've got to start making money somehow and the competition is going to get pretty intense. From a consumer of AI point of view, I'm very excited. Competition means stuff gets better, faster. And so yeah, great times for everyone using AI.

Indeed. But maybe on the competition thing, there also could be a dark side here. There's always been the discussion of an AI arms race could lead to irresponsible deployment of AI systems. Do you think that. That how big that risk is today?

Yeah. So I kind of defer to some great research from the center for AI Safety. So they've got some great stuff on like, catastrophic risks of AI. And there's only like a few ways in which AI is realistically going to go bad. But one of those I like to think of as the Westworld scenario, where, yeah, you've got competition and the organizations cut corners around health and safety and then bad things happen. I mean, minor spoiler for the show is that lots of dead bodies as a consequence of bad AI and many flashbacks.

And many flashbacks and weird twists and turns of the story. But that may not be the case if we have responsible use of AI. Yeah, indeed. It's going to be interesting to see. Hopefully we'll be able to avoid the dynamics that you've outlined here. And I think this puts us maybe in a good spot to talk about our next topic. Speaking of intelligent systems that are autonomous, let's talk about agents. OpenAI operator. AI controlling computers. Throwback to industry roundup number one. Richie, do you want to.

Yeah, sure. So in the first industry roundup a couple of months ago, we talked about Anthropic releasing an AI agent called Computer Use, which is designed to control your computer in order to perform simple tasks. So OpenAI has now responded, so they've released an agent called Operator. So rather than controlling your whole computer, it controls a browser inside a virtual machine. So a little bit. Bit more secure, bit safer. But it follows a similar approach where you describe a task of what you want to do and it takes screenshots of what's going on on the screen, just to check its own progress, to make sure it's filling out forms correctly. And then the space is kind of heating up. Google's announced something called Project Mariner. It's sort of coming soon. It's also like a browser control type thing. And the idea is that you can fill in forms with this sort of thing. There are also a few free versions available. So if you go on AgentLocker, which is a good place to find out what agents are available for different tasks, there's one called Open Operator and one called Smooth Operator. Very, very smooth name there.

You've been hit by a Smooth operator.

Yeah, yeah. Got a good Michael Jackson inspired name there.

Anyway.

Yeah. So These look like great tools for people who can't use computers themselves. So this might be people with disabilities or might just be people who are just away from their computer but need to control a computer. So ideally it's gonna be like voice activated, filling in forms to, I don't know, probably to buy stuff again is the big hope here.

Yeah. And maybe from your impressions and what you've seen, what do you think of the operator agents so far?

It's definitely early days for these things. And I guess my cynical take is that a decade ago we had Alexa and Siri and these things were like the big tech companies poured billions into these things to get people to buy stuff more easily. And people didn't do that. All they did was set the timer and listen to music. So it might be the same again. It might just be a case of, yeah, we've invented this really cool technology and yeah, you can set a timer.

With your voice, so that is fair. I do agree on the. I think it's exciting to see the potential of these systems with, especially for folks with disabilities. If you integrate with voice mode, even folks with really low digital literacy. Right. If you're able to access computers, like think about how many government documents processes are now digitized. If folks don't have digital literacy, they're really like locked out of the system to a certain extent. If you're able to kind of streamline that, it's still early days. Like I don't see operator being able to fill out your tax form anytime soon.

In the US it is impossible to fill out tax forms. This is what I've learned in the last eight years. In this country, no one understands taxes.

I can imagine. Same here in Belgium. So that's it. From the demo that I've seen. I'm also not super impressed now. Like, I don't see a use case for operate. Maybe if I am using repetitive tasks on Excel. Right. I could maybe see that. But I just don't see myself spending 15 minutes looking at a computer, navigating Uber Eats, ordering food for me and then struggling. Right. I can just like open the Uber Eats app and order food myself. Right. It just feels more intuitive. But if it becomes part of the OS of my phone where I can be like, okay, Google or oh, Google has woken up. Okay, Google, like order Uber Eats for me. Right? It will wake up. Like it will. It will do it. Fair enough. Right. But now Google is going to say informational purposes only.

For further information.

You've got to be careful. You're going to get second lunch there.

Yeah, no, but it's saying that I cannot order UberEats. Right. So if Google is able to order UberEats for me, it will be able to do it. So that would be an interesting future, for example. But I think we're pretty far away from having this like integration into the OSS that we use in a safe, secure manner that's like working locally on your, on your device. Right. So, yeah, I don't see myself paying $200 a month to, to watch a bot struggle on a website. Yeah.

Actually, so your UberEats example gives another problem here. You're going to have to give it access to your credit card and then trust it to order the right food for you. And food's like a fairly low stake scenario. So what happens if you want it to buy something bigger? Like it's got to be right 100% of the time to warrant that.

I mean, I can see like a workflow that is not fully autonomous. Right. And probably that's also best practice design that you can do here if you're doing any form of transaction or you need to present the user options. Right. So if I, let's say in this hypothetical future where I ask my phone to order Uber Eats for me, I assume it's going to come back and saying, hey, how do you feel? What do you feel like today? You feel like Lebanese, you feel like, like, I don't know, burgers, blah blah, et cetera. I'll give it my options and then it will make a selection for me, give me a few options, then it will make the options. Tell me, what do you think? Good to go? And I'm like, yeah. And then do I have authorization to pay with this credit card? Yes. Right. So I can see this working well. And going back to that deep SEQ example. Right. If you have really cheap reasoning models that can run locally on your machine, then that's also pretty secure because it's just running locally on your phone. But looking at operator, it's hard to see the use case now. But I do applaud OpenAI for taking the steps to make this happen. Right. This is still a research product. They mentioned this, they discussed this. It's not like they're marketing this to mass consumers. So it's going to be interesting to see how the quality of their. I think it's called, I forgot the name of the model. I think it's called an action model. Something improves over time. Like if that reaches some form of critical mass of quality and accuracy, it's gonna be. Could be transformative, especially if it's low latency.

Absolutely. I mean, yes, it's important to remember that GP2 generated terrible text, but like the series, eventually, the series of models eventually got very good. So, yeah, early days of computer usage models.

Yeah. So we still have our jobs. We're not gonna run on our job anytime soon. And this may be, I think, going back to kind of the deep SEQ example as well. We always go back to deep seq, the arms race that we've discussed. Right. I think kind of introduces our third story, which is the Stargate project. Right. Because in a lot of ways the AI space is heating up to a degree that it's taking on in national security complexion and a national for many of the competing nations here, whether, you know, the European content, the United States or China, it's taking on a complexion of national strategy. Right. So we have the Stargate project, which is a 500 billion investment program that was announced by OpenAI, SoftBank, Oracle, MGX. President Donald Trump had all of the CEOs of these companies next to him. They did this massive announcement. Right. Apparently ARM, Nvidia, Microsoft, Oracle, OpenAI are going to be the key technology partners. The only details so far is for AI infrastructure. So assume lots of computer. Yeah.

New chips, new data centers, and I guess, yeah, there's more bits of data centers, just the chips.

Probably nuclear reactors maybe as well. Probably nuclear reactors, given how they've been building it.

Absolutely.

Yeah.

I think most of the big tech companies have now signed deals to nuclear power for their own data centers.

Indeed, indeed. And then this is also maybe step one in Sam Altman's quest for a $7 trillion investment in AI compute to build the world's smartest grubhub ordering machine. And then new chips here and data center components. Definitely they're going to happen, but we don't know else. This of course, created many reactions. So Elon Musk is not necessarily the biggest fan of OpenAI. Given their recent fallout, people started buying the wrong MGX stock. And then it's interesting to see here what's going to happen. So I'll be blunt from the start. We don't have a lot of details. Right. So I'm not going to speculate over what's going to happen with Stargate. Will it actually end up coming to fruition? Yes or no. But given the trajectory that we're on with AI, the commoditization of reasoning models, the increasing national security complexion of AI and national strategy complexion of AI. I'M going to ask you a very simple question, Richie. Where do you think we'll be in the next four years?

Okay. Yeah. So, I mean, those numbers are really incredible. You think $500 billion sounds like a lot of money, but it's less than 10% of the way to this $7 trillion goal to create AGI. So I think for the foreseeable future, we're gonna have a lot of money shoehorned into AI. That's gonna keep the train rolling, gonna keep those improvements coming. The trickier thing, I mean, we talk about commoditization of reasoning models, but the number of players who can create cutting edge models is dropping. So as we see, like every generation of models requires this sort of increase in computational power, both the training and increases in data sets. And then it just means it's more expensive to create the next generation each time. And we've seen some players like Cohere Mistral kind of drop out from like the tops of the leaderboards. So there's fewer players now could afford to stay at the cutting edge.

Yeah, I mean, Deepseek is kind of the counter to that. Right. That they were able to train a cutting edge model with 5.5 million doll. But that's the question. Is that legit? Many people say that it's not legit. I'm not going to speculate, but generally I do agree with the trajectory of your statement. Right. That to build the more intelligent models you need more resources.

So with Deepseek, so the numbers I've sort of heard is $6 million of compute is about equivalent to like the medium sized llama 3.1. So it's kind of, it's this year's cutting edge model with last year's compute equivalent. So, so deep, it's got a generation or two ahead in terms of that kind of performance to compute ratio. But that's like, it's just a few generations over the period of like four years or whatever, it's still going to get more expensive to create cutting edge models.

Yeah, Just just to also caveat, I don't, no one has $6 million to spare to experiment with training. Doing one training run of an LLM. Right. So like I just want to caveat that. But yeah, it's going to be interesting to see how the kind of foundation model space heats up maybe based on just how much investment there is in the economy right now. In the AI space. Where do you see adoption in four years?

Yeah, I mean, adoption can only go up really. I don't think there Are any companies where we're like, you know, we use too much AI, let's start deprecating it. Yeah, that's not going to happen. So I think it's particularly in the spaces where things just take time to adopt. So you think about, like, government, there's often, like, processes in place where government's just harder. Same with highly regulated industries. So like healthcare, finance, anywhere you've got sensitive data, you just gotta do things carefully and right. So these things take time and that we're only just seeing adoption start right now. So, yeah, it's gonna be everywhere. It's just a slow process.

I'll maybe ask you one last question here on this one. On the prediction side. It's going to be a very easy one in four years. Do we have AGI, yes or no?

Okay, so I've seen like nine different definitions of AGI. Some of them we've got already, some of them not happening for decades. Some of them it's impossible to measure. So it just depends on which definition you use. But I feel like most tasks can use AI instead of humans. It's just not necessarily at the right price yet.

I think the AGI conversation for me is. Or the debate on AGI, is this AGI, is this not AGI is actually not super useful. The barometer that I use is, is this a GDP disrupting technology? Yes or no? Like whether it's a fully autonomous thing versus enabled augmented versus how is it, like integrated into the economy? Like, you can build maybe technically AGI in a box, but can you fit it in a modern organization? And does it have Excel integrations? These are, I think, kind of the important considerations to have when it comes to AGI. I'm being a bit facetious on the Excel integration. Right. But how do you actually use it? Can this disrupt the economy? And I don't see the conversation framed in this way a lot. I see it framed on, oh, it was able to code this Python Snake game really well. This must be AGI, right. I think it's more about what's really the impact of this technology and how will it be integrated in the economy and will this disrupt labor? And I think these questions are much less loosely defined from what I've seen.

Absolutely. So when O3 was announced back in December, things about, okay, we've broken records on the RKGI benchmarks, but then it was like it was $2,000 per task solved or something. And so that's only going to work in a very limited number of use cases in business where you're spending $2,000 on compute just to solve a simple task. Yeah, it's all about, like, solving things at the right price in order to get it into business.

Exactly. And maybe as we wrap up today, I would want to mention one honorable mention. Richie, have you seen the Replit mobile app, by any chance?

Replit always feels like it's very cool, but I never quite figured out how to make use of it. Go on, talk me through what's happening with our mobile app.

So I also agree with this, with the sense that Repl is actually, I think it's gotten better to use over the past few years. But it was always like, it had a higher barrier to entry than, like, your standard cloud IDE for me. But they seem to have redesigned their mobile app to just be around building apps for you using natural language. So it's an app that builds apps, and I found it to be interesting. I'm not going to pretend like it's this insane, groundbreaking, no more need for apps type, you know, I'm not going to do the clickbaity replit app is insane type content. Right. But I think if you assume the trajectory of improvement keeps going the way it's going, it could be quite disruptive for the app ecosystem in the next few years. For example, I created, like, in one shot, a Streamlit app that's on my phone that lets me track my weight on a daily basis. There's a lot of apps that do that, like. Right. There's a lot of apps that earn, like, ad money, right, from having downloads that let you do that or even premium apps with a subscription. I just need a graph that shows how my weight is evolving day over day. That's the only thing I need. And I was able to build it via replit. So it's actually quite interesting that, you know, because we've always had this debate over or, you know, this discussion, what will be the user interface of AI. And it could be just an OS that makes stuff for you. And I could see that with Rep. Right. That you only have an OS with a chatbot, and you're like, order me Uber Eats. And it just like, connects to the Uber Eats API and orders you something. You don't need the app on it. It's kind of like a chatbot interface to the world, whether it's making apps, booking flights, being an agent, I could see that in the future. This is what the repl app was able to show me. I'll frame it that way. Yeah.

Okay. So first, I feel like we should not record just before mealtimes because we keep coming back to ordering food examples.

Yeah, I'm getting hungry.

But, yeah, I do like the idea that you probably want a chat interface to most software. So, yeah, I mean, traditionally, like, at least technical software, you've had like, for a long time. You have two interfaces. You've got like a coding interface, you've got a graphical user interface with your point and click. And now it's like all software should have three interfaces because you want that natural language interface as well.

Indeed. And on this important change, we end today's industry roundup number three. Rich, do you have any parting words before we wrap up?

No parting words, but yeah, I'm excited for what's coming this year.

I'm excited for what's coming this month at this point, because there's just so many releases happening on a weekly basis, it's really hard to keep up. But I hope that we were able to give you some information in these 30 minutes of recording. And with that, Richie, I'll let you go enjoy your lunch and I'll order.

Some dinner using a computer use agent.

I'm going to have a healthy meal and track my way on the Replat app and take it from there. Cool. Thank you so much, Richie. Always a pleasure. Cheers.

Bye.

Cheers.

SA.