Google's Gemini 2.0: HUGE Update! [Flash & Pro]

The agent process is going to scale, which is going to be really interesting. Like, I have 10,000 emails I haven't read in the last three months.

Right. I was thinking emails. First thing I was thinking of, I wanted to handle all of that for me.

I've gotten in the habit now of, like, when I have, like, a little problem or a bottleneck that I need to solve, I'll just go prompt that software into existence.

That's the agentic future that we've been promised. I couldn't be more excited about that.

Hey, welcome back to the Next Wave podcast. I'm Matt Wolf. I'm here with Nathan Lands. And today we've got a special guest out of Google, DeepMind. It's the senior Product manager, and he's been on the show before. Today we've got Logan Kilpatrick joining us, and the day we're recording this episode is the same day that Google just released a whole bunch of new AI tools like Gemini Flash 2.0, Gemini Flash 2.0 Lite, Gemini Flash 2.0 Pro, and we're getting some updates about their Imagen 3 model. All sorts of really, really cool stuff coming out of Google right now. And Logan's going to break it all down, what it's capable of, where they're going in the future, and you're going to get a pretty grand overview of where the AI world is headed according to Google. So let's just go ahead and dive right in with Logan Kilpatrick. Today we're joined by Logan Kilpatrick, who is the senior Project manager over at DeepMind. Thank you so much for joining us. It's probably a really busy day, so I really appreciate you taking the time to join us today.

Yeah, it's been a ton of fun. I'm excited to catch up with you both and talk about all things Gemini and what's happening in the AI world.

Yeah, well, so this is actually your second time on the show, so we've already kind of dove into some of the backstory and introduced people to you in a past episode. So let's not waste any time. Let's just jump straight into it. Can you break down what is 2.0 flash 2.0 flashlight 2.0 Pro? What are the differences? What's better about these models than what was out prior to them?

Yeah, no, 100%. I think this is an exciting moment for us just because of, like, the amount of effort and work that's gone into bringing Gemini 2.0 actually into the world. And Matt you were there, Nathan. I don't remember if you were there at IO last year, but we announced 1.5 flash and the Long context and a bunch of this other stuff last May. So like literally less than a year ago. And for the last year, Flash has been this wild success story for us of building a model that developers really love. And a lot of that is rooted in like the, the right trade offs of like, cost, intelligence, performance capabilities. And 2.0 flash is like, you know, if you look at the 1.5 flash model and you think about like, how do we do this better? It's like, well, you, you have to make it more powerful, you have to give it more capabilities. You have to do that all while not making it cost a lot more money for developers. And it feels like we pulled a rabbit out of the hat to a certain, to a certain extent with this model because like the actual cost for developers, it was historically like 7 1/2 cents per million tokens. Now it's 10 cents. But we don't distinguish short context versus long context. Historically, like if the request was over a hundred K tokens, we'd, we'd charge you 15 cents. So the blended cost is actually less for this model. And we did, we did all that while, like, this model is actually better than Pro. It has all these capabilities, it's natively agentic, it has search built into it as code execution built into it. And I think it's. Yeah, it's just exciting for me as a developer. Like, I think ultimately you remove the cost barriers and all these other things for people to build really cool stuff and like, that's what enables the world and to make really cool products. So I'm super excited. So that's the sort of headline for flashes. Better, faster, cheaper.

I read that it's better than GPT4O but like 20 times cheaper. Is that like, roughly correct?

Which is, which is just crazy to me. And I think like, we've got a lot of work to do. I think there's like a lot of dimensions to the Gemini story. And I think one of the dimensions of the Gemini story is like, we continue to put out really great models. I think we need to do a great job as well of like going and telling the world about this technology that we're building because I don't think people really understand and actually for a lot of developers, like, the cost, if you go and talk to developers, like, the cost is the reason in many cases they don't build the stuff that they want to build it's like I can't afford to put this thing into production, it's too expensive. So I think Flash is really important in that story. But we also landed 2.0 pro. We also landed an even cheaper version of Flash flashlight which sort of has the capabilities pared down a little bit, but makes it so that we can keep delivering on that like frontier cost, performance, sort of trade off. And that model's in preview and it'll go GA in the next few weeks as we iron out the last few bugs. And I think Pro gives me a lot of excitement about, you know, this whole narrative of pre training being dead I think is so interesting to me. And it's as you look at the benchmarks, an interesting sort of realization I had after a conversation with Jack Ray, who's one of the co leads for the reasoning models in DeepMind is there's this non linear amount of extra effort it takes to make the model continue to get better. Like you look at like okay, what is 3% mean on some benchmark? Like you think 3% and we think of like the normal world where 3% is like actually 3% and like in the model world 3% is like actually like a 25% increase in like the amount of effort that went into this. But also that 3% is like the difference between unlocking a bunch of capabilities and a bunch of use cases that like just didn't work before because like the thing failing 3 to 4% of the time versus not like is the difference between you putting AI into production at your company and not so it actually matters a lot. And that's why I think we continue to push on that frontier.

So what's the big difference between these new models and the last ones as far as like how they were created? Is it like more parameters that are being trained on? Obviously the big narrative like you just mentioned with things like deep seek and O3 and things like that from OpenAI are the sort of what happens at inference, right when somebody enters a prompt it does all of this thinking and that's really what they're sort of like pushing on is like the next sort of breakthrough. Like what, what sort of breakthroughs? What, what changed between the last models in this one to make this one so much better?

There's two dimensions of this one. I think there's just a lot of like, it's a story of like the, the really difficult work of doing algorithmic improvements and breakthroughs and I think like the team at DeepMind stuff way beyond my understanding as far as how they're able to, to work. So I think there's like core fundamental research advancements that are happening. There's a lot of like data efficiency wins as well, which is, which is also exciting. But as far as like new capabilities of these models, I think the, the two big ones is when Gemini was first announced, it was announced as this model that's natively multimodal. And it was natively multimodal in the sort of input sense that it could really understand the videos, audios, images that it was being given. And that was one of the main differentiators today. The model's actually capable of doing that except on the output sense, which I think was a huge jump for us. And it actually requires again a bunch of non trivial amount of engineering work in order to make the models capable of doing that. And I had an interesting conversation a few weeks ago with someone on our research team which reminded me of this, which is someone asked a question of well, why does it matter if the models are capable of natively outputting these multimodal capabilities? We have really great, you know, text to speech models, we have great speech to text models, we have great image generation models. Like what is it like, why is it cool that the model can do this natively? There's all of these really great examples of the base models, world knowledge really unlocking these new capabilities that the image models are like not smart. And I think it's actually like a good example of like you know, a calculator versus like, I don't know, an AI model that has access to code execution. Like the code execution version can really like solve these problems in a really complicated way that you wouldn't otherwise be able to or at least that the effort is required as you on the user of the models. And I think that's the world of these custom models. Custom domain specific models like image generation and audio generation versus the native capability really feels like the model can just do the heavy lifting for you, which is really interesting.

So right now can Gemini actually output like an image if I give it a prompt to generate an image, does it generate an image right now?

Not, not accessible to everyone yet. And I think this has been the, this is the gap. So we, we have it internally and, and folks are using it in our early access program and we should get you both early access to, to play around with it and test it out and we'll roll it out more broadly soon, which I'm excited about. But that sort of same line of thinking is what takes us to like Native tool use as well. And like native tool use is available to everyone. And it's like the model was trained knowing how to differentiate questions that it should go and search the Internet for or questions that it needs to use a tool like code execution for. So you get like all of those like silly examples where the model would be like, let me try to solve this math problem, which I know I'm not going to be able to solve just because you asked me to with code execution, like it knows it needs to use that tool. And the. Yeah, there's a whole bunch of verticals where like the performance goes up significantly because of that.

So Gemini is actually like generating the image. It's not going and calling upon like Imagine three to generate the image. It's actually Gemini who's creating that image when it does generate an image.

Exactly. And I'll push on getting you both access after this conversation because I think the World Knowledge piece really highlights why this matters. And there's a bunch of examples that I played around with, of course, pictures of a room and like asking the like having the image change based on these like really complex nuanced prompts around like moving objects in certain ways. Like things that if it doesn't have world knowledge and understand like it understands physics and understands all these things that again require the World Knowledge piece. And I think it's actually there's some interesting trends of like, what is the outcome of like being able to take other domain specific models and bring them into these alarms that have world knowledge. I think there'll be some really cool capabilities that like we're not thinking of today that this is going to enable. Which yeah, gets me excited for people.

Yeah, it seems like that's kind of required for you to, for this also to work in like everything from like gaming to robotics to it actually have.

An understanding of the world 100%.

Yeah, yeah. I mean I actually had the opportunity to go out to London, go visit the DeepMind offices and got to play with Astra on the phone. And it was, I mean that was like the first taste I got of like an actual useful AI assistant like out in the real world. And I don't know if that was using Gemini 2.0 flash or if that was actually using the full Gemini 2.0 yet at the time. But you know, it was, it was definitely a very, very impressive model to actually see the tool use in real life and just be able to walk around and understand images and understand video and understand audio and understand text and all of that. All and it's, and it was all built from the ground up to understand that stuff. As opposed to, like, look at an image, use OCR to figure out the text on the image, then pull in the text or listen to the audio, transcribe the audio to text, and then use the text. It's actually understanding what it's seeing, what it's hearing, which I think is like one of the major differentiators of about, like, what Gemini is doing that you don't see the other models doing yet. So super, super impressive.

Yeah, I think the, the other piece of this is what, other than just the raw capabilities. From a complexity standpoint, it means that like, developers building stuff, you don't have to go and like, do a ton of scaffolding work in order to, like, make this happen. It's like the overall complexity of your application when the model is just able to sort of take in a bunch of things and put out a bunch of things. Makes life incredibly easy. And you don't have to. Yeah, you don't have to deal with like, frameworks on frameworks on frameworks. Which I feel like is a lot of the. I think this is one of the challenge of the agent moment is like, you know, so much of the agent world is like, hey, the models actually aren't that good at doing some of these things. And like, the way that we supplant that is by like building a bunch of scaffolding and frameworks. And like, that's where a lot of developers are focused today. And like, actually there's going to be a moment where like all of a sudden the model capabilities are just like, good enough that it kind of works. And then people are going to be like, well, why do I have all this scaffolding that's doing these things for me that the models can just do out of the box now. So it'll be interesting to see how that plays out.

Look, prompting is all about technique. Just like any craft, using the right approach makes all the difference. The same goes for your AI prompts. Leave out the key details and you won't get the results you're after. Well, HubSpot's got your back. They worked with YouTuber and former metadata scientist Tina Huang to develop the ultimate guide to crafting the perfect AI prompt. Using a three part framework. This framework will make sure that each prompt is chiseled to perfection so you can become a prompt engineer. Bro, grab the guide at the link in the description below.

Now back to the show.

Yeah, I know like 2025 is going to be sort of the year of the agent. Right? That term has already been thrown around quite a bit. But I also feel like everybody kind of has a different definition of an agent. You know, some people will look at something that like you can build over on make.com or Zapier where it's tying different tools together using APIs as an agent. But I'm curious, does, does Google and DeepMind, do they actually have like an internal definition of an agent that they're shooting for? Do you think we actually have agen based on their definition? Like where does Google stand on agents?

Yeah, this is a good question. I think I actually don't know what our like formal definition of agents are. I think the thing that I'm personally really and I have tried a bunch of the agent products and like haven't been super impressed of like what, what they're capable of. I think we're just not there yet. The thing that I want is, and I think a lot of users want this as well, is just like the models to be proactive and, and like all of the products of today that build on AI require me to basically change my workflows or put in extra work or put in extra effort like through this sort of guise of like, oh, this is actually going to save you time if you do this thing. And like really what I want is like the models to just be like looking at the stuff that I give them access to and like coming up with ways to be useful and like save me time and like, yeah, it's going to get some things wrong but like I don't want to have to be the one in the driver's seat all the time. And it feels like today's AI agents are again in a lot of cases because the models aren't good enough. Like have to be proactive. Like it requires the proactiveness of the human. And I think once that role reversal switches, I think that's where we see like billion agent scale deployments like all of a sudden just like happening and working. And this is also where like compute starts to be. Like things get crazy again with compute. Because again like actually if you look at how compute is being used today, it's like there's a, it's like almost a, in a lot of cases like this one to one correlation between like a human input and the, you know, token output. And I think the future is like, you know, thousands and thousands of X more usage of AI happening by the agents themselves than, than by humans, which will be, which will Be fascinating to see play out.

Yeah, no, I, I think what you described is sort of what I envision of an agent is almost like predictive. Like it sort of figures out what you need before you need it and then makes suggestions based. So I think, you know, that, that that's a world that I'm, I'm really excited to, to get into. But I do want to touch on, on the word compute that you just mentioned for a second because obviously there was a, a bit of like a, you know, a freak out, so to, so to speak, in the, in the US when that deep SEQ model came out and everybody thought that, well, this deep SEQ model uses a lot less compute than, you know, these other models that have been trained. So therefore Nvidia stock is no longer, you know, Nvidia GPUs are no longer necessary. And then we saw Nvidia sort of lose some market share as a, as a result of it. But I'm just curious, like, what are your thoughts on the compute? Because I saw all of that happening and I thought it was so bizarre. I was like, this seems like a bullish sign for Nvidia to me. Not a bearish sign, like, what's going on here? But I'm curious, like, what's your take on what happened there?

Yeah, there's a lot of complexity to that story and there's, there's parts of the story that I don't want to touch on. But the thing that I do that I do want to touch on is like, if you look around the world, I think this example that I just gave of like, who is in the driver's seat of using AI today? And like, today it's humans that are in the driving driver's seat. And like, we're just inherently bounded by the amount of humans who are using AI because, you know, it just takes a while for technology to assimilate into, into culture and real use cases and all that stuff. I think as soon as agents work, it's, you know, we're on this exponential right now. I think as soon as agents start to take off, that exponential becomes a straight line up into the right. Like, I think it's going to be pretty profound. How quick? Because like, again, the challenge is the, the human process just like doesn't scale and the agent process is going to scale, which is going to be really interesting. Like, I have 10,000 emails I haven't read in the last three months.

Like, thinking emails. First thing I was thinking of, I wanted to handle all of that for.

Me, it's going to be wonderful. And really like, go in and find the things, like, I know there's things that I'm, that I should be doing that would create values. Yep. Yeah, 100%. And there's like so many of those things where if you also like, think about like, what's the economic value of.

Of all that work.

Like, one of the crazy frames of mine that I look at the world through is like, you look at the world and like the world is just filled with all this inefficiency and like it's, it's beautiful in many ways, but like, it's also this really cool opportunity if you can be the one to create something that makes people more productive and explicitly makes them more productive at maybe the things they don't want to do. And I feel like that's the other part of this agent story and this compute story, which is a lot of the products that I see people building are actually going after the things that people really like doing. And like, maybe like shopping is this sort of tongue in cheek example of this, because some people really like shopping and some people don't. But like, there's, you know, if I, if I told my girlfriend and you know, that we could never go shopping again together and we could never go, you know, try out different experiences and go check out the vibe of different stores. Like, there's so much of. This is like such a fundamental part of the human condition actually of like going and seeing these different places and like, it's, it's really baked into who we are. And it's interesting that, yeah, that like, that's such like a traditional example of like, what agents are going to do for people. And it's, it's odd to me. I feel like people have like, kind of misconstrued what the value creation is going to be.

I mean, I think that's like the bias from like, you know, Silicon Valley nerds building this stuff. Right. They're like, yeah, I don't want to shop. I just want to be automated for me. And it's like, that's just like a small subsector of the world.

Yeah, I think there's like, there's an underlying story here which is like, first of all, there's a lot of variants in human preference, but also there's like, ways about going about a certain task that like, make it interesting or not. And like, I like shopping too. If it's like me getting to do it on the terms that I want and like, it is, it will be Interesting to see like how, how can agents and some of these products like actually help create that experience. And this goes back to like the whole separate thread around like the app which was part of this, this deep seek narrative around like the value creation happening at the application layer. And it really does feel like this is true. Like if you look back two years ago, the narrative was, you know, all these companies are just wrappers on top of AI. There's no value creation. All the creation, the value creations in the tokens. Like you know, don't, don't spend your time thinking about these companies. And it's so funny how quickly this like flip flops back and forth between like now all the value creation is at the application layer and like LLMs are this commodity thing that no one should think about. So I enjoy watching it all play out. It's fun.

So how do you think it's going to play out? Because like right now Google's kind of focused on developers, right? More than consumers, correct?

I think we have it. So on one hand I think like I spend all my time thinking about developer stuff. I think Google's got a ton of other people who are doing consumer stuff. I think a good example of this is like Gemini is both in search and through the Gemini app like you know, deployed across like billion user scale. Like there are literally billions of people who are interacting with outputs and the models themselves, which is, which is crazy to think about. And like it's a very consumer focused consumer forward use case for, for those folks. And I think it's also like still incredibly early for, for that, you know, Gemini in search. And there's, there's some interesting stories around that stuff but I mean I personally am incredibly bullish on the, the infrastructure layer and like the infra tooling and I think like actually a good example of this is, and you see this in some of Sam's recent tweets about sort of OpenAI and the Pro subscription are sort of good examples of this. Which is builders at the application layer have a lot of tension with building more AI and like actually back to the thread of like having AI be more proactive. Like this is why I believe in Flash so much and I believe in the direction that we're going as far as like reducing the cost for developers while continuing to push the performance frontier is because the story of AI to me is a story of like the actual infrastructure incentivizing developers to not use it. Like you literally have an economic incentive not to use AI because it costs you money and like the more AI you build into your product, the more expensive it is and like the more margin pressure you have as an application builder. And scary, right?

It's scary like to build an app and like all of a sudden you get like a gigantic bill because people are using your thing and you haven't figured out how to properly monetize it yet. As you know someone who creates companies, it's, it's kind of intimidating.

Yeah, 100%. And I think like that's, that is like the very, the realest reaction and like also just like the truest reaction developers have to the cost of the technology. So back to the point of like where the value creation happens. I think the nice thing for infrastructure providers is you have a fixed margin so like you know exactly how much money you're going to make by providing some infrastructure at the application layer. You like, you're constantly incentivized to almost like not add additional stuff. And I think this has been the story for like the ChatGPT plus and Pro subscription is like they built a subscription for $20 a month and they realize hey, we actually can't give people all of these things anymore. We have to make something different. And like even at $200 a month, it's like not a break even scenario for them yet. So it's super interesting to see that play out and it's a lot of food for thought for, for people who are building stuff to be like what are the new, like are there new economic incentive mechanisms that you can create as you're building a product? Like more so than charging a $20 a month subscription. And like the one example I can think of of this that, that has made me think is I don't know if you all have, are familiar with Open Router, but Open Router's product leaderboard has some really interesting, like they, I'm pretty sure they give you like discounts on tokens and stuff like that for showing you in certain ways and like certain, like I don't know, I don't think it's like actually all the data passing but like some metadata passing back and forth so they can understand like how people are generally using AI models. So like some interesting things like that, that again that's a very early example of this but I've heard of Open.

Router but can you explain what it is? Otherwise I'm kind of lost on what you're talking about.

It's a surface that lets you sort of swap in and out different language models and they have Alex atala who's the CEO who previously worked on OpenSea, the NFT platform. He has this quote which reminds me, which always rings in my head, which is like, usage is the ultimate benchmark. Like how many people are using your, your model or your thing is like the proof point of success, not all these other benchmarks that people are chasing after. So the like core benchmark for open router is like, it's a stacked rank usage of like how many tokens were processed in the last week on each of these different models. So interest. Super interesting platform. Just like one data point for people thinking about this stuff. But it's been, it's been fascinating to watch that, watch that happen.

Is that like a, like, like, do they actually publish like a leaderboard similar to like what LMSYS does?

Exactly. Yeah. Yeah.

So you can go.

It's like openrouter.com maybe.

Yeah, openrouter AI and it's forward slash rankings.

Awesome. No, I'm curious, a little bit of a topic shift here, but I know you're a proponent of open source and Google obviously has their Gemma models. Are there any updates, any idea of what's going on with Gemma and what we can expect next out of the open source side of things?

Yeah, I think that this is also the piece that makes me excited about what we're doing at Google, which is being able to like, it really is the exact same research that powers the Gemini models that ends up making the Gemma models. And Gemma 2 was, I think it's like the second most downloaded open source model in existence, which is awesome to see. And Gemma 3 is definitely going to happen. I think the timeline is soon, so you'll, you'll hear more. And y' all should do an episode with some of the Gemma folks because there's lots of cool stuff coming. And they just released like in the last. They've been doing a lot of like, interesting fine tunes of, for different use cases. Like they have a version for Rag, they have a version for Vision. I think they're probably going to do some agent stuff as well. So like there's lots of really cool explorations happening on making those, those open models.

Super cool. I want to talk about Imagine three, which I just learned is actually pronounced Imagine three. Like I've been calling it Imagen. I always thought it was like image generator, so like image gen. But I heard you, you pronounce it Imagine. So now I'll start saying it that way. But you mentioned that there's some, some updates with that as well. What can you Tell us about that.

Yeah, and you're. You're in good company. Don't worry. I swear, like 50% of the meetings I'm in, I hear imagine 50% I hear image Gen. So it is not. Yeah, there's no conclusive answer. I think it's Imagine, but some. Somebody should correct me if that's not the case.

Yeah, I think we. So we released the Imagine 3 model across a coup late December and then have been doing a bunch of work in the last few months to bring that to developers. So tomorrow or today or whatever day folks are listening to this, Imagine 3 should be available to developers in the API and is. Is the frontier image generation model across like quality and a bunch of like human ranking benchmarks, which is really is just as an aside comment, it's super interesting that if you look at text models, I think one of the reasons we've had so much. The world has had so much success hill climbing on making models better is like there's a definitive source of truth of, of in some of the tasks that the models can perform. I think with, with image models it's like actually not the case. It's really hard to eval and like you actually need humans in the loop to do a lot of those evals or it's like artistic and stylistic stuff that's like hard to put a finger on which of these two things are better. So a lot of those evals use human radars and human benchmarks. So there's some degree of error. But yeah, it's been exciting to see like the models available in the Gemini app, it's available to enterprise customers and now it'll be available to developers to build with, which is, which is awesome. I think this like gen media future is going to be. Is going to be super exciting and veo, hopefully sometime in the future as well.

Yeah, yeah. VEO is awesome. I love playing. I have access to it, early access to it. It's super fun to play with. Is the best to use the Imagine 3 inside of the image effects. Is that still kind of like the easiest way for a consumer to just go and play around with it?

Exactly. It's. I think it's also available for free to folks in the Gemini app. I think if you ask to generate an image, it'll just do it through the Gemini app. But Image FX gives you a little bit more controllability and stuff like that. So there's a few more like features that are built into Image fx. So that's definitely a place that it's publicly available.

Super cool. Yeah, I know you and Nathan, sort of before we hit record, we're nerding out a little bit about the whole sort of text application concept. Nathan was mentioning that he wished that there was a way that he can actually have Unreal Open on his screen, or Unreal Engine or Unity or something like that. I don't remember which one. You said he wished there was a way that you could have Unity open on the screen and then actually have an AI sort of assist you with where to click and what to do next while you're building a game in Unity. And I think both of you, both you and I in unison went, you can do that in AI Studio right now. So I thought, I thought it might be kind of cool to pick up where we left off on that conversation and talk about some of the cool stuff that's available inside of AI Studio that maybe a lot of people don't even realize exists and probably definitely don't realize you could use most of it for free still right now too.

Yeah, well, so everything in AI Studio is free, which I don't think people realize. Like the entire product experience, there is no, there is no paid version of it. There's a paid version of the API which hopefully developers can scale with and do all that fun stuff. But all of our latest models end up in AI Studio for free, including the experience that powers this real time multimodal live experience, which if folks haven't played around with it, aistudio.com live lets you do things like share your screen or show your camera and ask all these different questions and interact with the model. There's a bunch of different voices, there's a bunch of different modalities to choose from, but really I think it pushes towards this future of back to the conversation of like, what, what will agents look like? What do we want out of agents? This co presence of like the model. Like, one of the limitations for agents is you have to build all this scaffolding for the agent to be able to see the things that you do. Like to see my email and my text and my, et cetera, et cetera, my personal laptop and my work laptop or my phone and my watch. Like incredible amount of work to make that happen. Except if you have a camera, all of a sudden all of it just works and like you can, you can sort of make the determination of being able to show like the information you want to show and share the stuff that you want to share. And I think it's a, it's Definitely more of a showcase of what's possible. And that's why we put it in the API because like, you know, we don't have the answers ultimately like developers should go and build these products. But I do think. Matt, you mentioned the Astra Experience earlier. I think like this, the, the Multimodal Live API sort of gets to what Astra does at the core, which is like be able to be this co presence with tools and again through a, through a simple API, which is really exciting. I think the piece that the Multimodal Live API doesn't have that will, will build, that I think the Astra experience did have is this like notion of memory, which again is like critical for agents. Like, I don't want agents to just like forget that it, you know, I prefer sitting in window seats instead of aisle seats or whatever it is like you want all that context to be retained as, you know, agents are making decisions for you in the future. And I think that's going to require this, this sort of memory layer which we're, we're working on building, which is exciting.

Yeah, yeah. And I mean, Project Astra even sort of remembered between sessions too that like I was in London, right? So I was, you know, talking to it while in London, asking for, you know, restaurants to checkout, things like that close that session, started another session later on and it remembered that I was, you know, over in London, you know, and it remembered all those, the previous conversations. So it wasn't just like memory in the terms that like, you know, you can plug into the custom instructions on OpenAI and it'll remember your name and stuff like that. It was actually like remembering the past conversations and bringing in that as additional context, which I thought was really cool. And I keep on wondering why more of these AI tools aren't doing things like that, because that's really helpful to just like remember the past conversations you had.

I think this is an infrastructure problem and I think like actually back to this, you know, we didn't talk about this explicitly but like one of the other narratives over the last like year and a half has been like, not enough AI in production. You know, it's kind of this demo toy thing and no one really uses it. I think a lot of this is because it's just like taken a while for companies to build the infrastructure to actually put AI into production. And I think memory is this example of there really isn't a, there aren't a bunch of companies that are like building this memory as a service. And if you are building this, like, let's talk. I'd love to hear about it and hear about what you're building, but you could also start to think about, like, there's so many interesting ways to explore this. Like, where does all your personal contacts already live today? Like, how. How does that, whoever that provider is, like, you know, plug into the world of where all the other memory services are going to be? So I think there's a lot of, like, really, really interesting directions that, that need to be built for. For mem.

Logan, I'm curious. So you're talking about earlier, the models that a lot of people don't realize that all of it's free, like an AI studio. Like, like, why do you guys hide.

It in AI Studio?

I feel like recently I talked to a bunch of different people about Deep Seek and they were talking about how amazed they were by it. And I was like, yeah, but like, you can get the same stuff but better for free on AI Studio right now. And they didn't know. And it's like, there's a lot of people who don't know.

Yeah, yeah, no, I think you're spot on. And for what it's worth, like, I think we get this feedback pretty consistently. I think some of this is a factor of just like, the, the state of the world and the challenges that we have as a product. Like, I think in one hand, like, we are, we are a developer platform. Like, we're not building the front door for Google's AI surfaces. Like, that's not the product that, that I'm signed up to build. That's not the product that, like, we're. We're sort of directionally building towards. We're really focused on, like, how do we enable builders to get the latest AI technology. The Gemini app, formerly Bard, is the sort of front door to Google's AI technology. And I think from a consumer standpoint and also from an enterprise standpoint in, like, workspace and other places, there's all these interesting sort of organizational work that's happened at Google over the last couple of years. I think, like, one of the cool stories is, like, operationalizing Google DeepMind from doing this sort of foundational research to being an organization that builds the world's strongest generative AI models and like, actually delivers those to the world sort of as a product, and then now bringing the product services that are the front line of delivering those to the world. The gemini app, Google AI Studio into DeepMind, so that we can sort of continue to accelerate. Like, all of those things to me are like, Directionally the right stuff for us to do. I agree with you. I think we need to put the models in front of our, in front of the world as soon as possible. And I think having a single place to do that and makes sense. And it should probably be the Gemini app, it probably shouldn't be AI Studio, but we have done like, you know, at the same time I say that, like, we also want to be a surface, to sort of showcase what's possible. So there's a lot of, like, a lot of tension points. But I do think, I fundamentally do think we're going to get there and I think a lot of this is going to be solved by. The Gemini app is moving super quickly to like, get the latest models. Like, they just shipped 2.0 flash today. 2.0 flash, actually, before we did an AI studio, they shipped 2.0 Pro today. They shipped the thinking model today. So I think that that delta between the Gemini app and AI Studio is sort of going away.

Yeah, I always saw the AI Studio as sort of like the, the playground to test what the APIs are capable of. In the same way, like OpenAI has their OpenAI playground and you, you can kind of go and mess with some of the settings and see what the output will look like before using it in your own APIs. That's kind of how I always saw the AI studio because, like, once you get into it, if, if you're not very technically inclined, you might get a little overwhelmed seeing things like, you know, what is, you know, what models should I be using? What is this, like, temperature, you know, things like that. People aren't necessarily going to know, like, how to play with that. Like, what should I set my token limit to? Things like that. I don't really feel like general consumers want to mess with. They just want to go to a chat bot and ask their question. Right. So it's, it feels very tailored towards developers to me. And then the Gemini app feels like, all right, this is, this is their front user interface that they want people to, you know, the general public to go be using at least.

Yeah, I got a ping yesterday actually, from someone saying, why is the chat experience have all this stuff in it?

Like, why are there all these settings and stuff? And I was like, I was like.

You'Re in the wrong place. Like, Gemini app is the place that you need. And they responded right away on Twitter and they're like, ah, yes, this is much better for me. I don't want to see all that complexity, but I do think it's A and I think about this a lot is like what are the way. Like how do people actually show up into the. Like how are they finding their way into these products? I do think the Gemini app is like this very large front door. So it, it tends to capture most of these folks. Like it's literally built into the Google app on iOS and all this other stuff versus like you actually kind of have to do a little bit of searching to find AI studio, which probably makes sense in some cases.

You know Logan, you said you were super excited about text application. I would with like you were talking about like lovable bolt other you know, companies like that. Like what are you excited about in the space and where do you think that kind of stuff is. Is going?

Yeah, being able to democratize access to people, building software and like creating things. Like I. There's a, there's a ton of people in my life and I don't live in the Bay Area, so there's more a disproportionate amount of people who aren't in tech where I live and like. But the proportion of like people with interesting ideas I actually think is the same. I think it's just like the, the actual, the tools themselves that they have to go and execute on those ideas that I think much less distributed in places outside the Bay Area, New York and other places like that. So I think this, this frontier of text to app creation is going to be so, so interesting to see play out. And yeah, there's a ton of companies that are having like lots of like actual real early commercial success and traction today. One of those examples where like sometimes there's use cases that don't work and then all of a sudden like the model quality just gets good enough or you build the right sort of couple of things from a product experience and then all of a sudden it clicks and like now this thing is possible. And it feels to me, it feels like text to app creation like has had that moment and it's now possible and I think it'll take a while and there'll still be a bunch of other things to hill climb on, but I think especially now with like reasoning models and the ability for them to like keep thinking and writing more code and doing all that work, like I think we're. The complexity of the apps is also going to continue to, to like go up on this exponential. So it'll be yeah, I'm super and actually replit just had their launch I think today or yesterday of like this a similar sort of product of text Apps. I think there's more and more players showing up in this space and it feels, it actually feels like, I would assume that like probably 50% of products or something like that that are building with AI have this type of experience. And you could think about like, you know, how does that translate to someone who's doing something very, very domain specific. I think there's a lot of like companies that try to build extension ecosystems or connectors or like, you know, all these other like sidecars of their product. You could imagine, like you just let your users create those. Like here's the sort of generic set of APIs that talk to, you know, your email client for example, and like here's a text box and like go build the, the sort of product experience you want. Like it's sort of in your hands and like that's, that's a crazy world.

That you can totally customize it however you want to. That'd be so crazy, right?

Like, yeah, it doesn't have an email.

Client that's got like, you know, 80s style video game stuff. You know, it's like mixed in with the email client. That'd be so cool.

Yeah, no, I've been loving that concept. We've talked about this a couple times on this show of like, I've gotten in the habit now of like when I have like a little problem or a bottleneck that I need to solve. Instead of going and like searching out if there's like a SaaS company that already exists that has that product for me, if it's simple enough, I'll just go prompt that software into existence and I have like a little python script that runs on my computer to solve the problem for me, right? Like I, I made a little script where I can Input my, my 1 minute short form videos into it. It automatically transcribes them and then cleans up the transcription and adds like, you know, proper punctuation and stuff. I created another like mini app where I can drag and drop any image file, doesn't matter what type of file format it is. It'll convert it to a JPEG for me so I can use it instead of like my video editing. And these are probably softwares that exist. I could go and hunt them down on the Internet and maybe pay five bucks a month to use them. But I could just go use an AI tool and prompt the tool that I need and 15 minutes later I have something on my desktop that I don't need to go pay anybody else for anymore. Maybe it connects to an API, like my transcription one connects to the OpenAI whisper API. So it is costing me like a penny every time I use it. But so what? I just love this concept of like, when I have a bottleneck in my business, I can just go like prompt an app into existence that solves that bottleneck.

Yeah, it's, I think that carried out one step farther towards this like infinite app store where like, truly everyone is creating and contributing to this thing and like remixing and doing it. Like, I think there's, there's just going to be. This is the stuff that gets me excited about the future because like, there's so much cool stuff to be created. And really I think that the lens of all this is like, how do you democratize access and make it so that anyone can go and build this stuff? And as someone who, who can program but also knows how painful it is in a lot of ways, it's just like so cool that more folks are going to be able to participate in that. It's. It's going to be awesome.

Yeah. Yeah. Coincidentally, someone today from my hometown Alabama, like messaged me like, hey, I have this idea for an app and I get this kind of stuff all the time, have an idea for an app and. But who can I hire to build it and all that stuff? And I'm like, I'm about to send him a link to like repl it, you know, like, like try this yet. You know, it's like just, just go try that. And instead of like paying someone $5,000, that's probably like a ton of money for, for him. Right? But. And it's going to get better and better. Like, I've tried replit and all them and like, they're pretty good. It feels like there's something that's like slightly missing, but every time I check it, it's better than the last time. And, and it feels like probably within the next year or two, you're just going to make any kind of software you want just by talking. It's just.

Yeah.

Is there anything else happening at Google right now? Any other things that you're working on that you're allowed to talk about?

Yeah, I think, I think the only other thread, and we alluded to it a couple of times, is reasoning model stuff. I think this really does feel like the GPT2 era for these models. Like, there is so much new capability and so much progress being squeezed out of the models in such a short time. And if you look at the releases of the. We released our first reasoning model back in December, right? After the Gemini 2.0 flash moment.

One month later, like staggering like a normal, like six months worth of progress, honestly on like a bunch of the benchmarks that, that matter for this stuff. And if you look at the charts, we released an updated version like January 21st a couple of weeks ago. And if you look at the chart, it's like literally linear progress up and to the right across a bunch of the. And like it's, it's just crazy to think that a month ago the narrative was like, the models are hitting a wall, there's no more progress to be had. And it is, it's funny like how nuance some of the conversation lacks because it like these innovations are like deeply intertwined. I was, I was having a conversation earlier today about like long context and how long context is actually like a fundamental enabler of the reasoning models. Because like by themselves, the long context innovation, like it, the model's okay at like pulling out certain pieces of information. Like it can do needle and a haystack. Well, it can find a couple of things and a million tokens. But it's really hard for the models to attend to the context of like, you know, find a hundred things in this million token context window example. Reasoning models are the unlock for this because the reasoning models, the model can really just like continue to go through that process and think through all the content and like really do the due diligence. It's almost uncanny how similar it is to like, how would you go about this? Like I couldn't watch a two hour movie and then if you quiz me on a hundred random little things as part of it, like I'm not going to get those things right. Like it's going to be really hard to do that. But if you let me go through the movie and you know, watching an imovie and like add little inserts and like clip things and cut things and do all this stuff, like I'd be able to find those things if you ask me those questions again. And it feels like that's kind of what reasoning is doing, is actually being able to do that. So I think we're super early in this progress and it's going to be a lot of fun to see both the progress continue for us, but also just like again through this narrative of how all this innovation trickles into the hands of people who are building stuff and like there's going to be a ton of new products that get built. Like maybe text to app just like gets 10x better in the next year because of reasoning Models like that's possible. Which is, which is just crazy to think about.

Yeah, yeah. The reasoning models, they almost feel like they're like double checking, triple checking themselves in real time. Right. It'll be like sort of starting to give a response and be like, let me actually double check what I just said. And it like, and when it comes to coding, that seems like it's the ideal use case almost. Right. Because it can almost look back at its code and be like, oh, I think I made a mistake there and sort of continually fix its code before it finally even gives you an output, which I've just found to be really, really cool. But also from what I understand, that's where a lot of the cost in the future is going to come in. The cost of the inference to do all of this, like, analysis in real time as it's, you know, giving its output.

Yeah. And these models. The other thing to think about, which is interesting is we're seeing all of this progress with the reasoning models and they are doing like the most naive version of thinking like they really are. Like, if you were to think about like the human example of this, like you're sort of sitting in a box, like thinking to yourself, like, you have no interaction with the outside world. You're not able to like test your hypothesis, use a calculator, search the Internet, any of those things. And like you have to sort of form your thoughts independent of the outside world and you imagine what starts to happen when you give these things tools. And like, it really does feel like that's the, that's the agentic future that we've been promised is like all of these tools in a sandbox interacting with the model, letting it sort of have that feedback loop of trying things and seeing what doesn't work. So I'm. Yeah, I couldn't be more excited about that.

Yeah, no, that, that's really interesting to think about. So like right now it's just sort of thinking through things and sort of double checking itself. But in the future it could actually be working with other tools that could also like, assist in the double checking and things like that and get even smarter in those ways.

Yeah. And I think put a different way, like, to be more extreme, I think it has to do that. Like, I think the version of the future that we're going towards is like, it's not like we're, we're not going to be able to see the progress, continue to scale unless the models can do that. And again, this goes back to this thread of like, there's lots of hard problems to solve in the world like making it so the models can do that efficiently and like securely and safely have that sort of sandbox to do that type of thinking and work is going to have to happen and it's probably a lot of work that hasn't been solved today.

It's crazy to think that probably soon AI is going to be helping create all those tools as well. So that's what we'll see. Things just go exponential. Right. It's like it already is people with AI or AI itself creating the tools and just feed that back into the system and it's going to be wild how fast things are going to get better.

All the engineers being powered by cursor. I mean it's, that's the, it's crazy. Like it's happening today. Like so many people are. I feel this way for myself. Like I'm. I write more software now than I did when I was a software engineer because I have AI tools and I can do all this crazy stuff.

Yeah. How far do you think we are from like AI actually being able to update its own weights based on conversation so it actually learns based on like new input that it gets through conversations that it has?

Yeah, I think for that we are I think in the small scale example sense like you know, you could probably already do this to a certain extent. I think in like the like real frontier use cases probably far from that. Because like you, if you think about and, and some of the OpenAI operator stuff was talking about this around like you know, the need for having evals of like basically like creating economic value, like actually creating money.

And where we are in that. And like you probably don't want the models to do things that have a high cost today because if they get it wrong it costs you a lot of money. And training frontier models is definitely on the list of things that would cost you a lot of money if you, if you got that wrong. Like you don't want a bunch of training runs that are just wasted compute. Like that's you know, you know, millions of dollars of potential loss money. So, so I think there'll be a human in the driver's seat for those things. Gotcha for a while. But I do think you can sort of accelerate this small scale feedback loop and I think that's why small models matter. Like this innovation that's happening of being able to compress the frontier capabilities down into small models, I think it enables that rapid iteration loop where maybe AI is more a co pilot in that example. Well, cool.

Logan, this has been absolutely amazing. If people want to follow you, is what's, what's the best platform to pay attention to what you're doing and to keep up with what Google and DeepMind are up to?

Yeah, yeah, I'm on Twitter, I'm on LinkedIn, I'm on email, so. However, whichever one of those three is easiest to get a hold of, me would love to chat with folks about Gemini stuff or the like.

Yeah. And you're, you're, you're pretty active over on X slash Twitter, whatever you want to call it. Whenever there's a new, like Google or DeepMind rollout, you're pretty much either tweeting about it or retweeting about it. So very, very good resource to keep up with what's going on in the world of AI with Google. And Logan, thank you so much for hanging out again with us today. I'm sure we'll have you back in the future if you want, but this, this has been an absolutely fascinating conversation. So thanks again for hanging out.

Yeah, this is a ton of fun. I'll see you both at IO, I hope. I think hopefully we'll get, we'll get the gang back together and we'll, we'll spend time in person, hopefully at IO. It's going to be fun.

Would love to do it. Thanks.