From Models to Agentic Applications with Sam Witteveen

Here's some cool models, but here are the products that we made with those models. I'm very frustrated that a lot of the big providers, starting with OpenAI anthropic today and also Google are deciding not to give the raw thinking tokens out of these reasoning models. What we're hearing from the teams behind the scenes, not only are they releasing it as they're announcing it, they're literally releasing it a day or two after they finish it. You've touched the nerve, right? Sets me off. Watch out.

Hey, everyone. Welcome back to the channel. So this week I was at Google I O and I met my great friend Sam. So we were going to be talking about some of the stuff that we saw at Google I O Also some of the model releases this week, especially the cloud this morning. So make sure you watch the video till the end. Hey, Sam, how are you doing?

I'm good. We're here in town for Google I O. Google's big week. A lot of new stuff, a lot of things to talk about. Where do you want to start?

What do you think about it? How do you feel about IO?

It was really impressive this year. I got to say that I think this year was probably more impressive than perhaps the previous two years. I think we really saw this year the sort of fruits of a lot of the groups inside of Google that have been working on things that they kind of perhaps presented in preview, things like Project Astra. So stuff like that we saw in preview last year, but this year it was really cool to see things like that. Literally, Sundar say, well, later today you can open up Gemini Live or you can play with it, you can do stuff. So I think that was a really cool theme.

Yeah. At least we're not waiting for things anymore, right? They are releasing it as they're announcing.

It, literally from what we're hearing from the teams behind the scenes, not only are they releasing it as they're announcing it, they're literally releasing it sort of like a day or two after they finish it. Sunda made a really interesting comment at the start of the keynote that you're just as likely now to see a model drop on sort of an average Tuesday in the middle of March as you are to see them hold it for Google I O. And I think that that's really interesting because we've certainly seen that happen, that you and I are both testers for the Gemini models and stuff like that. I'm on brand today with my Gemini top. Certainly we've sort of seen that, like over the Past six weeks, month, there's been multiple releases of Gemini. And it did have a lot of people sort of wondering, like, oh, what are they going to release?

Right.

If they've been releasing all these things along the way, what's left kind of thing?

That is true. Well, apart from, like, in all the models that were released, what were your favorite things? Not just the model in general.

So for me, the big idea, you know, that I made a video about yesterday and stuff like that was that this IO was really about sort of models versus products. So I kind of feel not just Google, but across the industry, industry. Over the past two years, everyone's been like, oh, here's our new model. Check out the benchmarks. Check out the, you know, all these sorts of things. And while that's been really cool, it's being focused on very sort of technical stuff for those of us in the industry, that kind of thing. I felt that with this IO it moved beyond that to like, okay, here's some cool models, but here are the products that we made with those models. And for me, that was something, you know, had already sort of going into it, knowing a lot of the things that they were going to announce. I was still surprised just by, like, how good, you know, the. The products were when you actually saw them, how good the reactions were to those things. And it seemed that, you know, there was a lot of excitement, not just around that we've got a new Gemini Flash model or a new pro model coming, but the products that people are actually building with those.

Did you get to try any of the products?

So I've tried out quite a number of the products already, yes. You know, like, I think they're. They're every. They're like a wide variety of products. Oh, yeah. If you want to sort of break it down, I guess we could start off by talking about the. The sort of Gemini products. So that was like, there's a bunch of different coding things, you know, out. You already made a video about Juuls.

Yeah, Juice.

So what did you think of that?

It's pretty interesting idea. So they say Juice is the asynchronous polling agent. Right, Right. So far we have been using my Cursor or Windsurf. The idea is you're basically babysitting these clues. Right. They will generate code. You need to accept this. Guce is, I think, one of the first, or I would say, like the. There's a new category. Codex is another one. Devin is another one. Right. You give it a task, it's going to go in the background, complete the task and then just tell you that, okay, here's a peer. Right. So I think this is what the agentic coding system was supposed to be. Right. And we're actually seeing that. So this is pretty neat to actually see the realization of what was promised.

Yeah, I do think it was very interesting that sort of we had OpenAI try to preempt everyone last Friday. Last year they did it on the Monday before. I o trying to steal Google's thunder. This week it was the Friday before, I guess, because we also had Microsoft do do big announcements on Monday. I agree that like clearly a lot of the, you know, a lot of these frontier labs have been working on this idea of agentic coding, of, you know, coming up with some kind of software to do it and stuff like that. OpenAI rolled theirs out on Friday. Google announced Jules at IO. I. You know, it's going to be really interesting to see where they go with this. I think at the moment it's still kind of limited into what you can do with it. The number of agents you can have going, the number of calls and stuff like that. So it's perhaps a little bit still of a tester kind of.

Yeah, I think it's like more of early iteration. Right. And hopefully they're going to get a feedback from the community and improve on that. Right. But you alluded to a number of model releases this week. Seems to be pretty crazy.

Totally. Yeah. Maybe we'll come back and talk about the other people's models. But like just at IO we had, you know, just model wise, we had a new version of Gemini Flash 2.5. Right. Which is, if I think about it, it's now the second or third public iteration of this model to come out. Going right back to sort of December I think it was back then they were calling it the thinking model. It's really interesting that, you know, so Demes announced that, okay, this is going to go ga probably in June. The interesting thing is that just the number of iterations that Google's done on this, that was really fascinating. On top of that, they announced a new version of Gemini Pro, of Gemini 2.5 Pro with DeepThink. So this is sort of your, you know, even more of the test time compute, perhaps in parallel, perhaps doing a bunch of different things. We had Gemini Diffusion, which is really interesting.

Excited about that.

What did you think of that?

Yeah, that's a really fascinating idea. I think that will open up. The first thing is it's going to be extremely fast. Right.

And I think it is, yeah. Like, I've been playing with it and you can get up to sort of like, you know, 11, 1200 tokens per second. You're sort of averaging, I think, 8, 900. It is insanely fast. It's not the smartest model out of all of them. And Google was quite honest with that, that, hey, this is an experimen. We're not even sure where people are going to use this. But is this something that's really valuable to the developer community? I certainly think it will be. I think it's going to be really big for agents. We're going to see people use this because you can imagine one of the things that people haven't really talked about it is that you can imagine combining this with the code execution server side, where it can literally in a split second decide that, oh, I need to write the code, write the code, run the code, return the response to the user in sort of like seconds or fractions of seconds really, for doing that, I think that one's going to be really big to see as well. So there definitely was a lot going on with the Gemini models.

Yeah. And I think even it kind of shows that Google's philosophy is changing. Right. They're not just waiting for a model that are fully developed. Right. They're really experimenting with things and especially like some of the demos, like Live demos. That was, I think, an interesting thing to see.

Yeah, we certainly saw like, Gemini being used for a whole bunch of different things. We talked about the project Astra. So the other sort of Gemini models that got announced was sort of a new version of the Live API. Originally we used to refer to it as the Biday API, the bidirectional API. This now allows people to just speak to Gemini in real time and not only that, have video feeds. And I think that's going to be huge. Right. And going back to my point earlier on about, you know, that it wasn't just models. So they showed that, but then they showed that, like, okay, that's going to make its way into the Gemini Live app and that's going to be free on Android and free on iOS for people to use. And I think, like, while a lot of people were kind of wowed by it there, my guess is over the next couple months we're going to see just people just start using this thing. It's literally going to be like, anytime you have any sort of issue or something like that, people are just going to pull this out, show it to it, say, hey, help me here, or hey, they showed some really nice examples of things like where Gemini can be incorporated into search. Again with these kind of things. Just pull out your phone, take a picture of these sneakers and say hey, go and find the best price on the web for this kind of thing for me and tell me what it was. And, and the thing was, you know that like you can see that while that's a really playful thing, we, you know, we took a picture of Sundaz sneakers and I won't tell you how much they cost but if you go and sort of do some research, pretty nice. But they also things like, you know, you're just going to see people using it for lots of different things. Everything from kids homework, you know, through to work uses to a whole bunch of these different things in that app and then also sort of making its way to search. So search was one where clearly the Gemini 2.5 models are now starting to make their way into search. Google obviously is known for search. Everyone's been sort of saying for the past year, oh, all these other companies are out innovating. Google, I gotta say Google showed that actually they've, you know, in search they're still probably the number one player. Right. Not just, not just on volume but on innovation as well.

And even like the like I'm really impressed by the amount of progress that they have made right. From the initial bar to Gemini 1 or Gemini 1.4. 1.4. And it's been fascinating to see. Right now I do want to ask you something. I think we both were at next they were talking about AI agents. Right. They even had a counterfeit. How many times those two words were used? This time we didn't hear about agents.

So definitely there was a lot less talk about that. I did find that really interesting. Suddenly Google, it wasn't like that they had no announcements about that. Adk, the agent development kit which we talked about here in Vegas, that is now up to 1.0. There's been a bunch of changes for that. There are a number of new built in tools for that framework that they've release and that are coming, you know, some of those are still coming down the pipeline. That brings us back to another use of the Gemini model that we didn't talk about was Mariner. Right. So that was, I guess the agent story here was more of a consumer story than sort of perhaps even a developer story.

Yeah. Converting it to basically a product. Right?

Yeah. So it looks like Project Mariner, which again sort of was announced last year as a preview thing coming later in the Year later on, got released as a Chrome extension just for testers. Now it looks like, you know, this thing is coming out. You can get it on the Ultra package or whatever that's going. We can talk about the video models in a sec. But for me, the interesting thing there was that clearly this is the future, Right. You know, clearly this is going to be built into people's browsers, built into their phones, and we're going to see this whole sort of thing of, you know, getting these, these, whether you want to call them agents or assistants, probably in some ways, assistance is perhaps a better word, getting these assistants to actually just go out there and do stuff for you. Now that, that's, I think a really big leap is that people have talked a lot about, you know, learning from these things or getting them to, you know, sort of aggregate information for you, but now they can, you know, in the going forward, they're going to be actually going out there doing their own searches, getting things, taking actions that you want it to take. And I think that's really fascinating.

Yeah, I think like, we as consumers or users will have to really adopt this new paradigm, right?

Yeah, like, you know, it'll take a bit of time for sure. And we, we still sort of yet to see, you know, it wasn't like they announced that this is 100% in Chrome today release kind of thing. So there'll be some, you know, iterations probably before it comes out. But the thing, one of the big takeaways that I would say from that is that they probably couldn't do that last year. I could tell you they couldn't do that last year because the models just weren't at the right level then. Right. So for the fact to get those models to this, this point now enables all these products. Yeah. And I think that's one of the things where there was a lot of ideas around products, you know, last year and, and before that, but the models just weren't good enough. It wasn't that Google's models weren't good enough. It was everyone in the, you know, in the, the industry. And so, you know, over the past year, people have improved their models, you know, all the different providers and stuff like that, and that's enabled, you know, all these new things and we haven't even touched like Veo or Image I actually.

So everybody's talking about Vo. I don't think we need to talk about that because that's really fascinating. Right. I do want to ask you about Gemma. What did you think about that? I think that that's a very important piece which is not getting a lot.

Okay, so this was kind of delegated to the developer keynote in the afternoon. Certainly a very interesting model. Again, probably more interesting not because of any of the specs about it, but because of what you can do with it. So the Gemma 3 release was a couple of months back that had everything from sort of a small 1B text model right up to a 27B multimodal model that could only do images and text. So interestingly, they released these sort of mobile versions or the nano versions. This is what the 3N is, and it can do audio and images. And we've seen like demos of where people are starting to make the sort of like the Gemini Live app fully local on a phone. Right. That's kind of crazy, right?

Yeah. You would be able to run all these things on your mobile phone with like your privacy is going to be preserved. Right. It can be pretty interesting to see.

Yeah. And I think one of the cool things with that, you know, talking to some of the Googlers behind it, you know, that they even don't know exactly all the use cases they're interested to see. Okay, where will developers use this and how will developers use this? And I think that's going to be really fascinating. The obvious one is stick it in a phone and let people use it on their phone. But my guess is we'll probably see really cool things that people are going to develop for their laptop or for their desktop for using this, where it can just watch you as you're using your screen or something and can comment on it. You could even sort of customize these models to do a little bit of the computer use stuff where, you know. So all of that is fascinating. It sounds like they're just getting started with this. You know, there are a whole bunch of technical things around these models that are really interesting about how you can resize them, you know, and from what we're hearing, that this is just the stuff. This is just the beginning for this.

Some more cool stuff are going to be coming in. What do you think about model capabilities? For me, it feels like we are able to do more and more with much smaller models now.

Yeah, I think, look, we've definitely crossed this bridge of where ideas were really cool last year, but they just didn't work in the models to now where suddenly, oh yeah, this can work. And it doesn't mean it's always going to work perfectly yet or for every use case. But more and more people are able. Coding is a Huge one. So, like, we talked about Jules, but the other one was Stitch. So where basically you can use these models to generate a design for an app. And I got to think that looking forward, you're projecting out sort of six months from now, a year from now, us as developers, I don't want to use the word competing with, but we're going to see, like, just people just with a good idea be able to come along and build an app. Even though they've never learned to code, they've never learned any of this stuff. I think that's going to happen whether, depending how quick and stuff like that. That's going to be something to see. But it is interesting that Google's kind of planting a flag for some of these apps, you know, now with the Gemini 2.5 model. So that means that they are giving good outputs and stuff like that. A lot of it. I think they want to sort of see, like, okay, how are people going to use this? Yeah, right. We've seen some really nice demos, for example, of people building apps just by talking to it.

Yeah.

Like, hey, I want a mobile app. I want it to be, you know, lime green. I want the corners to be nicely curved or all this sort of stuff.

I think people with ideas are going to be the one that.

Yeah. Like. And, you know, and okay, we won't talk about Veo too much because I guess it's like everywhere. But that, for me was the big takeaway with VO3, right. Was that the model is awesome. Right. So we're talking about a, you know, a model that can not only generate really unbelievable video now, but can also generate the audio, sound effects to go with it, the music to go with it, and the dialog. Right. You know, that. That was. That was cool. But also the whole idea of them, you know, not just. They could have just put that model out. Yeah, right. And that's what would have happened in the past. So in the past, they showed sort of vo, you know, basically with the model. This year, they put out, like, this new product called Flow. And the idea, you know, of the Flow app is that really, it's taking these models and, you know, allowing you to make, like, a full movie. And I think that's really fascinating. I, you know, I want to see, like, the fifth season of Westworld that never got made.

Right.

You know, someone's going to go out there and make that. And I got to think, like, you can imagine all the sort of Star wars or Star Trek fans or all this. We're just going to See, people make all this sort of stuff now, there may be issues around copyright, you know, that kind of thing. But. But my guess is that you will also see really, you know, co stories that just couldn't get made because it would cost too much to make before. It would have been too technically difficult, that kind of thing. So those are things going to be really interesting to see. We also, we already sort of attended a thing with Darren Aronofsky and Dennis Hasabas talking about this whole stuff. It was really interesting to hear Darren Aronofsky talk about that. You know, it's really about the storytelling and this enables different storytelling that you couldn't do before. So for me, looking at it as a sort of technician and knowing how the model works and you know, like, wow, it's such an impressive feat of technology. But it's really cool to see other people just look at it as like, oh, wow, this is an awesome tool that I could do a movie with or do something like that.

Yeah. And actually I was in a session with demos. He talked about, somebody asked him about the future of the future capabilities that he's going to be looking at. Right. So something he mentioned was very fascinating that today we are building tools, giving those tools to the LLMs to decide, but in future they're going to be looking at these LLMs and models, building tools on the fly. Right. What do you think about that idea?

So in many ways, if you look at it, that's kind of what I was talking about with the diffusion Gemma, and code execution. That thing can write code so quickly that if it's got the ability to execute the code, it could just literally decide, right, I'm going to ping this API or I'm going to use this mcp, write the code, do it. If it doesn't work, update the code, do it again, all in the matter of a second for doing this. So, yeah, I think he's totally right. I don't even see that as being a hugely future sort of thing. Right. You know, I think at least the beginnings of it, you know, certainly going forward we may see the bet, you know, the more intelligent models, your 2.5 pros, those kind of models get, you know, really good at sort of working out perhaps sort of scientific tools that, you know, that, that we haven't even thought about yet. Like, that's going to be really interesting.

So. So that means, I guess we will not really need one model for everything, or we're going to have multiple different models with different capabilities doing different things.

It could Be that or it could just be more sort of like the one big model with lots of different prompts or lots of different agents running the definitely the diffusion one. That speed of that it just inherently gives it properties that your 2.5 pro with deepthink that can think for tens of minutes, hours or whatever it is, you know, is going to be quite different. I do think though that this, this whole idea, you know, of these things, creating their tools and stuff like that, giving them, you know, a sort of sandbox where they can run code themselves so it's not dangerous to anyone outside of. That's going to generate a lot of interesting things. You know, we saw just, I think it was last week DeepMind announced Alpha Evolved, this idea of having a model or a set of agents generate ideas, having ways to test those ideas and then update that loop and then just spin that loop faster and faster. And I know talking to a lot of the researchers that they see that as being sort of the path to discovering the cure for cancer and that being the path to discovering new physics issues and things like that.

Yeah, a lot is happening right in Feed and I think we're going to see more and more innovative ideas. What do you think about all the different releases that we saw this week? Maybe even last week?

Right, yeah. Like we kicked it off. I flew into San Francisco last Wednesday. I had like a day and a bit to get over jet lag. And then sure enough, Friday morning, OpenAI was kicking it off with the Codex release. On Monday we had Microsoft with the Microsoft build. Any new copilot stuff, Obviously, Tuesday, Wednesday has been mostly Google and Google I O. We've also had just literally maybe what, an hour before we were recording this Claude anthropic dropping Claude4 it certainly everyone seems to be wanting to jump in on that. No one wants to get left behind.

That is true, but give us some time, right?

Exactly. For you and I trying to cover it or evaluate it and you know, make a video about it or something, it's. It's frustrating. It would be nice if they just spaced them out. So one a week for the next four weeks. Unfortunately, that's not how it's going to work.

Did you get a chance to look at Cloud 4?

I didn't play Quick Play with Cloud Floor when it came out and look at some of the things that they announced. I thought there's some really interesting things in there. One, they also announced that whole idea of code execution. So AI Studios had that for quite a while now. Most people haven't Talked about. I know you made a video about it. I made a video about it a while back. I remember you talking about when people were talking about the whole strawberry, how many Rs in strawberry thing. You got around it by using the code execution.

Right.

So it's good to see that Anthropic has also adopted that. They've also adopted some other tools. One of the things we saw at I O on Tuesday was that the Gemini API is also going to basically support MCPS and Claude. The Anthropic API now is also doing that and it looks like they're doing that server side. That is really interesting to see where that goes.

So something which I'm noticing is there seems to be convergence in capabilities and every provider is basically doing the same thing now.

Yeah, I think it's not just convergence in capabilities of the models, but convergence in the sort of add ons. If you think of the add ons as being search being, you know, code execution being some kind of tool use or MCP kind of thing, pretty much everyone's got that now. You know, like I think OpenAI was the first one to have code execution quite a while back. Google introduced it, you know, again a while back. So yeah, like there is this sort of feeling that a lot of the add ons, as I'm calling them, are kind of any, at least the obvious ones. Everyone's going, oh yeah, we need to have that. Right? Oh yeah, we should have that. You know, as well that, you know, perhaps the sort of other one would be the sort of computer use or spinning up a browser in the cloud kind of thing. Yeah, they're really, they. We are seeing some kind of convergence around some of these ideas. Yes.

So something which I personally have strong opinions and I think you also have. There's also this convergence on how they're enabling grisly traces. Right. So now they're summarizing it rather than giving us like the actual drawer.

Okay, so you've touched the nerve. Right. It sets me off. Watch out. Yeah, I'm very frustrated that a lot of the big providers, starting with OpenAI right through to it looks like Anthropic today. And also also Google along the way are deciding not to give the raw thinking tokens out of these reasoning models, but to give a summary of it. And they're trying to claim that, oh, the summary is kind of, it's nicer, it's more useful and stuff like that, but it's nowhere near in depth that you actually have from the real model. One of the things that I found so impressive about Gemini 2.5 are those raw thinking tokens. And unfortunately, now they're gone.