Ex-OpenAI Insider on AGI, DeepMind & AI | Logan Kilpatrick

Let's welcome Logan Kilpatrick. He was building lunar rover software at NASA and then training ML models at Apple. He then went on to lead developer relations at OpenAI early.

OpenAI was an incredible experience. People who were at OpenAI at that time, some of the most incredible people that I think I'll ever have a chance to work with. The cost curve of AI going down into the right over the last three years on a separate independent curve of this is like consumer understanding and awareness of AI is like going up into the right. And also in parallel to that consumer, consumer willingness to pay for AI products also going up into the right, like, are willing to spend their money, like if this thing can actually provide hundreds of dollars of value to them. Like, there are truly companies that seem like we're going to build a general purpose assistant powered by AI and like it's going to do everything for you. I think there's a lot of room between these two things. I think because of the number of models there are, there's like actually a short to medium term edges because like, eventually if it's really that great of an opportunity, everyone will figure it out. But like there are edges at the model layer that you can build into your products and services. I think about in the context of Gemini, there's a lot of really interesting product experiences that could be built because of long contacts that like the conventional wisdom is not to use long context to solve some problem. And thus for you as a founder, there's potentially an edge to be found there, which I think is really interesting.

Hey guys, it's George Bandiran. Welcome back to the Gentex podcast. Today we're going to hear about what's happening at Google DeepMind. It's going to be an exciting episode. My guest graduated from Harvard with a bachelor's in computer science and a master's in digital media design. In his early career he was building lunar rovers, software and NASA and then training ML models at Apple. He then went on to lead developer relations at OpenAI. That must have been an exciting time. We'll talk about that. And now he's leading the world's developers and helping them build with Gemini over at Google DeepMind. Lots to dig into here. Let's welcome Logan Kilpatrick. Welcome, Logan. How are you?

I'm doing great, George. Thanks for having me on. I'm excited to talk about all things AI.

Awesome. Awesome. So before we get into that, maybe we could back up. Tell me a little bit about yourself and your journey growing up in Chicago. Studying at Harvard, your early days at NASA and Apple, what was that like? And what sparked your curiosity for coding and building things for.

For coding specifically? I. I saw Flappy Bird. I don't know if folks remember Flappy Bird, but it was this like, very simple mobile app, at least in my memory, like the earliest viral digital moments that sort of captured the imagination of the world and the news and et cetera, et cetera. Um, and I got really interested in like, how, how can I make my own version of Flappy Bird? So I started learning how to program and failed miserably at it and ultimately decided I wanted to study computer science and then got lucky a bunch of times, which sort of translated into me sort of falling into the work that I was doing at NASA, doing sort of early reinforcement learning work for lunar rovers to help find like, optimal traverses, and then ultimately training computer vision models, becoming a software engineer, becoming a machine learning engineer. Ended up at a few startups and then like the world of AI and LLMs exploded and was. Was at the right place at the right time and been having a ton of fun since then.

That journey led you to OpenAI. So tell us what that was like. Paint the picture of when you got there. How far along were we? Was it just you and Sam or. I think you were there pretty early. So yeah, share a little bit more about that journey and some of the things you did there.

Yeah, early OpenAI was an incredible experience. I think I joined at the end of 2022. Bunch of reasons. I just moved out of California when I started interviewing, et cetera, et cetera. So that by the time I actually fully joined, my first day was OpenAI.

The launch of ChatGPT.

Yeah, the launch of ChatGPT. And I remember that day sitting, sitting in a training, talking about security and what we were doing to secure the future of AI, etc. Etc. And how critically important that was for the company and looking over the window and looking out the window and seeing Elon walking in with his, with his kid and like his whole like, you know, posse of people. And it was like that type of craziness from an experience perspective. Like the entire time I was there, it was like there was always something crazy going on, but really, actually it was the things that stand out the most to me were the people who were at OpenAI at that time, when it was around 200 people, to some of the most incredible people that I think I'll ever have a chance to work with. I think in large part because of their conviction that this direction worked and like, I imagine actually this was very similar for like the folks early at DeepMind is like sort of the, the organization that started actually this interest in the AI space. But a lot of, you know, the folks who joined OpenAI after it was like obviously successful of the thing is just like somewhat a lot of still incredible people in that cohort. But just like getting to know the folks who like truly believed in the direction that OpenAI was going in when like it wasn't super clear. And like, even to me joining at the time that I did at the end of 2022, like the direction was not super clear. It's not like I had full understanding and appreciation for what the scaling laws were going to look like.

It was a nonprofit. It was not what we know of it today for sure.

Yeah, it looked like a very different place and I think like the, at the end of my time at OpenAI, it also looked like a very different place. And I think like for me it was leaving. Leaving OpenAI and coming to Google was like a chance to go 0 to 1 again and help build a platform for develop and get them excited about Gemini and do that work that I lived through at OpenAI for the last year and a half now at Google instead. And it's been a fun ride so far.

Yeah. So before we get to Google, so you were leading the developer relations at OpenAI. I assume that that means everything that goes under the APIs. So all the developers that want to use the APIs, making sure the APIs are available, working the support core. Is it that whole branch of everything that goes under the API kind of business of OpenAI? Is that what we're talking about here?

Yeah, it was, you know, the, the fun thing about being at a small startup is like you get to wear like not just many hats, but all the hats. So for me it was everything from literally doing customer support to writing developer documentations, to going and meeting with world governments on behalf of OpenAI to, you know, shipping code as part of ChatGPT and the, the platform.OpenAI.com, which is sort of the developer platform for folks, to helping design APIs. And like that's the thing that is, you know, if I can reflect on my own experience and what gets me excited as a human, it's like that level of breadth and I think that's actually like very similar for a lot of folks at early startups and stuff like that. And for the founder Persona is like being able to be very horizontal across everything. And so yeah, got to do all of that. It was actually much more like I formally transitioned when I came to Google to a product role, but my job at OpenAI was actually much more product focused than traditional developer relations. And part of this was because OpenAI didn't have like a get developers interested, excited about AI problem, which they actually did have when I was originally being hired, which was like, hey, we have LLMs. You know, the only people using LLMs in 2022 were Copyai and Jasper. And like, we, we know this technology is going to do more than just copywriting. How do we sort of educate the world and get them excited about this? And like, by the time I joined, that was actually no longer the problem. It became more like, how do we build the right product experience for developers? So got to do that.

And so then you touched on like GPT3, GPT4, I would imagine, Whisper, Dall E. So what were some of the key lessons that you learned during that time, either in terms of developer relations specifically or more broadly about AI and kind of where this field is going?

Yeah, a couple of things, the two that stand out in my mind right now. Greg Rockman had a good quote which at one point was that the best developer experience is really good models. And it actually, you know, the OpenAI developer experience is sort of pointed to as this, like, you know, the gold standard for this. And it's interesting, like in many cases how much of the focus was on shipping new models and capabilities rather than like the art of API design, which I think has become like more important for OpenAI over time, as, you know, the stakes got higher, etc. Etc. But really that focus on, like, if you have great models and they're differentiated, that's the thing that matters. And like, developers will jump through any and all hoops you put in front of them as long as your technology is good enough. Which was sort of an interesting reflection of the place that we're in today where sort of there's a lot less delta between model providers along certain axes and dimensions. I think it's interesting to reflect on where the value creation has been over the last few years. I think if you, this narrative has stuck with me that, you know, AI is a commodity. There's like all the, like the, you know, rappers have no value creation and both of these things actually being true at the same time of like, this thing is a commodity, but also there's no value created by the wrappers. And then if you look at sort of the narrative in the last two months, it's like back to AI being a commodity and all the value is actually going to the companies that are building at the application layer and there's no value to be created at the, at the model layer. And like, the truth is obviously something in between those two. But it is really interesting to see how, like, the conventional wisdom shifts over time from, like, where builders should be thinking about creating value. And I think part of this is just people are so fast to make this assessment of, of what's happening that it, like, doesn't give founders and builders, like, enough time to actually build a product experience that's useful. And I think that's actually been, to me, the story of the last two years, which is like, not that AI is not this great thing and this incredible product and innovation for people building stuff. It's actually that to use that and wield it in the right way, it takes time. It doesn't matter how much money you have. Money in some cases can accelerate some of these things, but really it actually just takes a lot of human effort in order to build a really great product experience and the fun. And this also goes back to, like, you know, people are always asking, what's the advice? What should I be building? Yada yada, yada. Like, the fundamentals of what it means to build a great product actually haven't changed in the last two, three years. Like, it's always been the same things as far as, like, what it means to be successful. Technology is, of course changing, but, like, the actual fundamentals don't change. And I think not enough people appreciate that. They think there's some silver AI bullet that's just going to, like, make, you know, make it so they don't have to do all the hard work of building a great product. And I don't think that's proven to have been right so far.

Yeah, so, so let's drill into that a little bit because you have some interesting perspective. You, you have been part of these companies, OpenAI and now Google, DeepMind, working on Gemini. You're also an investor. And so you, you, you have friends that are building stuff. So, so I know you bring a different, a number of different perspectives. So what do you mean when you say what you said? Can you go, let's peel that onion a couple layers deeper and provide the audience a little more insight as to what you're talking about?

Yeah, I think the best, like on the nose example of this is cursor. Like, I think if you look at what was cursor a year and a half ago today. Yeah, it was useful and like, yeah, it was a cool product and like I was using Cursor a year and a half ago. Was it this thing that like went, you know, was it at that point the company slash product that had the ability to go from fastest company ever from a million to 100 million ARR. Certainly not like they didn't, they didn't earn the right to, to do that because they had yet to build the product to actually do that. And I think it's taken, I think for a lot of these companies, like multiple years of product work in order to like put themselves into the position to be successful. And I think a lot of this is actually conflated by the ChatGPT moment, which was this wild explosion in growth. And then 1.5 flash and flash, for folks who aren't familiar as our sort of most popular Gemini model. And really I think that the root of why it's become so popular is it's this really elegant trade off between cost and performance. And I think all of it boils down to how can we give developers the most amount of performance while eliminating the cost? And I think for founders this has become. And like for people who are building, I think like my worldview is actually like one of the inherent constraints of how much AI products are created in the world is like bounded by the cost of AI. So like as the cost of AI continues to go down, the amount of interesting AI products in the world like exponentially goes up because all of a sudden, you know, you don't have to worry about the really tricky economics of like, how do you actually put this product into your user's hands because you know, you're paying for every single token and tokens are expensive, etc. Etc. And GPT4, the orig original cost was like $30 per million tokens or something like that. And Today with Gemini 2.0 flash, it's 10 cents and the model's better than the original GPT4 model and it has all these other capabilities. So it's like this huge success story for people who want to build.

That started this was the first one, was a little concerned to release the technology. OpenAI wasn't, they released it and Google's like, oh my God. And then there was Bart. Bart was sent to the corner and replaced by Gemini. And so where are we at today? Where are we going? So yeah, give us a little bit of that timeline.

I think for me joining Google was again this opportunity to go 0 to 1 again. And I think I'D sort of lived through as much as Google's a big company and it doesn't look on the outside like it's the zero to one experience. I think Gemini and all the AI stuff happening inside of Google really is this zero to one experience across the developer surfaces and the consumer surfaces. And I really wanted to be able to be a part of that again and sort of take the lessons that I'd learned helping OpenAI do it and do it for someone else. And I think just separately as someone who likes building stuff, I think people having a lot of options in the world of great platforms from a developer perspective to build with is a very positive outcome for the world. So I think that was another angle of my, the excitement for me joining Google and joined Google April of 2024, so almost a year ago now. And yeah, got to sort of, you know, took over AI Studio as a product surface which was, had, had been through its own sort of transitions originally from a product called MakerSuite designed for, you know, sort of exploring AI to ultimately transitioning into this sort of fast path for developers to experience the latest Gemini models and also build using the Gemini API. And we've spent the last 10 months now doing two core things which is one, building a great products platform in order to let developers access the latest models and also scale with them and two, building sort of trust with the ecosystem around the Gemini models. And I think a lot of this has been just, you know, again what people see me doing externally which is like what looks like developer relations to a certain extent but like a lot of education about like where we are with Gemini and it's been, yeah, it's been a ton of fun. I think like it really has been the true extent of the zero to one experience. I get to wear lots of different hats which I really love like across the entire stack of DevRel, product engineering, marketing, comms, you know, PR, doing all types of things like that has been, has been a lot of fun and Google has all of the. The nice thing is like, you know, you can sort of wear those hats and then when the time comes, hand it off to someone incredible who can sort of run with whatever that, that thing is because there's a really, you know, talent dense network of people at Google across all these domains.

So basically let's talk about it from a few different Personas, right? Let's talk about it as a. I'm an AI found they're building in the application layer. When would I want to use Gemini versus maybe some other tools I'm a B2B customer, I'm a Google, you know, G Suite customer. That seems like a natural. And then I'm guessing if I'm a Microsoft customer, this may not be the best tool set for me. But help us walk through those three Personas and what the benefits and where, you know, your platform would be like the slam dunk best pick for each of those. What use cases are you excelling in? Where's the differentiation value add in each of those use cases.

Yeah, so the, the arc of this story for me has been there's like a few different chapters of this. I think chapter one of the Gemini story was launching a fully native multimodal model with a million token context window. And I think like the two pieces of that story that were important is long context and multimodal. And I think at the time that Gemini launched, neither of those things were possible and was really the sort of first step for Google into this like, new generative AI area. Of course, Google's had AI in production for decades with search and every other product service that we have, YouTube, etc. I think at that point the reception was positive. I think the first time we really started to see the sort of floodgates open as far as traction from a developer perspective was Google IE last year where we announced Gemini 1.5 Flash and Flash, for folks who aren't familiar as our most popular Gemini model. And really I think the root of why it's become so popular is it's this really elegant trade off between cost and performance. And I think all of it boils down to how can we give developers the most amount of performance while eliminating the cost. And I think for founders this has become, and like, for people who are building, I think like my worldview is actually like one of the inherent constraints of how much AI products are created in the world is like bounded by the cost of AI. So like, as the cost of AI continues to go down, the amount of interesting AI products in the world like exponentially goes up because all of a sudden, you know, you don't have to worry about the really tricky economics of like, how do you actually put this product into your user's hands because you know you're paying for every single token and tokens are expensive, etc. Etc. And GPT4, the original cost was like $30 per million tokens or something like that. And today with Gemini 2.0 flash, it's 10 cents and the model's better than the original GPT4 model. And it has all these other capabilities. So it's like this huge success story for people who want to build. And I think that continues to be our sweet spot. So for like developers today, you know, you can see on the order of like 20 to 30x cost savings with a model that's like actually better quality and has a bunch of differentiated features like long context, like native search, we showcased in December, native output multimodal, so you can both generate images and generate audio natively. With Gemini, which will hopefully roll out more broadly soon, which I'm super excited about for people to get their hands on. So like, as, as the model crank keeps turning, I think like the story and the value for developers continues to go up, which is, which has been awesome to see happen.

So I want to make sure I understand and appreciate what you're saying. So, so you're not coming on here saying, hey, look, you know, I think, you know, for these specific use cases, I think we're better than the other guys. You're saying that you believe that generally speaking, horizontally across the board, you guys feel like you have a good enough models, better at certain things, maybe a little, a little less than and other things. But at, I think you said a 20x less cost point. I mean, that's dramatic. So what you're saying is we think that everybody should take a look at us when they're building, whether it's just a normal AI system, whether it's a gentic, whether it's horizontal, whether it's vertical. Is that, is that kind of what you're saying today?

Yeah, 100%. And I think there's like a couple of domains and I think like code is the most publicly sort of debated one of this where like, we've made a ton of progress and like having the world's best coding model is like extremely top of mind for us. There are other models that have like, better coding performance today. I think for those models, it's like, it's the, it's that cost intelligence trade off, like as you put AI into production that does coding, like, are you willing to, you know, pay the cost of having a few percentage points lower performance in order to get, you know, order of 20x cost saving. And like, I think my, my, you know, conversations with founders and developers is like, in many cases they are willing to do that because they want to put this thing out and scale it, but across, like general horizontal capabilities, I think the Gemini models are specifically 2.0. Flash is at the frontier level, significantly lower cost, which is Google's unique approach to Building this technology playing out. If you look at the vertical integration of hardware, software, compute, all playing out, it's a really interesting story of.

It's the AWS model that was the thing with Amazon in the early days. You could switch and then they kept lowering their own costs and they kept lowering your bill and you're like, wait, no company's ever done this where it's lower to start and they keep lowering the cost and then at some point you stand back and you're like, wow, this is really respectable. They want to dominate the entire market. They're not just trying to milk me for a gross profit on. They know that if they keep reducing costs, we're going to be happy, that we're going to do more and then we're going to tell all of our friends about it. And their plan worked. They ended up, you know, AWS took a large share of the market coming out of nowhere. And so you're saying that that's kind of what the play is here as well with you guys now with your AI model where, because it's vertically integrated and you, you guys think that you're, you know, you believe your models are better, equal or better, but you're not trying to price it on par. You want to be able to provide a significant cost advantage so that you can be able to grab a large piece of the market.

Yeah, I think, I think the story for us has been how can we. Like, it's like the two. You know, I made this the comment to someone the other day of like pulling the rabbit out of that. How do you lower cost while also giving people a better model? And like, that's the story for me of 2.0, of 2.0 flash is like lower blended cost per million tokens while also giving you a stronger model that has like truly frontier capabilities. And the exciting thing for me is like, who benefits from this? It's like developers, because as you look at sort of a bunch of different cost curves is like the cost curve of AI going down into the right over the last three years on a separate independent curve of this is like consumer understanding and awareness of AI is like going up into the right. And also in parallel to that consumer willingness to pay for AI products also going up into the right. Like, people are willing to spend their money, like if this thing can actually provide hundreds of dollars of value to them, like, people are willing to spend the money to do that, which is super interesting. And that's again, the gets me excited for people who are building stuff because like the value is, is being created and, and accruing to those who are actually building stuff.

Gotcha. So, so basically you're saying all AI founders should take a look at it. Certainly G Suite customers, but Microsoft, big Microsoft shops.

You think this is part of the strategy for us right now as well is like the, you know, we have two places where developers like sort of the quote unquote developer Persona can access Gemini if they're trying to build something through Google Cloud and sort of the enterprise side of the business. And that's usually the situation where like if you're a really large enterprise customer, it can be difficult to make a full migration from one cloud to another. That's like a multi year process. And I hope you use Google Cloud because I think it's a great cloud provider. But if you just want. What we're providing is LLMs behind an API. If all you want is Gemini behind an API, that's the service that we're providing for folks. So you don't need to make the commitment of we're going to build our entire cloud infrastructure. If you're a Microsoft customer, for example, you can just get the Gemini models. And I think this is actually the strategy that's playing out across a ton of people who are built like not even just startups, but GitHub Copilot's a great example of this. They have historically been built on just the OpenAI models and within the last few months are now sort of model provider agnostic with anthropic models, with gemini models, with OpenAI models. And it seems like more and more products are sort of going that direction of being, you know, providing all the different options.

Agentic AI is a hot topic right now. From your perspective, where do you see agentic AI as being and going? And is Gemini a model that just plugs into a step or is Google planning some kind of an agentic platform like OpenAI released Operator, which is a little different. But you know, in my mind Google is in a great place to have something that's probably more like a land graph or CREWAI in terms of a platform for its customers to set up these agentic workflows. So if I'm a G drive, if I'm a G suite customer and I want to go, which I am, which we are, and we want to go agentic and we want to, you know, let's just take our investing process, we want to automate some steps of our investment process from stuff that comes in. It could be a Google form or a type form and there's like a Google Doc that gets created with. Right. So do I have to go outside or are you guys building tools for me to be able to run agentic processes within my G Drive and G Suite environment?

Yeah, that's a great question. I actually, I'm not super close to the G Suite side of the business, so someone else might be able to better speak to what the offerings are. It makes a lot of sense. Like I'm hopeful if we don't have something yet, I'm hopeful we'll build something because I'm also been a G Suite customer for many years now across all my email docs, et cetera, et cetera. And there's a huge opportunity to make that product experience great. I know Gemini is built in across G Suite today. It's actually now I think it's included in the G Suite offering is like Gemini and docs, email, etc. Etc.

I got that update a couple weeks ago. That was pretty cool. I've been using it and it's nice to have it built in in a way that's useful.

Yeah, no, a hundred percent. I just don't know if they let you do the like, workflow automation piece, but I think that would make a lot of sense. And from an agentic AI perspective, I think that's the direction that that sort of consumers and businesses are excited about. Like, there's so much activity in this like vertical AI agent market right now that I have to imagine. Yeah, we'll, we'll have some cool stuff there.

How do you define AGI and where we're at in relation to it and, and how we're gonna, not how we're gonna get there, but who's gonna get there in what way. There's so many different approaches and you know, maybe this is a good time to throw in deep seek into the conversation because they inserted themselves into the, into the conversation a few weeks ago and you know, we heard Sam's position on, on that. You know, I don't know if Google's released an official position, but I'm curious what your take is on them, whether inside this context of the AGI question or just independently.

I, I think on the AGI question, AGI is one of those like fun moving targets where like, I think as our, you know, if you, if you had showed some of the stuff that we have today to people five years ago and talked about, talk to them about AGI, they would be like, yep, that thing is AGI for sure. Like, no, no chance that it's not. I think, like, the human expectation curve is this, like, exponential thing where, like, we just get, like, very accustomed to, like, super awesome technology very quickly.

I agree. I mean, we. We passed the Turing test a long time ago. It already can do things more than most humans can. Most systems can do things that most humans can't, at least in terms of figuring stuff out, answering questions, knowing the knowledge agentic is the next step. So I. I agree with you.

I have.

I have it at 2029, and I think it's. It's, you know, we're in 2025, so it feels like it's going to happen sooner to whatever definition we use. If we're using the definition of, you know, a system that's smarter than all humans put together, it's a little hard to know exactly what that means. But assuming that just means that if we all worked on one problem together, then this computer would solve it better and faster than us. I think we're kind of there because there's still a lot of problems we can't figure out. But, you know, when we spend enough time with the right systems, they could figure it out.

I think the thing that I personally want is a system that helps me proactively, helps me make my life better in, like, really, like, meaningful, tangible ways. And I think that's. We haven't gotten to that experience yet. Like, I think, you know, point in time, here's a task. Maybe the models are able to help do the. Some specific thing, but they don't have the generality yet, and they don't have the sort of. I think there's a lot of interface challenges. So, like, AGI is as much as, like, a research problem as it is a product problem, really, which is interesting to think about. Like, there's a lot of, you know, a lot of challenges. And then there's this separate point around, like. And I put something like this out on Twitter. We could have AGI literally today. And, like, if you were, I think, like, people extrapolate what it would mean to have AGI, to, like, what is the actual impact to the world. And, like, you could have AGI today and, like, I could look out my window and, like, everything's gonna look the same. Like, there's no. Like, the world is. Is, like, to move. To move bits on the Internet is, like, actually incredibly easy and efficient and moving atoms in the real world. And, like, changing the real world that's around us is a very. Is a very different problem space. And I think a lot of the, like, proverbial things that people look to from an AGI perspective, like infer this outcome on the real world. And I actually think there's going to be this like massive disconnect between these two things where like we have really incredible digital experiences but. And like hopefully those will in turn, you know, make the physical world that we all live in like a better experience in a bunch of tangible ways. But the, the time horizon between those two is actually pretty reasonably long. Like it's not easy to like, you know, they're building a building across the street from from where I live and like it's going to take, you know, if we had AGI today, like it's still going to take that building pretty much the same amount of time as it is to be built today. Maybe there's like some order of magnitude of efficiency improvements in the planning process or something, but like it's not that substantial. So I think we're a while away from like systems like even longer perhaps than 2029 that will like really fundamentally change the way that we sort of all live and operate in our lives, which I think makes sense and is like a positive outcome for the world from this like slow takeoff perspective of like, you know, not wanting everything to just like suddenly change one morning when you wake up. It'll be good for it to be this linear process. The Deepsea question I think is, has captured a ton of attention. I think it's a super strong team that's done and released a lot of that work, which is awesome to see. I think developers in the ecosystem love open source models, so I'm not surprised by the excitement around that. And I think like the reasoning model situation is also exciting and like that's why we put out our reasoning models in December and back in January as well. And seeing the progress that's happening for us on our side like of where the scaling is happening on a reasoning model perspective, like it really does feel like the GPT2 era for these reasoning models of just like how fast progress is happening and how much, how much improvement you can see with like a pretty reasonable order of magnitude of investment. And I think like this it's a lot harder to like squeeze out those levels of gains from like pre training as an example or even post training at this point. Like really the RL scaling stuff is like the most compute efficient way to scale the models today. But like that will obviously like dry up at a certain point and then people will go back to being excited about pre training and post training or whatever comes next. After. After that.

Yeah, I have a lot of comments on that too. I'm actually going to record a solo podcast on this tomorrow morning, kind of providing a deep dive on my perspectives, the questions I've been asked from investors and from founders on what's happening and what the impact will be and all that good stuff. But speaking about that, speaking about investor and founder perspectives like we talked about earlier, so. So you made some investments, you know a lot of people, you've had some great opportunities. What would be your advice to founders, new founders that are just getting started, you know, anywhere from the ideation phase to pre seed seed all the way. What jumps out at you as you know, either what's, you know, what's going to win, what are some opportunities, what are some trends, what are some red flags or some common mistakes. You still see that surprise you that AI founders are making? Any thoughts on any of those?

First, on the model side, I think there's still. Back to that story. The thread about Greg's advice around the best developer experiences is a great model, I think because of the number of models there are in today's world. There's actually a bunch of. And these are more short to medium term edges because like, eventually if it's really that great of an opportunity, everyone will figure it out. But like there are edges at the model layer that you can build into your products and services. And like, I think about this in the context of Gemini. There's a lot of really interesting product experiences that could be built because of long context that like the conventional wisdom, you know, is not to use long context to solve some problem. And thus for you as a founder, like the, you know, there's potentially an edge to be found there which, which I think is really interesting. So look at all the different models, find the different things they're great at and sort of see if there's an edge to be created there. The second thing is there's this really interesting sort of challenge for I think founders right now in this moment of like it seems like everything is up for grabs. There's this incredible platform shift happening and thus there's like, you know, perhaps the tendency to like try to go really horizontal and like solve this like massive like general problem. And at the same time there's also this like push. And you see this if you look at like YC startup batch data as an example, like very vertical companies of like, we are going to go and tackle this like one very specific thing that like you've never even heard of before but is Like a really big problem for companies somewhere and we're going to use AI agents to do it as an example. And it's like, it's interesting to see this play out where it's, you know, I think founders are like, almost like being asked to choose one of those things. And my take is I think there's something in between and I think it's a little bit harder to define what that experience is where you can create all this value and not go sort of fully horizontal, not go so totally vertical. And like it's, it's worth pushing on that to see if there's something there, if there's like space between, you know, two of those different product verticals.

To.

To see if you can create something there. So I think, I think there's, it's worth pushing on that thread. And then the last one is, but.

Before, before you go to that one. So like, so what would that look like? Because we have this debate a lot. This is definitely a common thing that I've brought up on my panels with Silicon Valley, IVCs and founders, this horizontal versus vertical. And my conclusion is that there's a playbook for each of them and part of it also has to do with timing. So horizontal was great in the beginning, but now already like Sierra's one horizontal customer service along with Maven and a few others. Right. And so you're too late if you're just going to start a new horizontal customer service agentix solution. The vertical ones are also quickly filling up. But if you're going to jump in, go jump in, build the best customer experience agentic solution for a certain vertical, but also show that you can, you know, you have some ability to go, you know, what are the next adjacent markets that you're going to quickly be able to grow into. You're putting forth this notion of something like in the middle. And I'm curious, like what's, what, what does that mean? What does that look like?

Yeah, George, you need to go check me on this to see whether or not this makes sense. Because I, I, I'm sure I spend less time thinking about it than you, I think in the companies. And like I think about this in like my, the investments that I've done. Like there are truly companies that seem like very horizontal where like we're going to build a general purpose assistant powered by AI and like it's going to do everything for you and like it's going to be great and solve all these problems. And then there's like speech pathologists with paperwork and like these are on two very ends of the spectrum and like, all I'm trying to posit is that I think there's, there's a lot of room between these two things and like maybe, and I don't know if this is fully true, like cursor is like perhaps a bad tongue in cheek example of this where like it is general purpose, it's an IDE that solves this thing. But like it's not like they're trying to go after like solving every problem. Like they're not a platform and all this other stuff as well. Like they are a specific tool that is being built to do this thing. And I don't know if they would be considered horizontal in your mind or like what the definition is formally of these companies that are horizontal. But I think there's a lot of interesting product experiences to be built and I think the thing that pushes me to have this perspective is like the models sort of guide me in this direction which is like at the same time as the model wants to, to be really like, it has this really horizontal capability, it also has the ability to be like a domain expert at some very specific thing. And I think like that artifact transcribed towards the company creation process, like makes me think that there's something to be pushed in that space because again, yeah, because the AI models are the ones that are powering this and like they have those capabilities.

Yeah, it's interesting. It kind of reminds me of when Ash Maria, who wrote the book not, not the Lean Startup but running Lean and scaling Lean and I've spent some time with them and one of the things he talks about is don't, don't categorize them in the typical ways your customers or potential customers, your market look, look at, look at what is their job to be done and what is the triggering event that causes them to make a purchase to solve the problem that they have. And so I think that's what came to me when you said, what you said is if we don't focus on a big boil the ocean horizontal and if we don't focus on like a very super tiny vertical that might limit our art.

And I think that continues to be our sweet spot. So for like developers today, you know, you can see on the order of like 20 to 30x cost savings with a model that's like actually better quality and has a bunch of differentiated features like long context, like native search, we showcased in December, native output multimodal. So you can both generate images and generate audio natively with Gemini, which will hopefully roll out more broadly soon, which I'm super excited about for people to get their hands on. So, like, as the model crank keeps turning, I think the story and the value for developers continues to go up, which is, which has been awesome to see happen.

So I want to make sure I understand and appreciate what you're saying. So. So you're not coming on here saying, hey, look, you know, I think, you know, for these specific use cases, I think we're better than the other guys. You're saying that you believe that generally speaking, horizontally across the board, you guys feel like you have a good enough models, better at certain things, maybe a little, a little less than other things. But at, I think you said a 20x less cost point. I mean, that's dramatic. So what you're saying is we think that everybody should take a look at us when they're building, whether it's just a normal AI system, whether it's agentic, whether it's horizontal, whether it's vertical. Is that kind of what you're saying today?

Yeah, 100%. And I think there's a couple of domains and I think code is the most publicly sort of debated one of this where, like, we've made a ton of progress and like, having the world's best coding model is like, extremely top of mind for us. There are other models that have like, better coding performance today. I think for those models, it's like, it's the, it's that cost intelligence trade off, like as you put AI into production that does coding, like, are you willing to, you know, pay the cost of having a few percentage points lower performance in order to get, you know, order of 20x cost saving? And like, I think my, my, you know, conversations with founders and developers is like, in many cases they are willing to do that because they want to put this thing out and scale it, but across, like general horizontal capabilities. I think the Gemini models are specifically 2.0. Flash is at the frontier level, significantly lower cost, which is, is like, is like Google's unique approach to building this technology playing out. If you look at like the vertical integration of hardware, software, compute, all playing out, it's a really interesting story of.

Like, it's kind of like the AWS model, right? That was the thing with Amazon in the early days is like, you could switch and then they kept lowering their own costs and they kept lowering your bill. You're like, wait, no company's ever done this where it's lower to start and they keep lowering the cost. And then at Some point you stand back and you're like, wow, this is really respectable. They want to dominate the entire market. They're not just trying to milk me for a gross profit on. They know that if they keep reducing costs, we're going to be happy, that we're going to do more and then we're going to tell all of our friends about it. And their plan worked. They ended up, you know, AWS took a large share of the market coming out of nowhere. And so you're saying that that's kind of what the play is here as well with you guys now with your AI model where because it's vertically integrated and you guys think that you're, you know, you believe your models are better, equal or better, but you're not trying to price it on par. You want to be able to provide a significant cost advantage so that you can be able to grab a large piece of the market.

Yeah, I think, I think the story for us has been how can we. Like, it's like the two, you know, I made this the comment to someone the other day of like pulling the rabbit on that how do you lower costs while also giving people a better model? And like, that's the story for me of 2.0, of 2.0 flash is like lower blended cost per million tokens while also giving you a stronger model that has truly frontier capabilities. And the exciting thing for me is who benefits from this? It's like developers, because as you look at sort of a bunch of different cost curves is like the cost curve of AI going down into the right over the last three years on a separate independent curve of this is like consumer understanding and awareness of AI is like going up into the right. And also in parallel to that consumer willingness to pay for AI products also going up into the right. Like people are willing to spend their money. Like if this thing can actually provide hundreds of dollars of value to them, like people are willing to spend the money to do that, which is super interesting. And that's again, the gets me excited for people who are building stuff because like the value is, is being created and accruing to those who are actually building stuff.

Gotcha. So basically you're saying all AI founders should take a look at it. Certainly G Suite customers, but Microsoft, big Microsoft shops.

You think this is part of the strategy for us right now as well, is like the, you know, we have two, two places where we're developers, like sort of the quote unquote developer Persona can access Gemini if they're trying to build something through Google Cloud and sort of the enterprise side of the business. And that's usually the situation where like if you're a really large enterprise customer it can be difficult to like make a full migration from one cloud to another. That's like a multi year process. And like, you know, I hope you use Google Cloud because I think it's a great cloud provider. But like if you just want like what we're providing is LLMs behind an API. If all you want is Gemini behind an API, that's the service that we're providing for folks. So you don't need to make the commitment of we're going to build our entire cloud infrastructure. If you're a Microsoft customer, for example, you can just get the Gemini models. And I think this is actually the strategy that's playing out across a ton of people who are built, not even just startups, but GitHub Copilot's a great example of this. They have historically been built on just the OpenAI models and within the last few months are now sort of model provider agnostic with anthropic models, with gemini models, with OpenAI models. And it seems like more and more products are sort of going that direction of being, you know, providing all the different options.

Agentic AI is a hot topic right now. From your perspective, where do you see agentic AI as being and going? And is Gemini a model that just plugs into a step? Or is Google planning some kind of an agentic platform like OpenAI released Operator, which is a little different. But you know, in my mind Google is in a great place to have something that's probably more like a land graph or CREWAI in terms of a platform for its customers to set up these agentic workflows. So if I'm a G drive, if I'm a G Suite customer and I want to go, which I am, which we are and we want to go agentic and we want to, you know, let's just take our investing process. We want to automate some steps of our investment process from stuff that comes in. It could be a Google form or a type form and there's like a Google Doc that gets created with. Right. So do I have to go outside or are you guys building tools for me to be able to run agentic processes within my G Drive and G Suite environment?

Yeah, that's a great question. I actually, I'm not super close to the G Suite side of the business so I. Someone else might be able to better speak to what the offerings are. It makes a lot of Sense like I'm hopefully we don't have something yet. I'm hopeful we'll build something because I, I'm also been a G Suite customer for, for many years now across all my email docs, et cetera, et cetera. And like there's a huge opportunity to make, make that product experience great. I know Gemini is built in across G Suite today. It's actually now, I think it's included in the G Suite offering is like Gemini and docs, email, et cetera, et cetera.

I got, I got that update a couple weeks ago that was pretty cool. I've been using it and it's nice to have it built in in a way that's useful.

Yeah, no, 100%. I just don't know if they let you do the like workflow automation piece, but I think that would make a lot of sense. And from an agentic AI perspective, I think that's the direction that sort of consumers and businesses are excited about. There's so much activity in this vertical AI agent market right now that I have to imagine, yeah, we'll have some cool stuff there.

How do you define AGI and where we're at in relation to it and how we're going to. Not how we're going to get there, but who's going to get there in what way? There's so many different approaches and you know, maybe this is a good time to throw in deep seek into the conversation because they inserted themselves into the, into the conversation a few weeks ago and you know, we heard Sam's position on, on that. You know, I don't know if Google's released an official position, but I'm curious what your take is on them, whether inside this context of the AGI question or just independently.

I, I think on the AGI question, AGI is one of those like, fun moving targets where like, I think as our, you know, if you, if you had showed some of the stuff that we have today to people five years ago and talked to them about AGI, they would be like, yep, that thing is AGI for sure. Like, no, no chance that it's not. I think like the human expectation curve is this like expectations exponential thing where like we just get like very accustomed to like super awesome technology very quickly.

I agree. I mean we, we passed the Turing test a long time ago. It already can do things more than most humans can. Most systems can do things that most humans can't, at least in terms of figuring stuff out, answering questions, knowing the knowledge agentic is the next step. So I. I agree with you.

I have.

I have it at 2029, and I think it's. It's, you know, we're in 2025, so it feels like it's going to happen sooner. Whatever definition we use, if we're using the definition of, you know, a system that's smarter than all humans put together, it's a little hard to know exactly what that means. But assuming that just means that if we all worked on one problem together, then this computer would solve it better and faster than us. I think we're kind of there because there's still a lot of problems we can't figure out. But, you know, when we spend enough time with the right systems, they could figure it out.

I think the thing that I personally want is a system that helps me proactively, helps me make my life better in, like, really, like, meaningful, tangible ways. And I think that we haven't gotten to that experience yet. Like, I think, you know, point in time, here's a task. Maybe the models are able to help do the. Some specific thing, but they don't have the generality yet, and they don't have the sort of. I think there's a lot of interface challenges. So, like, AGI is as much as, like, a research problem as it is a product problem, really, which is interesting to think about. Like, there's a lot of, you know, a lot of challenges. And then there's this separate point around, like. And I put something like this out on Twitter.

We.

We could have AGI literally today. And, like, if you were, I think, like, people extrapolate what it would mean to have AGI to, like, what is the actual impact to the world. And, like, you could have AGI today and, like, I could look out my window and, like, everything's gonna look the same. Like, there's no. Like, the world is. Is like, to move. To move bits on the Internet is, like, actually incredibly easy and efficient and moving atoms in the real world. And, like, changing the real world that's around us is a very. Is a very different problem space. And I think a lot of the, like, proverbial things that people look to from an AGI perspective, like, infer this outcome on the real world. And I actually think there's going to be this, like, massive disconnect between these two things where, like, we have really incredible digital experiences, but. And, like, hopefully those will in turn, you know, make the physical world that we all live in, like, a better experience in a bunch of tangible ways. But the. The time horizon between those two is actually pretty reasonably long. Like it's not easy to like, you know, they're building a building across the street from from where I live and like it's going to take, you know, if we had AGI today, like it's still going to take that building pretty much the same amount of time as it is to be built today. Maybe there's like some order of magnitude of efficiency improvements in the planning process or something, but like it's not that substantial. So I think we're a while away from like systems like even longer perhaps than 2029 that will like really fundamentally change the way that we sort of all live and operate in our lives, which I think makes sense and is like a positive outcome for the world. From this like slow takeoff perspective of like, you know, not wanting everything to just like suddenly change one morning when you wake up, it'll be good for it to be this linear process. The deep sea question I think is, has captured a ton of attention. I think it's a super strong team that's done and released a lot of that work, which is awesome to see. I think developers in the ecosystem love open source models, so I'm not surprised by the excitement around that. And I think the reasoning model situation is also exciting. That's why we put out our reasoning models in December and back in January as well. And seeing the progress that's happening for us on our side of where the scaling is happening on a reasoning model perspective, it really does feel like the GPT2 era for these reasoning models of just how fast progress is happening and how much, how much improvement you can see with like a pretty reasonable order of magnitude of investment. And I think like this, it's a lot harder to like squeeze out those levels of gains from like pre training as an example, or even post training at this point, like really the RL scaling stuff is like the most compute efficient way to scale the models today. But like that will obviously like dry up at a certain, certain point and then people will go back to being excited about pre training and post training or whatever comes next after. After that.

Yeah, I have a lot of comments on that too. I'm actually going to record a solo podcast on this tomorrow morning, kind of providing a deep dive on my perspectives, the questions I've been asked from investors and from founders on what's happening and what the impact will be and all that good stuff. But speaking about that, speaking about investor and founder perspectives like we talked about earlier, so. So you've also made some investments, you know, a Lot of people, you've had some great opportunities. What would be your advice to founders, new founders that are just getting started, you know, anywhere from the ideation phase to pre seed seed all the way. What jumps out at you as you know, either what's, you know, what's going to win, what are some opportunities, what are some trends, what are some red flags or some common mistakes you still see that surprise you that AI founders are making? Any thoughts on any of those?

First, on the model side, I think there's still, back to that story, the thread about, you know, Greg's advice around the best developer experiences is a great model, I think because of the number of models there are in today's world. There's like actually a bunch of, and these are like more short to medium term edges because like eventually if it's really that great of an opportunity, everyone will figure it out. But like there are edges at the model layer that you can build into your products and services. And like, I think about this in the context of Gemini. There's a lot of really interesting product experiences that could be built because of long context that like the conventional wisdom, you know, is not to use long context to solve some problem. And thus for you as a founder, like the, you know, there's potentially an edge to be found there which, which I think is really interesting. So look at all the different models, find the different things they're great at and sort of see if there's an edge to be created there. The second thing is there's this really interesting sort of challenge for I think founders right now in this moment of like, it seems like everything is up for grabs. There's this incredible platform shift happening and thus there's like, you know, perhaps the tendency to like, try to go really horizontal and like solve this like massive like general problem. And at the same time there's also this like push. And you see this if you look at like YC startup batch data as an example, like very vertical companies of like, we are going to go and tackle this like one very specific thing that like you've never even heard of before but is like a really big problem for companies somewhere. And we're going to use AI agents to do it as an example. And it's like, it's interesting to see this play out where it's, you know, I think founders are like almost like being asked to choose one of those things. And my take is, I think there's something in between and I think it's a little bit harder to define what that Experience is where you can create all this value and not go sort of fully horizontal, not go so totally vertical. And like it's worth pushing on that to see if there's something there, if there's like space between two of those different product verticals to see if you can create something there. So I think it's worth pushing on that thread. And then the last one is before.

Before you go to that one. So like, so what would that look like? Because we have this debate a lot. This is definitely a common thing that I've brought up on my panels with Silicon Valley VCs and founders is horizontal versus vertical. And my conclusion is that we're, there's a playbook for each of them and part of it also has to do with timing. So horizontal was great in the beginning, but now already like Sierra's one horizontal customer service along with Maven and a few others. Right. And so you're too late if you're just going to start a new horizontal customer service agentic solution. The vertical ones are also quickly filling up. But if you're going to jump in, go jump in, build the best customer experience agentic solution for a certain vertical, but also show that you can, you know, you have some ability to go, you know, what are the next adjacent markets that you're going to quickly be able to grow into. You're putting forth this notion of something like in the middle and I'm curious, like, what, what does that mean? What does that look like?

Yeah, George, you need to go check me on this to see whether or not this makes sense. Because I, I'm sure I spend less time thinking about it than you. I think in the companies and like I think about this in like my, the investments that I've done. Like there are, are truly companies that seem like very horizontal where like we're going to build a general purpose assistant powered by AI and like it's going to do everything for you and like it's going to be great and solve all these problems. And then there's like speech pathologists with paperwork and like these are on two very ends of the spectrum. And like, all I'm trying to posit is that I think there's, there's a lot of room between these two things. And like maybe, and I don't know if this is fully true. True, like cursor is like perhaps a bad tongue in cheek example of this where like it is general purpose, it's an IDE that solves this thing. But like, it's not like they're trying to go after like solving every problem like they're not a platform and all this other stuff as well. Like they are a specific tool that is being built to do this thing. And I don't, I, I don't know if they would be considered horizontal in your mind or like what the definition is formally of these companies that are horizontal. But, but I think there's a lot of interesting product experiences to be built. And I think the thing that pushes me to have this perspective is the models sort of guide me in this direction, which is like at the same time as the model wants to be really, it has this really horizontal capability. It also has the ability to be a domain expert at some very specific thing. And I think like that artifact transcribed towards the company creation process, like, makes me think that there's something to be pushed in that space because again, yeah, because the AI models are the ones that are powering this and like they have those capabilities.

Yeah, it's interesting. It kind of reminds me of when Ash Maria, who wrote the book not, not the Lean Startup, but Running Lean and Scaling Lean. And I've spent some time with them and one of the things he talks about is don't, don't categorize them in the typical ways your customers or potential customers, your market look at what is their job to be done and what is the triggering event that causes them to make a purchase to solve the problem that they have. And so I think that's what came to me when you said, what you said is if we don't focus on a big boil the ocean horizontal and if we don't focus on like a very super tiny vertical, that might limit our total tam. But if we could kind of look at people like speech pathologists and people like that, what are the issues that they have, but we're not going to make it super general then kind of reminds me. The first episode of this podcast was with Maven AGI. I was with the three founders in Boston. And so what they said was they purposely didn't pick a vertical. They, they kind of just tested a few different verticals to make sure that it was, you know, they came from HubSpot and Google and Stripe. They wanted to make sure they could build a HubSpot like company that could go across multiple. But they're very horizontal. So. But that's the other thing that reminded me of. So, so maybe that's the middleware. Not middleware, but the middle layer that we could think about to. What you're saying is coming up with A way to look at things a little differently and get to an optimal solution. I don't know. But I did like that. So thanks for that, Logan.

Yeah. Yeah. And my other angle of this is the challenge that I've had with a lot of companies that are super vertical is like, it just doesn't feel, it's hard for me as someone who's like outside of my job at Google, like does capital allocation, like it's hard for me to allocate capital to ideas that like don't seem like they're like ambitious enough I think is part of it. And like that's, that's what has led me to be like, is there some other space to where these companies are being created? Because like on one hand you have this very ambitious, like we're going to solve everyone's problems and then on the other hand it's like, oh this, this one very specific problem that we're going to solve. And it's like, is there. And maybe this is, you know, the sign that I'm making the wrong investment decisions, which is a completely independent conversation. But like, to me it's like, is there some way to get closer to the middle where like you don't need to, to choose one or the other? Which is interesting.

Yeah, no, I like that. So as we're wrapping up here, anything else we should know about what's coming down the pike from Google? What are you most excited about? I know you talked about the latest release, Gemini 2.0 flash, right? So is there a 3.0 coming out later this year or 2.5 or what are you really excited about?

The thing that I'm in the medium term most excited about is we released in December something called the Multimodal Live API. If folks haven't played around with this, aistudio.google.com live has this really immersive experience where you can share your screen with the model and show your camera and input text and have an audio conversation. And this is like this truly multimodal conversational experience. And I think it hits on this like broader point of what's, what's going to happen for this AI UX 2.0 which is, you know, AI UX 1.0 is this like very chat centric experience. And I think the 2.0 version of this is going to be AI co presence where you're, the model is able to actually for it to be as useful as it can be, should be able to see all the things that you see should be present with you doing the work that you're doing and able to get the right context to actually help you solve the problems that matter to you. And the multimodal Live API, which developers can use and go and build into their products and such, really hits on this. The sort of early innings of this. Like it's not perfect and we got a bunch more work to do to make it really great for developers, but I do think this is what the future is going to evolve into and we're sort of seeing the early signs of that already in our product and other products.

So you described it in a way that if I'm the user of, would be seeing what with my permission, it would be have my camera, would have my desktop. Ideally it's learning for me and all that. But would it also apply if the use case is in like the opposite where I want, I'm interviewing a candidate, a vendor, or in my case a founder and I want them to log in, I want them to do their pitch, maybe answer a few questions. Then I want to take that multimodal input of them. You know, their, like you said, ambitiousness, their, their confidence. And I want the model to understand and rate that, the startup, the company, I want the model to understand and rate that and stuff like that. Is that kind of what's part of that release as well?

Yeah, you could probably build an experience like that though. My, I would, I would probably posit, I think the most value is going to be created by like someone using this thing as a tool. At least like the early signs from customers that we've seen is like putting this thing into the tools that you use can be like this huge accelerant because you now have like the, you know, you have the partner across everything that matters to you. And like you think about this for developers like in your ide, being able to just like click a button and then it's not just like a, you know, copilot spitting out text code, but like all of a sudden you have someone pair programming with you and like that experience, like with a single click of a button. You can imagine this in like the education vertical where you're stuck on a problem and you click a button and then all of a sudden like you have a co present tutor who's able to, you know, look at the scratch pad of notes that you're writing on the right hand side of your screen or you know, the math problems that you have open on your computer, et cetera. So I think that, yeah, I'm super excited about this because I think there's a lot of product spaces where this starts to make sense and you can like take it in a completely opposite direction around like you know, building systems for like sports and in the real like the physical world you could imagine that like really coming to life as well.

Yeah, no I like that. That's super cool. Awesome. Well Logan, thank you so much. This has been such an awesome conversation to learn more about yourself and what's happening at Google DeepMind Gemini, the AI studio. So for our listeners, hopefully you got a pretty good update on how you can leverage this to be able to get some great performance at a fraction of the cost quite frankly. And if you enjoyed this episode, make sure to subscribe, share and leave a review. If you're an agentic or AI founder raising around, reach out to us. You know that we're actively investing in this space and until next time keep reaching your untapped potential.