What Nat Friedman Expects From GPT-5, Microsoft's AI Strategy, And How He Invests In AI Startups

Welcome to another bonus episode of the Tech Meme Ride Home. I'm Brian McCullough.

And I'm Chris Messina.

And our guest today is long overdue, the great Nat Friedman. Hi, Nat, how are you?

Yeah, fine. I don't know if I'm great, but I'm fine. Thank you for having me.

Well, you're great, especially now in this AI era. You're, you know, the investor that I hear about in almost every deal that we see. Let's start off with some high-level stuff because the last week has been— I mean, it's always crazy in the AI universe right now. But let's do some high-level stuff on like, you know, Suleyman going to Microsoft, everyone fleeing Stability AI, even the CEO. Both companies cited difficulties in finding a business model for why They are doing what they're doing. Does this suggest that there could be trouble for, you know, especially the consumer-facing AI model companies, that maybe they're largely undifferentiated, maybe they're peddling commodities at this point? What do you think?

Well, there's a lot of variance in how well these companies are doing. I mean, there are consumer-facing AI companies that are some of the fastest-growing companies I've ever seen. And are just doing spectacularly well. I mean, Midjourney is a great example of a bootstrapped consumer AI company that has just absolutely spectacular usage and revenue and product and so forth. So I think it's very uneven. I think in the case of Inflection, look, there's 2 sides to every transaction. And I think if I'm Mustafa, And I'm, you know, I'm not obviously, but you know, if I'm Mustafa, I think I, there's a couple things I'm looking at. One is, you know, I don't have a hit product. I have a decent product, but it has relatively low usage and I believe in scaling laws. And therefore, you know, what I really want to do is build AGI and compete with Demis and Sam Altman and everyone else. And I, you know, no matter how good my product gets, I don't actually see a path to the compute scale that I need to do that. So I think Mustafa's probably quite excited about going to a place where he can do that. And then on the other, excuse me, on the other side of the transaction, you have Satya. And so you have to say, okay, well, what's Satya thinking? And obviously I don't know exactly what Satya's thinking, but you can sort of imagine looking at what's happening that, you know, he's had this very important early prescient bet on AI, put $1 billion into OpenAI before GPT-3, $13 billion into it now. His stock is way up. He's at the sort of forefront of this revolution, and he's got all his eggs in one basket. And, you know, November's board event highlighted potentially the fragility of that structure. So I think he wants to have a backup plan. And my read is that he acquired Inflection without acquiring it in order to have that backup plan. And now what does he have? I mean, I think the steelman interpretation is that he's got a team, he's got a model that claims to be GPT-4-ish, roughly. He's got a cluster that's working and a place to collect talent and data and and compute. And so he's got an internal OpenAI. It's, you know, there's a lot of gap there too. OpenAI is still the best, but I think he's got the makings of a hedge. And so that's how I read that one.

You mentioned, obviously, you know, it's like an acquisition in name, all but name. Like, I heard you and Daniel Gross talking about this idea of like a priesthood of like training these models is such a sort of bespoke thing, like it's an artisanal thing, like it almost like you have to have taste, I think is the term that you all use. Are we in a moment where it's almost like the IP doesn't matter so much as getting the artisans is the most important thing? So like you almost don't need to acquire the company if you can get the talent.

Yeah, I think that's true. The set of people who The amount of tacit knowledge that's involved in successfully training a high-quality large model is still quite high. So you can read the papers, you can look at the open source, but getting these things to train and converge and doing it over these large clusters and managing all of that, there's still quite a lot of that knowledge that's sort of not published, not written down. Many of them, many of the individual items are probably small, but they really add up. And the field is new. And so the set of people who really know how to do that is small. And then if you believe in pushing the research frontier and kind of pushing out, you know, the, the capabilities, then, you know, the set of people who are kind of at that frontier is— it's grown, but it takes time for that to grow. And so that's also still relatively small. In fact, it's roughly the same size it was before the entire sort of AI, you know, industry revolution that's attracted so much attention and capital and so forth. And so when I look at like what stands in the way of people training good foundation models, you know, the 3 key inputs are compute and data and talent. And I think if you're a mega-cap tech company like Microsoft or Google or Meta, you can get the compute. Actually, on compute, you're not limited by capital, you're limited by logistics. I think this is actually true for everyone right now. You can have $50 billion to spend on compute, but actually converting that into useful compute is a logistics problem that many people, like, I don't know anyone who's super happy with their cluster that it's like—

By logistics, do you mean like physical space or like cooling or like what is the limiting resource there? Everything.

It's everything. It's the— there's a lot of pieces that go into it. And so you have to have your supply chains lined up. If you're short, you know, 10% of your InfiniBand transceivers, it doesn't matter that you have everything else and the other 90% of the transceivers, your cluster's not done and it doesn't work. And yeah, sometimes it's space and power, cooling, as you said. Sometimes it's just infrastructure. Microsoft has historically Virtualized GPUs for OpenAI training on Hyper-V, and I don't know if they're doing that anymore, but they used to. And so does Hyper-V virtualize all the GPU features that you want to use? That you know that kind of thing. So there's just it's a big stack. Lots of stuff can go wrong. We've seen clusters not come up because of missing transceiver cables or transceivers or fans for a while were an issue. You know that like you know. There's a, we have our clusters in Nevada and it's actually at the end of a, it's outside of Reno and it's at the end of a long road. And that road, I think the locals call it Fury Road because there are so many—

Nice, like Furiosa, like Mad Max.

Yeah. There's just like so many accidents on the road and when there's an accident, the road can be blocked for like a couple hours. So like your smart hands can't get to the data center for a few hours. You know, it's like all these sort of things can go wrong.

So—

I see real world problems.

Yeah. And then, yeah. And then it's also getting into the supply chain. Is Jensen sending you the chips in the first, in wave 0 and wave 1, you know, like that sort of thing. Plus Nvidia stuff when it ships isn't quite done. So, you know, you've got a—

Yeah. What does that mean? Like just there's a lot of config to do or?

Get the, there's firmware bugs.

I see.

There's software, there's CUDA kernels that haven't been implemented yet to take advantage of the instructions. Uh, that the chip exposes. Sometimes there's hardware bugs that give you NaNs and you've got to RMA a bunch of GPUs that— so anyway, there's a logistics side, the data side, and the talent side. And, um, yeah, the talent's so scarce and critical resource right now. And, and I think that's reflected in like rapidly escalating ML engineer salaries, which are like going up very quickly. Yes.

I, I kind of want to lead into asking you what you're expecting from GPT-5, but a way to do this, to come back to Mustafa going to Microsoft is, you know, one of the more out-there theories that I heard was that, you know, if OpenAI achieves AGI, you know, by their bylaws, Microsoft can no longer access the technology. So is Microsoft making bets like this to sort of, you know, get out ahead of the fact that maybe AGI is right around the corner?

Oh, yeah.

What do you think of that as a theory? And then that can lead into what are you expecting GPT-5 Yeah.

Yeah. I, I don't, that's probably not, that's probably not the dominant force driving Satya to want to have Microsoft AI. Um, like I, I, my guess is that people at Microsoft don't mostly don't believe AGI is right around the corner. I think it's just like a single, absolutely critical vendor relationship that they have with OpenAI, which has strange governance. And that was demonstrated conspicuously a few months ago. And, and yeah, I mean, it's just tough if you have this critical input to the future of every product in your company to have it have a wall there. But, you know, I built GitHub Copilot with OpenAI and it was a great success and one of the proudest moments of my career, honestly, and incredibly grateful to them. They were so obviously so important to that. But it was also tough to have to like work across these organizational boundaries inside big companies. It can be tough to work across just internal boundaries and external ones. Yeah, like throw that in and it gets even harder. So I think that's more the driving force. Microsoft still gets as, well, I don't know what they have in the agreements now, but what I've heard is that they still get the pre-AGI technology as part of their deal. So, and then I think like AGI is in the eye of the beholder and ultimately it's something that You know, it's just someone's opinion. Someone told me recently that, um, Alex Tabarrok, I think he tweeted that AI is when it takes someone else's job. AGI is when it takes your job. Um, and so I think I thought that was—

It's pretty good.

Yeah.

What do I expect from GPT-5? Well, um, you know, I was really personally impressed by the leap from GPT-3.5 to GPT-4. Um, I thought we saw kind of emergent reasoning abilities that were like noticeably much better and less hallucination. And to me, to me, it was a noticeable leap. Um, obviously depends on what you're doing, but in the realm of coding, for sure, I prefer to reach for the smarter model. And so I would, I guess I expect more of that. And, you know, the weird thing about emergence is it's actually like, this is not something even the labs can do is to predict which new capabilities past the threshold of reliability. And I think that's how you should think about it. It's like all these capabilities are there, they're just not reliable. And then as you get the model bigger and bigger, they kind of go from 20% reliable to 70%, 80%, 90%. And then whatever your threshold is where it becomes useful, once it crosses that, it's almost a binary event where you're like, oh, now I'm really willing to use this. And so it feels binary, even though maybe the emergence is actually kind of gradual. And so, yeah, I think the big thing is, yeah, like just dramatically improved reasoning and coherence and the ability to therefore do agent-like tasks reliably. So, you know, many-step operations. I think people are finding ways to do this on GPT-4 now. I mean, we saw Cognition and Devin. I thought that was like an absolutely killer demo. And people have discovered that if you just repeatedly in a smart, very smart way invoke, you know, inference GPT-4, you can squeeze a lot more intelligence out of it. And so yeah, like, gosh, imagine when the models are bigger and better. So I think it'll be scale, you know, there'll be like another multiple of scale.

What do you think about multimodality? Do you think that'll be part of it too?

Yeah, I would expect it would be natively multimodal.

You know, what, what about like, what about active reasoning? Like, uh, are we like within months of that happening? Like this year, you feel like someone's going to make a breakthrough there?

I've seen some things. So I do think this idea of kind of looping in latent space and like pondering, you know, pondering a topic, um, is, is an exciting idea. Um, there's probably ways to do that without Without looping in latent space, like chain of thought reasoning does work. And so you can just like internal monologue to do that. You sort of spread the compute that you need to figure out the hard token over lots and lots of intermediate tokens. And then, you know, the big breakthrough everyone's waiting for is sort of this RL on text where you can use RL on text to like learn reasoning tokens to, to get better at reasoning basically, um, in a way that makes sense to the model. And maybe some of that is what's in Claude. I'm just speculating. I don't know. But, um, but, uh, yeah, I think actually we're going to learn a lot in the next year. Like, what if GPT-5 is not that impressive? That would be a huge deal. Um, if it doesn't feel like a 3.5 to 4 leap, that would be a sign of some kind of asymptote, um, at least temporarily, you know, while we find the next sigmoid to climb, essentially.

So.

So I think that a year from now, we'll be— we'll know a lot more about the future of humanity, I think, like, when we see GPT-5 and many other things that are in progress.

Speaking about a year from now or beyond, you know, given your background and experience at GitHub, you know, building Copilot and at Microsoft, I'm very interested in what the future of open source kind of looks like, especially as it relates to LLMs, but also software development and to engineering. Sort of writ broadly. So if we were to sort of unpack those things, you know, I guess I'd just love to understand how you see software development evolving or co-evolving with the presence of a product like Copilots. And, you know, what does it mean for the next generation of developers in terms of what skills they need to actually be gathering or gaining in, you know, in learning comp sci?

Yeah. Well, I'm sort of glad AI came around. I sort of felt like software development was starting to stagnate a little bit.

Yeah. Sure.

Prior, prior to AI. So I, I think, you know, the productivity of an individual developer today compared to 20 years ago is just incredible from my point of view. Like the, the, and there's a few reasons for that. One is, you know, we maybe compared to 40 years ago, we, we added these important high-level productivity tools like compilers and garbage collection and, and then, you know, object orientation. So, you know, certain things that were just sort of somewhat helpful patterns. And then open source created this, you know, reusable library of components that people could draw from. And then the introduction of software registries, like stuff like npm really accelerated that reuse, sometimes far too much. But so those were like quite radical changes in how software was written. I think the web was a big change too because people got view source.

Yeah.

Which for a while was a big deal. It kind of oriented, it created a new generation of open source people who were not ideological, but just highly practical. And, and so I think that's sort of where we were pre-AI. And then, you know, AI comes along and it, it's this compression engine that's able to sort of distill the collective learnings of all the open source code that's out there into into a helper and maybe soon a colleague who, you know, can help do some of the work with you. I personally still think it's incredibly useful to learn to code. You know, I take, I guess I take the different side of what Jensen said last week. The reason I think it's useful—

For the audience, what did he say last week?

He said like, you don't need to learn to code or something. I'm going to misquote him and I have immense respect for Jensen, but he said something like, you don't need to learn to code and there'll be like little coding agents, you know, thousands of them. And then they'll be like super prefer agents supervising the coding agents. And he may be right, but, um, to the extent that we will live in a human-dominated economy for a long time, uh, you know, human reasoning will be valued. And—

Which is, by the way, a wild statement to say, but yeah, continue.

Yeah, it's, I think we will for a while. So during that period, which no one knows how long it is, I think human reasoning will still be very valuable. And learning to code is a great way to refine your reasoning abilities and your problem-solving abilities. Not to mention when the AI writes buggy code, suddenly like it all comes into focus. You need to debug something, you know, throughout the stack. And so I think definitely we see like with GitHub Copilot that it really raises the productivity of not as good programmers. Um, and initially at least senior, very senior developers were just got much less value out of it. Uh, now I think we're reaching the point where senior developers are getting value. Um, but they have an advantage in that they can like debug when things go wrong. And so, but I agree with John Carmack, you know, who said the core skill is problem solving.

Yeah.

And there will, until, until we are fully surpassed, you know, human problem solving will be incredibly valuable. And I think you can learn and refine that skill through coding. And, um, but I use Copilot and all this stuff all the time now too. It's fantastic. You feel sort of naked without it. If you're a little bit rusty or you're an episodic programmer, you know, you can like dive right back in and you don't feel totally lost, especially given how sprawling APIs and SDKs and so forth, libraries, you know, have become. It's so sprawling that—

I mean, it sort of feels like an exoskeleton in a way where essentially you're sort of going into, you know, these kind of like forests of code and trying to like figure something out. Like I've been writing some extensions for Raycast and essentially, you know, I have a copilot there sort of helping me out with TypeScript because it's not something I've written before. And of course, it just sort of autocompletes and suggests things that's like, you probably should know this, but whatever, we'll let it go. And here you go and you can move on. Like you said, I'm simply trying to solve problems, right? This is not mission-critical code. So I guess the other thing that I'm wondering about, given your experience, is what it looks like for a younger generation that's coming up with the ability to learn to code and what Like, I do feel like it's kind of like, you know, I studied Latin in high school and it's like helped me a great deal in terms of understanding just the, you know, the root of language and, you know, word etymology and things like that. Is there going to be a similar type of benefit, you know, to learning basic, you know, skills in coding based on like, you know, the structure of language or that coding itself is a type of structured way of expressing ideas and concepts that is very terse and very efficient. relative to spoken language, you know, which evolved over thousands of years such that now this becomes a way for humans to communicate better with machines. And I guess I'm just trying to like understand, you know, like GitHub is such an interesting place if you think about it from a different type of like creator economy lens. You know, we think about sort of development and engineering as being something that only very technical people do, but increasingly it just feels like in, you know, whether it's 1 year or 5 years, there's going to be a generation that grows up with the ability to code as a like second language, like coding as a second language. So I guess that's kind of what I'm trying to think about based on, you know, just the presence, the growth, the popularity of GitHub, but also a set of tools that, you know, bootstrap you into the ability to solve problems, as you said, as opposed to having to go deep with any particular language.

Yeah, I think the bootstrapping is really valuable because the activation energy to get started on something is often the barrier to getting anywhere. And if you can kind of get some momentum and some dopamine quickly, You're just, I mean, like, I definitely, even, you know, me, I mean, I've been coding for so long and, and, um, thanks to GPT-4 and Copilot, the likelihood that I start a little project is so much higher because I just know I'm not alone and I'm in there with, you know, a little agent that knows the terrain and knows the API.

It's getting over the blank slate problem. You just like get started.

Yeah.

I think of it like an e-bike where it sort of flattens the hills a little bit and, you know, biking is still fun and, you know, but like I could bike around.

You can go faster.

Yeah. And you just don't have the, the schleppy parts aren't quite as bad. Um, so yeah, I think that's right. I mean, some people are worried that, you know, we're going to sort of outsource our cognition to these prostheses and we'll, we'll become soft. And I think there's some risk of that, but mostly that's like an eternal conversation. You know, when I was learning to code, it was like, oh, if you didn't know assembly, You weren't a real programmer and these kids today using compilers or, you know, they don't really get it. They're so confused. And, and so I just, I think, you know, mostly we figure this out and smart people figure out what they need to know to get something done. And I think AI just increases the chance that a smart person will try to get something done. And I think that's a good, that's basically a really good thing. So people seem to love it. Like, they demonstrate that by using it.

Totally. Yeah, it's been amazing to see. On that point, one more question on this topic. You know, one of the, I guess, memes or tropes that was going on a couple of years ago was that data is the new oil. And, you know, with, with the fund that Bryan and I run to invest in AI companies, we sort of thought about that a little bit differently in that we kind of think about what we call AI varietals. Now, obviously, being in California, sort of understand where that might come from. But the thought is kind of like data is more like the new terroir, if you will, and that the region and the culture and sort of like the ground from which the grapes, you know, grow is sort of like data that is particular to a certain vertical or to a certain industry. And that understanding the nuances of that data and information is actually quite important to building valid or useful LLM-driven products. In other words, you can't just— I now understand that we're trying to move towards this AGI kind of future where, you know, you have one bot that does all the things, but it just, it seems like where, as they say, the, the rubber meets the road is actually quite important to the effectiveness of these different AI products. So I guess I just wanted to hear you talk about kind of like the idea of like data as the new oil being this general purpose resource versus the pursuit of, or the creation of specific datasets that come from different areas and realms and, and the utility of one or the other.

Yeah, I mean, first of all, I definitely agree that data is really important. And in fact, like, sort of more time I've spent in this field, the more I feel that the models, in a way, they, they are the data.

Interesting.

Um, that, like, you know, Mistral has demonstrated very impressive results with their small models. And one of the ways they've done that is just by very, very carefully curating their training data. And not just kind of throwing the raw internet at it, but, you know, by really cleaning, cleaning their training data with a cascade of models that sort of, you know, remove duplicates and like take out the garbage basically. And, and turns out if you're raising your little baby AI and you're sort of screaming nonsense at it half the time, like it doesn't do as well as if it has like really clean training data that's very high quality and high IQ. And so. Um, so I do think that's true. And, you know, by the way, this is also a reason, it's actually sort of very good news for labs because they, in many cases, people publish papers, but they don't publish their training dataset. And so to the extent that you really clean up your data, you have this durable advantage, um, you know, that you don't have to release. And so, um, and they can't walk out the door in quite the same way. So, um, so yeah, I think data is really, really important. There's domains like robotics right now that are fundamentally limited, I think, by the training data. And, you know, we should see foundation models in robotics do unbelievably impressive things in the near future. And the gap between here and there is not hardware and it is not models or compute. It is purely the training data. And I think, you know, my bet would be in a year or so you've got, you know, robots, you know, 2, 2 arms sprout out of a desk and they assemble a Lego and they wash dishes and they open an Amazon box.

But they assemble themselves and then they, yeah.

Yeah. Maybe they do that. They, you know, they tie shoes.

They— What, what are they training for robotics on right now, essentially?

Well, there are some sort of academically constructed datasets that are quite limited.

Um, are those based on the real world or based on simulation and synthetic data?

Yeah, good question. Um, so there are, all of those things exist. Um, in the world of kind of locomotion navigation, simulation seems pretty helpful. Um, in the world of dexterous manipulation, it's hard to simulate. tying shoes and stuff like that quite as well. And so there you may need to just record stuff happening. And the ways that people do that are teleoperation. So you get a robot, you have a warehouse and you have a bunch of teleoperators come in and they just use the robot to do it. Super expensive. People are also just taking egocentric YouTube videos and using those in pre-training. So you don't really learn robotic stuff from that, but you do learn some of the—

There's a lot of TikTok papers that have come out that basically use the dances as a way to like train the models.

Yeah, I guess you could do that. But you can take like just unboxing videos from YouTube, for example, and use that in your pre-training and you start to learn some basic manipulation physics and stuff like that. And that's helpful for pre-training. And then there's, there are these devices called Data Hands that are floating around just a few weeks ago.

Physical devices or digital devices?

Yeah.

And so they have like cameras in them and they have like grippy spots. And you can just grab them and use them to do stuff. And then they record that. And so you're the robot.

That's so cool.

And you can use that in training data. So there's lots of stuff like that happening. So yeah, so basically I think data is just like this critical thing and the quality of it really matters. Um, as to this question of like general models versus sort of specific models, the results that we see are that general really does well, basically. And so like just these general models, you know, GPT-4 properly prompted, Um, can in certain domains, like do as well as a $10 million specially trained model in that domain. And so, you know, it's hard, it's hard to be sure about what the reason is for that. Um, maybe that $10 million model was not properly trained, for example, but, um, but that does seem to happen again and again. And so, yeah, whatever world we want to live in, nature seems to be telling us that generalization is very powerful.

And I think the other, the other thing is a little bit about like the person who is either controlling programming or prompting it, and to the degree that they have acquired the vernacular of a different industry or vertical and/or they have expertise.

That's true.

You know, so having someone who's actually an architect or someone who's actually a doctor working with an AI engineer, whether it's, you know, necessarily a general model or a specifically trained model, I think having that person that has that domain knowledge and has experience is probably necessary to evaluate the outcomes.

Yeah, I think that's right. Um, yeah, I mean, the other sort of data matters argument is like, uh, I think GPT-4 got a lot better at chess, 4 Turbo. And why was that? It was because somebody added like a chess eval. And so they added a bunch of chess training data so they could do well on that eval. And, um, so yeah, I just think they're, The other example I look at is GPT-4 can do ROT-13 quite well, but it can't do ROT-12 as well. And they're like cognitively equally difficult exercises.

Is it a matter of the evals or what is that?

It's because there's a lot of ROT-13 training data on the internet.

I see.

Okay. There's not a lot of people doing ROT-9 or ROT-11 or stuff like that. So someone should fact-check me. Maybe they fixed that. Maybe it's good at that now too. But But that was certainly true for a while. So that's sort of like an interesting observation about, you know, how these models work and how important the data actually is.

Could I— could we talk a bit about you really briefly? Like, the story that I've heard was that, you know, you were clued into the fact that AI was on the brink of a transformational moment by doing Copilot, GitHub. Is that true, or does your interest and activity in the AI space Go back further than that even.

Well, yeah, that is true. I mean, definitely GitHub Copilot woke me up to the fact that language models are working and they're going to get much better. And so large deep learning models are, are the future. I mean, that was super clear to me, um, when we did that. But, uh, no, my interest goes way back. I mean, I actually trained my first neural network in 1992 and, um, it was not very good, but it was a neural network and I went to school, I went to MIT, and I really was excited about my artificial intelligence classes. And then I found out, you know, I didn't know this at the time, but we were in like this AI winter. And in fact, like all the techniques and classes were pretty— it was not that, it was not that, you know, it was not that impressive or interesting to me. And then, um, I didn't really do anything with AI. In 2017, um, I started this thing called AI Grant that was at the time conceptualized as like grants to open source AI research projects. And we gave out a bunch of money to open source AI research projects. And then some of those people went on to found AI companies like Cohere and Cresta were both founded by people who received those original open source AI grants. And then we sort of repurposed that as like an AI startup accelerator because I already had the domain. So we did, like, we repurposed that a couple years ago. But yeah, it was, it was really GPT-3 came out and I had somehow not really noticed GPT-2. I'd sort of, it had passed by without me paying a lot of attention to it. And then GPT-3 hit me in the face and I was like, oh my God, this is incredible. We have to build some kind of product with this. And we started working on that and working with OpenAI and prototyping things. And like the models got better every couple of weeks. And, you know, I was, we, it was the most fun product I've ever built. It was so much fun. And, you know, we had this internal Slack channel where people were just like freaking out and it was passing everyone's interview questions and, you know, this kind of thing. So I knew it worked and I knew it was going to just get better. It was not going to get worse. It was going to get better. And so I left GitHub believing that we, GitHub, this is a very egocentric view, but that we had sort of shown the world what OpenAI's models could do and that AI was real and it was coming. And because our customers were developers and developers are the people who build products, that they would go off and build a whole bunch of AI products. And then like there was an 18-month period where that just did not happen. There were just very few startups started and very few products built. I thought there'd be copilots for everything, for law and for medicine and for architects. And it didn't happen. And then like come August '22, I was starting to go crazy because, you know, Stable Diffusion was out and Midjourney was out. So that was neat. But like in this realm of text and reasoning and nothing was occurring. And so that's why we reanimated AI Grant to like say, hey, you guys should build AI products, you know, products, not papers, apps, not archive. And, um, and then ChatGPT came out a few months later and that was the actual starting gun for people, which shocked us. Uh, I didn't, yeah, I didn't, I didn't expect it would take that.

How, how did you hook up with, uh, Daniel, in terms of investing together?

Yeah, I mean, Daniel and I ran in sort of similar Silicon Valley circles, and so we knew each other. I actually was at his YC Demo Day in 2010. I met him there briefly, and then we had friends in common and, you know, we would meet at dinners and we liked each other and we started working on some projects together. We'd angel invest together. We co-led the, with some other friends, the first big round in Retool. And that was a really fun experience. And we did AI Grant together back in 2017 and some other things. And so we just liked working together and bit by bit, our amateur investing activities together became more serious and larger scale. Really what happened is we invested like small amounts of money in companies that got very big. And then like couldn't, didn't have enough money to buy more in the subsequent rounds. And so we went off and fixed that problem and found ourselves in the position of like full-time investors eventually.

Do you, do you have, do you have a thesis that you're working from that you like, or are you just like, we want to see everything and we want to go with whatever the new is?

Well, we're pretty entrepreneur-focused. I think if there's any thesis, it's that the entrepreneur builds the company. And, you know, this is a mistake that if you're someone like me, you can easily make is that, you know, you've done things in the past, you've built products, and so you fall in love with ideas, but you project yourself into the company. And so you think, oh, if I were running this, I would do this and this and this. And you get kind of, you build this intergalactic plan to take over, you know, take over the world. And And then like you forget to evaluate whether the founder will do any of those things or have better—

Founder product fit is like so important.

Yeah. And just like how good is this founder gonna be at recruiting people and how good is this founder gonna be at making hard decisions and how much do they want to win? Like there's a lot in the psychology of the founder that I think is really important. So that's been Something I've updated on a lot personally is, yeah, like I tend to really fall in love with ideas. And so this is something I have to work on.

What are some characteristics of like AI founders that you think are like more promising that you've learned recently?

Right?

Because I feel like this era is just, it's different in terms of building software and products that people use and that working with AI requires a slightly different sort of attitude.

Well, there are a lot of evergreen things, like you have to care about building great products and you have to care about winning and you have to, Yes. Have, you know, so those things are not different. The biggest difference I'd say with AI founders is whether they've deeply internalized scaling laws and whether they actually believe them or not. And the founders who have the courage to scale and believe that high quality data, lots of high quality data, plus lots of compute, plus good talent, combining those things will lead to like a better product, those founders will, yeah, they will outpace the ones who don't.

How do you, I mean, like, just to make this clear, can you, what is the inverse of that? Like a founder who doesn't believe in scaling laws, what are, what choices would they make now as they're building their company?

Well, they would think that our model is worse, but our product is better. And so it's fine. And The other version of this mistake occurs a lot in the AI field right now where people are like just into the ML and they don't spend any time on the product. And this is the most common issue, but we have also encountered the founder who thinks, well, our model is good enough, but our product's better. And the problem with that is if the next competitor's model suddenly becomes 10 times better, the switching cost seems so low now too. Well, and it just doesn't matter how much better your product is if their model's 10 times better. So I learned a lot about that from Midjourney and my proximity to Midjourney because He, um, he was— he's been on Discord this whole time, but he keeps winning because his model's the best. And the model actually is the best in ways along dimensions that users care about, like controllability and aesthetics and stuff like that. You know, when DALL-E 2 came out, it was very clear that OpenAI did not think of it as a product. They thought of it as a technical capability. They thought of it as text-to-image. And David thought, I'm building a tool for imagination.

Yeah.

And so of course this is a product that must have aesthetics and, you know, be, be pleasing and, and, you know, it should have certain types of controllability. It's not a technical capability demo, a research artifact. It's, it's something that people use for their imagination. And so—

You know, the other thing that's, that's really, sorry, just to build on that, like I've been thinking more and more that good products nowadays are kind of the artifacts of healthy, useful conversations. And the fact that Midjourney was first built in Discord means that instantly he's getting feedback from thousands of people who are using the product, complaining about the product. He's able to see it in real time and then make adjustments and changes as opposed to the conventional method of building a software that ends up on a website or, or in an app store where you really don't have that bidirectional kind of meaning searching process with your constituency.

Yeah, I think the mid— The Discord thing has been incredibly beneficial for them. I think you're right. It's, he's like incredible at using polls and listening, you know, he does office hours with all of his users and it's bidirectional. And so I think, you know, that's spectacular. And then it also makes the product self-documenting and more social and fun. And so those are valuable things. It's time for him to get that web app out there, I think.

It is.

Agreed. Agreed.

Yeah.

Yeah. Just final one for you. I'm going to bring back that period when you said, you know, there was 18 months where not a lot was happening. You were expecting this Cambrian explosion and it didn't happen. What would you fear would be the thing that would lead to another sort of roadblock period where things slow down? We're expecting, you know, GPT-5 and next week something else and now video and robotics and stuff. But like, What would be the fear that you have from anything that would put a pause on sort of the revolution that we're experiencing right now?

Well, I think there is just baseline friction of product people doing great things. There's just not that many people who are— I mean, I thought the, you know, Cognition demo was very telling in that way, and that, you know, GPT-4 had been out for a year, and they were the first ones who figured out how to repeatedly inference it and get agentic behavior. And so like, why couldn't someone have done that 6 months earlier? Is there a good reason? Maybe there is, I don't know, but I would be surprised if it wasn't possible to do 6 months earlier. So I think that's the baseline thing. And the world's just incredibly contingent. The frontier is extremely inefficient. It feels like there's millions of people at the frontier. There's really hundreds. And so it's just like extremely, extremely inefficient. Yeah. What could slow things down? Well, I think the biomarkers to watch are probably, yeah, like, is GPT-5 noticeably and conspicuously much, much better? I think that's obviously a big one. I think that probably Microsoft's AI revenue would be another one. If Microsoft Copilot revenue were shockingly low somehow, or sort of missed expectations, or if Microsoft starts to attenuate the guidance there, they're extremely good at guiding the street. That would be, that would be very interesting. I think that would take the heat off a little bit. If scaling, basically the entire future is determined by this, like how much AI capabilities improve over the next 10 years and what the shape of that curve is. Like the one thing I feel incredibly confident about is that in 2034, AI capabilities will be dramatically better than they are today. I feel a little less confident at drawing the shape of the climb from here to there. you know, how much of it happens soon, how much of it happens later, you know, how, how gradual it is. Is it a bunch of stacked sigmoids or is it just this like exponential curve? I'm not really sure. But, um, but I think that's the key question about the future. And it's like, if this asymptotes soon, it's a revolutionary new tech platform. And, you know, it's as big a deal probably as the web in some ways, but it's just something we can grok. And if it doesn't asymptote for a while, then it's civilization altering. And, um, I think that seems somewhat more likely to me than the former.

Nat, uh, thanks for coming on and painting the picture of all that.

My pleasure. Thanks for having me.