Benedict Evans on AI, Web3, VR and the Future of Tech
In this Upstream interview, analyst Benedict Evans discusses the current state and future trajectories of several transformative technologies. He examines the rapid advances in AI, highlighting uncertainty around model costs, error rates, and the proliferation of foundation models, while critiquing the hype around ChatGPT plugins and AutoGPT as premature. Evans explores the challenges of turning AI breakthroughs into usable products, questioning the adequacy of prompt engineering and the need for better user interfaces. The conversation shifts to the metaverse, where Evans argues that VR adoption will be gradual and driven by specific use‑cases rather than a wholesale consumer shift. He then delves into Web3, outlining the infrastructural layers, governance dilemmas, and the difficulty of delivering compelling user experiences in decentralized systems. Throughout, he emphasizes “radical uncertainty” as a guiding mindset for technologists and points listeners to his newsletter and podcast for deeper analysis. The episode concludes with reflections on how software continues to reshape industries and what questions might dominate the tech discourse in the coming year.
Chapters
I encounter people who think that you'll be able to use the metaverse on your smartphone, at which point to me that I have literally no fucking idea what you're talking about. Like, that's the internet. With metaverse is like everything will be interoperable, everything will be portable, you'll be able to move assets between every different thing. Why does that make sense? You know, I go into Fortnite and I buy a 15-foot-tall yellow banana character with a gun that shoots exploding tomatoes. Cool. I go into Call of Duty. And I'm sneaking through the forest, crawling through the mud with my other VR special forces guys, except I'm a fuck-off great yellow banana shooting a spray gun. Even if you believe that VR is going to be the universal device and 3 billion people will have one in 10 years' time, any prediction you make today about the specific market structure of that and who's going to be in charge and how it will work is going to be wrong. I feel ambivalent about ChatGPT plugins. Like, if this thing is absolutely fucking amazing and changes the world, how is it that's using a Web 1.0 API to talk to a Web 1.1 company to answer this question?
Benedict is one of the most important thinkers and analysts in all of technology.
Every year he creates a presentation that talks about where tech is going.
In this episode, we discuss exactly that. What is the future of AI, of crypto, of VR slash the metaverse? Where is it all going? Nathan LeBenz from the Cognitive Revolution joins us at the end of this episode.
Benedict Evans, welcome to Upstream.
Thank you for having me.
So, Benedict, every year you come out with a presentation at the end of the year, where you talk about where things are going next. And it's a, it's a must-read in the industry. And I'm curious for you to reflect on the presentations the past, you know, 5 years or so, and reflect on what you think you saw correct, what you think you predicted, like what holds up from prior presentations, and where you think you perhaps didn't foresee what would happen. Of course, it's impossible to predict everything, but why don't you reflect on that a little bit for us?
Yeah. Every year I do a presentation that says these are the fundamental trends in tech for the next decade. And then about 9 months later, I say, no, these are the fundamental trends in tech.
Exactly.
I don't know. There's a very peculiar sort of naivety that you sometimes encounter from people in tech that sort of news is just facts. And you try and kind of patiently explain, well, yes, but there are infinite facts and which facts you choose to convey is an opinion. So you can't have news that's just facts, that's nonsensical. And in the same way, you know, I do this presentation and I can't do all of tech, partly because that would be 500 slides, partly because there's great chunks of stuff that life is too short and I don't know anything about, and it would take me a year to understand what everyone in that space knows, like Chinese semiconductors. Partly because I sort of think about where is there scope to say something new and interesting and point out some new trend. Partly because I think, well, who's my target audience for this that's going to find this useful? And so, within that sort of framework, I look for sort of Okay, here, let me give you half an hour of a lot of stuff that's happening in a way of understanding what's going on, or some things that are going on. So, that's one way of looking at it. Another way would be to say the place where there's something new and different changes over time. So, we spent sort of 3 or 4 or 5 years saying, okay, smartphones have kind of happened now. And So what's the next thing? Is there a next thing? What are the new trends that come after smartphones? Smartphones can completely remake the tech industry. Okay, yeah, but it's happened. It's a kind of classic S-curve model that one S-curve follows another. And there's one way of looking at that, which was to talk about machine learning and then to talk about VR and AR, which somehow got wrapped up into this kind of ridiculous word metaverse that we can maybe talk about later. And then on the other side, crypto and Web3 and ideas about what you would do with that. So, the one side, kind of a new device, on the other side, a new architecture, a new business model, a new framework, maybe kind of a new open source. And now, in the last sort of 6 months or so, we've had this sort of second wave of machine learning, and that's become the only thing and everything. And it's a lot easier to see that than to see VR or to see crypto. So, that's kind of one framing to think about. But the other framing would be, sort of meanwhile, the rest of the tech industry and the rest of the broader economy isn't so much working on stuff that's going to happen in 5 or 10 years' time as deploying ideas from 5 or 10 or 20 years ago. Ideas like SaaS and cloud and automation and workflow. This hilarious phrase, digital transformation. Which sounds like a kind of a parody of marketing nonsense, but actually just means, like, in the '70s, you've got a mainframe, and in the '90s, you've got PCs, and now you move to the cloud, and that's kind of a big deal. That's basically what digital transformation means. I mean, like, maybe people will buy stuff on the internet, maybe people will watch TV on the internet. So, you've got enterprise tech companies deploying ideas from 10 years ago, and consumers deploying ideas from 20 years ago. That's how long it takes for this stuff to diffuse. So, you've got this kind of spread of What are the ideas for what's going to happen in the future? What's going on right now? What's actually just deployment from ideas of 10 years ago, 20 years ago? How is that changing other industries? So a lot of what I've been talking about in the last year or two is sort of, okay, how is that changing advertising? How does that change big consumer brands? How does it change TV? And the sort of thesis within that is most of that has nothing to do with tech. I don't think Netflix is a tech company. I don't think Tesla— and there's a big argument, is Tesla a car company or not? But basically the bear argument for Tesla is it's a car company. In which case it's got a very different margin structure to the bull case. So you've got these kind of different layers of, here's all this stuff happening. What within that should I try and talk about that I can say something and that would be useful to an audience? Where is there some point of tension or friction or difference where something meaningful is changing? What is the sort of the fundamental megatrend, if I can use that again, another ridiculous word, As opposed to, here are 100 things happening. But above all, what I try and avoid is just saying, here are 100 charts. I don't want to do Pinterest for charts. I don't want to do, here's every slide, every chart I've seen in the last 18 months. And there are big presentations that do do that. It's basically, here's every stat from the last year. I always try and say, well, there's stories here and there's reasons why I'm showing you this.
Just because you mentioned it, why isn't Netflix a tech company? And what does that mean or imply? Oh, yeah.
I mean, I often say about my presentation that each slide could be a half-hour conversation. I think a kind of a basic way to think about Netflix is all the questions that matter for Netflix are TV questions. Like, what shows? What rights structures? What do you pay the artists? Do movie stars decide they don't want to do TV because it affects their box office pull? How do you depreciate the TV shows? You capitalize them and depreciate them over how long? What's the right mix of content? I don't know, they're all TV questions.
But is the idea that they get the answers from looking at the data of what people watch?
No, that was all bullshit. I mean, it was total bullshit. And there's this hilarious story in, I think, either Variety or Hollywood Reporter. And there's a point here. And if you want to know what's going on at Netflix, read Variety and read Hollywood Reporter. Don't read TechCrunch. But there's this whole thing on the people that they'd hired and how how well they were doing, and they hired these people because they had those relationships and that person didn't. And you read this whole thing, and the subtext here is basically Netflix went to LA, hired LA people at LA salaries to have lunch with other LA people and buy LA stuff from LA people. There's no technology here. And I'm being kind of entirely serious that I think there's an important phenomenon, which is that there are companies which, as Marc Andreessen put it, Their software eats the world companies. So Uber or Airbnb, kind of canonical examples. Uber doesn't sell software to taxi companies, Uber changes what taxis are. Airbnb doesn't sell software to hotel companies, it changes what it is to be a hotel. And it's something that Hilton couldn't do. And it's something that a taxi car company, cab company, couldn't do. But what Netflix does, or indeed what, say, Shein is doing, is using the internet as a new channel. To enter the market. And this new channel has different characteristics. So it doesn't, you know, streaming doesn't have time slots, so you don't have a limited number of Saturday nights. And Shein doesn't have to put inventory in 2,000 stores the way Zara does. So they can make a product and literally make 50 of them or 25 of them in the entire world and then wait to see who buys it, which Zara, which kind of pioneered fast fashion, obviously can't do. So the channel works differently. But that's kind of like saying that Walmart's Supercenter channel works differently from a convenience store. And yes, it does. And therefore it's all different, but it's still a grocery store.
Yeah.
And that's kind of my point about— there was this sort of moment when everyone in tech said, oh, well, legacy media companies will never be able to do software. Guess what? Disney+ has got as many subscribers as Netflix. Yes, some of them are in India and not paying very much, and some of that's ESPN and so on. But in principle, the Disney+ app is fine. Maybe the Maybe the video compression is 10% worse than Netflix. Guess what? No one really cares. And if Netflix had 50% better video compression, but all they had was reruns of Cheers and Friends, no one would care. It's a TV company. It's no different from a satellite TV company.
If you were advising either of those companies, would you encourage them to try to become a tech company, or that's not what's needed?
Well, so I think, like, I'm almost arguing against myself. You can get into this very hack and yeasy thread about, well, what is a TV company? And they have this ridiculous argument about what's tech and what's TV. My sort of Occam's razor is, well, what are the questions that matter here? And the questions that matter are all TV industry questions. I think the point for Netflix is, like, the software is a commodity. And that's something that a lot of people in tech would get very upset about. But commodities can be hard. Flat panel screens are a commodity. They're also a very hard commodity full of lots of science, but they're still a commodity. And cars are a commodity. Like, making a reliable engine that can do 250,000 miles without breaking is very difficult, but it's still a commodity. And yes, there's only— maybe there's only 10 people that can make that commodity, but it's still a commodity. And the point here is that for streaming TV, making a great streaming TV app is actually not something that only a software company can do. You can make an adequate— you can make one that's good enough, Because what matters is The Mandalorian.
Yeah. If Quibi had worked, would you have called that a tech company, like an innovation on the form? Or is that—
It's an interesting hybrid. I mean, I think the better cap— because it's too hard to know the counterfactual there. It was done in a very un-tech sort of way. You spend billions of dollars before you've met a customer. I would say the counterpoint to Netflix is both YouTube and TikTok.
Yeah.
Because Netflix spends sort of $15 to $20 billion a year commissioning content, which is as much as any of the legacy TV companies. I think Warner and Discovery spend more, that's more, but that's all. And I would look at that as not like, oh my God, Netflix is such an amazing company. They've broken in, they are able to spend this much. No, it's they have to spend that much. The only way they get the business that they have is by spending what you spend to be a TV company. You have to buy TV shows. And software doesn't have a net— there's no network effect. There's no software doesn't give you a shortcut in the way that software meant that Airbnb didn't have to build lots of hotels and Uber didn't have to buy lots of taxi cabs. That was a shortcut or a way of routing around, or better, WhatsApp didn't have to build global telecommunications networks. They didn't have to build millions of base stations. So software let them get around that infrastructure requirement. That doesn't apply to Netflix. If you want to be a TV company, you've got to buy a shitload of TV shows.
Yeah.
YouTube's creator payouts are actually— they don't disclose the number, you can kind of estimate it— roughly equivalent to Netflix's content payments. But that's it. They actually have a completely different model. Netflix does not fundamentally have a different model to any other legacy TV company. They pay people to make TV shows, whereas YouTube doesn't. YouTube is doing something else.
Yeah, that's interesting.
And I'm curious, software has eaten the world, but some fields it's eaten less than others, like education, healthcare, housing, some others that are highly regulated. I'm curious, how much of the questions there are technological versus specific to those fields and the regulations within those fields?
Well, so I think it was Bill Gurley sort of 20 years ago said there's no Moore's Law for backhoes. Cheaper digging machines. I mean, there's a recurring fantasy in tech that you're going to use software to disrupt cellular operators. And cellular operators are infrastructure companies and property companies. The tech is actually a pretty small part of the business. Most of the business is guys going up ladders and climbing onto the roof of buildings and bolting stuff to the roof of buildings and putting big metal antennas up there. There's no software for that. And the similar one for housing, Yes. Part of Marc Andreessen's software eating the world point was, like, the first 20 years of the internet is basically information arbitrage. It's providing stuff around the edges of the existing industry. So, it's like mortgage price comparisons, or it's Yelp and TripAdvisor and so on. One of the ways you could look at something like TripAdvisor or Booking is that everything is probably disruptive to someone. So, online flight booking is massively disruptive to travel agents. That whole industry has basically disappeared at this stage. Unless, you know, I don't know, like booking a 3-week safari in Namibia or something. But the idea that there was a shop you had to go to in order to buy a plane ticket and that they knew what the planes were, like, that's completely evaporated. But it didn't really disrupt the airlines. I mean, yes, it kind of changes how their business works because you have this price transparency and liquidity and so on. But, you know, the basic business of an airline is flying airplanes.
Hey, we'll continue our interview in a moment after a word from our sponsors. Hey everybody, I want to recommend a couple other shows that we also run. One is Moment of Zen, which I co-host with Dan Romero and Antonio Garcia Martinez. We talk about everything from tech to history, philosophy. We've also featured Marc Andreessen and Balaji on those podcasts, so I recommend you checking them out. My other show is Cognitive Revolution. It's an AI show that I co-host with Nathan Labenz. I recommend listening if you wanna stay up to date with all things AI. Secureframe is the leading all-in-one platform for security and privacy compliance. Get SOC 2 audit ready in weeks, not months. I believe in Secureframe so much that I invested in it and I recommend it to all my portfolio companies. Sign up for a free demo and mention Upstream during your demo to get 20% off your first year. Now more than ever, startup founders need a safe place to put their cash. Mercury protects your money and also provides the streamlined user experience that great founders expect. Through partner banks and their sweep networks, Mercury offers up to $5 million in FDIC insurance, which is 20 times the per-bank limit. They also make it easy to invest any cash above the FDIC-insured amount in a money market fund. 100,000 startups trust Mercury with their finances. I've been a happy Mercury customer and have found their team incredibly helpful and responsive. They even got an important wire out of purgatory on Christmas Eve. After all, your Christmas is my opportunity. Visit mercury.com to get started. Mercury is a financial technology company, not a bank. Banking services provided by Choice Financial Group and Evolve Bank and Trust, members FDIC. MarketerHire is one of my favorite resources for growing startups looking to hire marketers. Hiring is hard, and the time it takes from founders can be better leveraged elsewhere. MarketerHire pre-vets top-notch marketers across a dozen roles. Whether you need help with growth marketing, SEO, lifecycle content, or any other aspect of growth marketing strategy, it's free to use and you only pay if you end up hiring someone. If you wanna hire a great marketer the easy way, MarketerHire is offering Upstream listeners a $1,000 credit for first-time customers. Go to marketerhire.com/upstream and use code upstream to get your $1,000 credit.
And the same thing, in fact, actually for cellular, that smartphones completely disrupted the handset business. didn't really disrupt the cellular business at all. I mean, for all that the pricing structure has changed, here we are 25 years later, basically the same companies running basically the same business with basically the same revenue and the same margin structure. Vodafone today is basically the same business it was 20 years ago. Same with Verizon. And so everything is disruptive to somebody at some level, at some level in the stack. And it may change who your suppliers are and it may change what your go-to-market looks like. I think one of the interesting things that's sort of happening now in consumer is, should you go direct to consumer or not? Because if you are L'Oréal or Procter& Gamble, for the last 100 years, you were a B2B business. You sold trucks full of soap and you sold trucks full of makeup and whatever, and razor blades and whatever the product is. You didn't actually sell products to consumers. And now you kind of have this question, should we be doing that? Should we be selling direct to consumer? What kind of is that? Because that would be a whole other kind of company. What would that look like? And, you know, the answer for some companies is probably no. And obviously, we've had this kind of moment in the last sort of 5, 10 years where everybody wants to have a direct consumer relationship, which is, of course, also what you see in TV streaming. And of course, not everybody's going to be able to have that. But then you have this argument in TV, okay, what are going to be the direct-to-consumer streaming brands? It's going to be net in the US, obviously, and it's going to be Netflix plus Disney plus maybe one or two other people. And who will those be?
And that's Variety conversation.
It's a TV industry conversation. It's not a tech conversation.
Zooming out again, when we go back to your earlier presentations, I'm curious what has surprised you about what has perhaps taken longer to happen or what has happened sooner? I mean, there was this moment in COVID where we thought the world was getting pulled to the future sooner than we thought, and perhaps there's been a bit of a reset there. But I'm curious if you can reflect on that broader question.
We overestimate how quickly stuff will change in the long term and underestimate how much it will change in the longer term. But one of the aspects of the kind of takeoff in machine learning right now is that it's sort of sitting on top of, standing on the shoulders of giants. It's sitting on the top of the whole consumer internet, all the cloud infrastructure, all of the data that's already been kind of created and accumulated. And if you tried to do this 10 years ago, it would have just taken massively longer. Nobody has to go, no one has to buy a device. You don't have to go and ingest all this data. It's all kind of there, ready to go. You just need to spend $1 billion training the model, which is maybe another conversation. Whereas I think what a lot of people didn't understand about crypto was, and particularly the more interesting parts of crypto to me, which are the idea you actually could run this as software as opposed to using it as currency or as a sort of mechanism for speculation, which I find very boring. But the idea that you could basically build Instagram on a blockchain is very interesting. But it was kind of like talking about YouTube in 1990. Like, yes, but we actually, we have to have the web first. And nobody's got— nobody even has a modem, never mind broadband. Everyone's got dial-up, except we have nobody's bought a dial-up modem yet. And that's sort of where crypto, I think, still is. It's like there's more years more of those kind of low-level, 4-letter acronym infrastructure to get built. Something similar, I think, with VR. You know, I mean, one of the charts I kind of— I think I had in this year's presentation and last year as well was just showing how long it took for smartphones to happen. So people start getting interested in obviously the internet and mobile internet in like '99 and 2000. European mobile operators spent €110 billion buying cellular 3G spectrum in 2000, which is like €150 billion now or €200 billion now, whatever the calculation would be. People are buying these devices and there are smartphones and people are using the mobile internet and there are camera phones and there are mobile games. And so there is sort of stuff happening and the charts are going up every year, but the iPhone is launched in 2007. iPhone-type sales, so iPhone and Android sales don't really take off until 2010. It takes until sort of 2015 before you get to the point that most people have got one of these things. So it's sort of 15 years after people really get excited about this stuff. And I think there's a sort of similar point to make about VR here that Oculus, the Oculus acquisition was now almost 10 years ago. And we're still not there. And it's probably at least another 5 years. There's another thesis that says, no, it's not 5 years, it's 20 years, that this is sort of like looking at the Newton or the IBM Simon or General Magic and the right idea come back in 20 years.
And when you talk about the infrastructure that's not yet there for Web3 and VR on the Web3 side, is it the scalability technologies or is it something else? Because even if the scalability stuff was there, it feels like the What is there to scale? It feels like the use cases aren't quite super popular that they're exploding, um, aside from some NFT stuff or something.
But so I don't like the use case argument, and the reason I don't like it is that, you know, I started my career as a telecoms analyst, and so I was, as I sort of just alluded to, I was looking at mobile and mobile internet in sort of '99, 2000, 2001. And the question absolutely every investor asked was, well, what's the killer use case? What's the killer app for 3G? So people talked about, well, maybe it's mobile banking. No, M. So you'd have these reports and they would list 150 things that had M at the beginning. So it was M banking and M commerce and MTV, M television, and M this and M that. And video calls were going to be a big thing. And it's kind of funny how people have no historical memory because I said people thought video calls were going to be a big thing on 3G and they never happened. And someone of these kind of anti-crypto would-be influencers jumps in and says, look, Benedict, you're an idiot. Everyone's using FaceTime. Everyone's using Zoom. It's like, yes, but that's 15 years later. FaceTime was launched— I think Apple announced FaceTime something like a decade after the 3G auctions. And it didn't use 3G. It needed 4G. It wouldn't have worked on a 3G connection. And when people were actually talking about 3G video calls, what they meant was a circuit-switched Which calls using a PSTN numbering system with a 64K video channel where you would be billed by the second like any other phone call. And guess what? That didn't happen. Nobody did that. And so everyone was asking, well, what's the 3G killer app? And the answer was, there was the wrong question. What you were actually saying was, what's the killer app for having the internet on your phone? And the answer was having the internet on your phone. Like having everything. On a phone where that was, like, a good experience? And actually, the answer was being able to have a computer in your pocket that was connected to the internet that you would take everywhere. So, again, it was like, it was the wrong question in, like, 4 or 5 different ways. And I think kind of the same, it's like, well, what was the killer app for the internet? Well, I suppose you could say the web was the killer app for the internet. Fine. So, what was the killer app for the web? If you kind of say that, well, it was Amazon, or it was—
No.
No, the killer app for the web was the web. And I think the same thing now. Like, this is an idea. It's like saying, well, what's the killer app for open source? It's the wrong layer of abstraction to understand it. This is an architectural layer that allows you to build a broad class of things in a different way. What's the killer app for that? There isn't a killer app for that.
I've always seen you as sober on the Web3 issue because a very cynical take would be we've been focusing on Web3 crypto for the last a few years or so, and we should have been more focused on AI, missed it right under our nose. 2023, what are even people doing with Web3 and crypto? And so I'm curious, as you've reflected on the hype over the past few years, what are the things that were overhyped that aren't going to be the vision of the future, like the visions that you never bought? And what are the things that people are underappreciating now about the impact that Web3 and crypto will have?
I mean, I mentioned, I think part of the challenge for both metaverse as a word and Web3 as a word, it's funny, this is kind of going to be kind of a bizarre analogy, but this is sort of phenomena that you notice if you ever pay attention to this stuff, which I rarely do, which is that you've got around Europe these sort of royal families where they lost the kingdom. So like the Russian royal family or the Greek royal family or the French royal family, and the kingdom went away 100, 200 years ago, and there's like 3 different claimants to the throne. So there's 2 different people who claim that they are the rightful head of it. And it's to do with, like, were you allowed to marry that person? Or who is this cousin or that cousin the next in line? Which can actually be less obvious than you would think. And the reason I mention this is, like, if you actually had the kingdom, there wouldn't be any argument because somebody would have become king.
Right.
And that would be it. And you wouldn't be arguing about, well, your great-grandfather, the line split in 1870. Because there would be a king and there wouldn't be any argument about who the king is. And it's kind of the same with— this is what reminds me of looking at metaverse and Web3, is there's no actual product. If there was an actual product, then you could say, well, that's what it is. But because there's no product, people just sort of invent all sorts of imaginary things and say, well, that's metaverse and this is metaverse and that's metaverse. And the same with Web3. Suddenly sort of anything that you can think of that might be cool, that might happen in the next 10 years, is metaverse, and also metaverse is somehow also Web3. And so these words sort of completely lost any meaning. And I don't mean that in a kind of rhetorical sense. I mean, quite literally, if somebody says metaverse, you cannot know what they mean because there's like 10 different things they might be talking about. And the same thing for Web3. The same thing for Web3. And it's kind of easier to say, what was Web2 originally supposed to mean? What Web3 was originally supposed to mean was that you could build software on a blockchain. And so you could actually build Instagram on a blockchain. I mean, it wouldn't literally be Instagram. The new thing is never the old thing, but you could build a social thing that billions of people were using and have it be running on a blockchain instead of be running on private servers owned by Meta or indeed like Snap runs on Google Cloud. Instead of being running on a private cloud system, where there's one company that writes the software. It could be— you should think of a blockchain as being a distributed open-source computer. That is like, it is a distributed open-source cloud computing system in which anybody can see all the code and see what's happening and see how it's working. That to me is very interesting. Which things would be better if you ran it like that becomes a conversation. And how do you deal with sort of tragedy of the commons problems around that becomes an interesting question. But I don't think you could build Instagram on Ethereum now, even after the merge. There's still layer 2 and layer 3 and all sorts of other stuff that would have to happen before you'd be able to do that. It's a little bit like there were all these arguments in the mid-'90s saying, well, the web isn't going to scale, the internet isn't going to scale. And just because people said X in the past and were wrong doesn't mean that if people say Y today, they must be wrong. It's kind of a basic fallacy that people have in talking about this stuff. But you kind of have to also understand, yes, it can be hard. You can be wrong when you say this stuff and you can get these predictions. These predictions can be difficult. But it is, again, as I said, it's sort of like looking at the internet in 1989 or 1990. You're not going to be able, never mind YouTube, you can't build Amazon in 1990. Right.
And so let's fast forward to whenever you can build the Amazon. Maybe it's a decade, maybe it's 2 decades. whenever it is, how do you think about what will be built on a blockchain, or even more broadly, what will be decentralized? And then there's also this vision of effortlessly giving users upside. So what kind of vision will be using in this Web3 world? What part of the web versus, hey, maybe it's just better centralized the way it is now.
So there's a sort of— people kind of rush for metaphors here. Metaphor I always kind of liked was that if I stand next to a highway, I trust that the drivers won't hit me. If I stand next to a railway line, I don't have to trust that the train won't hit me. It's like those Harold Lloyd movies where he stands in the middle of the railway junction and the train goes straight past him. He knows he's safe. He can see where the tracks are. He can see the junction. He can see the movement of the switches. He knows the train isn't gonna hit him. And so, part of the idea of running this stuff on a blockchain is that you can sealed code, so you can see how it works. And Chris Dixon, I think, called this the can't be evil thesis. And so you would need a vote in some way of all of the users of the system in order to be able to do that, to change anything. And then in principle, all the users of the system would have a vote and they would have some sort of share in the economics. And so you use this thing and it works, and therefore you were there early, and so you've got more Equity is the wrong word, but you've got more ownership, you've got more stake. And so you build your Instagram, just go with this as a metaphor. And so the more followers you have, the more share of the ad revenue you get, or whatever the revenue is, if people are paying to post or paying for whatever it is. The earlier you join, the more you get, the more followers you get, the more engagement you drive. And that can all be in the code, and you can see the metrics, and you can see, it's like looking at a piece of clockwork, you can see how it works. So that's all very sort of compelling and interesting. And I think a lot of this reminds me of open source, both literally in the sense it's sort of an open source computer, but also in the sense that open source people were very religious. And so there was this whole idea that open source was going to destroy Microsoft, that no one would ever pay for software, that buying software was evil, that selling software was immoral. It wasn't just like a bad model. It was immoral. to do this. It was an ideological kind of religious objection. Of course, what happened is 25 years later, the internet, everything runs on open source, but consumer— and yet it kind of doesn't. Everything is open source under the hood, but Facebook isn't open. And macOS runs on open source, but it's not open. And yet it has millions and billions of app downloads. So what does it mean to say that it's open? Microsoft owns GitHub and people still pay for Office 365. So the reality when the religious thing kind of gets deployed can be a bit more complicated than that. And, you know, if you kind of describe that model of, okay, you've got this Instagram running on this open source system and everyone can see how it all works. And so I'm an influencer, I post an ad, the advertiser puts the money into an escrow account on the blockchain. I can see it's there. When I post the ad, mechanistically, X% of the money goes to me, Y% goes to the early stakeholders in the system, Z% goes to whoever it is that's running the infrastructure. And it's all in the code and you can see it all. Now, the problem with this, I think, is basically to take everything that people in the late 18th and early 19th century said for why democracy won't work, which is, yes, but how do you deal with the tragedy of the commons? What happens with the mob? What happens if people are tricked? And so you construct these elaborate constitutions and say, we'll have separation of powers and we'll have a court and we'll this and we'll have that. And we have a rule that says the magistrate has to step down over this period. And if you look at the history of the last— Americans tend to not really understand this, but if you look at democracy in the 19th and 20th century, you can point to probably 10 or 50x more places where that didn't work. Look at all those African countries that got their nice model constitution the day of their independence, and where were they 5 years later? Look at all Latin America. They all had these nice model constitutions. And what's the history of democracy in Latin America in the last 200 years? Or in Europe, look at Germany, look at Italy. They had these nice democratic constitutions and they kind of broke. So it's very easy to say, well, we'll structure, we'll write this constitution for how this Instagrammy thing is going to work. What actually happens if it turns out that you need to change something and the people that have the votes don't want to change it? What happens then? How does that work? What happens next? Is it a good thing for everything to be securitized? Is it a good thing if every mechanic in a social network has money, could have money attached to it? Is it a good thing if I'm being paid for every post? Is it a good thing if I'm being paid for every penny of engagement, every unit of engagement that I get? Because that creates— what's the phrase? I always forget who it is, but there's somebody who won the Nobel Prize for pointing out that if you use any economic metric to set economic policy, then that metric will stop reflecting Goodhart's Law? Yes, exactly. It's Goodhart's Law. Yeah. So Goodhart's Law would apply here, that if you suddenly— if all of the metrics and the engagement within a social network are now subject to money, that will change how it works. That won't just be kind of a neutral externality, that will change the mechanics of it. So you kind of— you can kind of propose this kind of model, this ideal fantasy world, and then you say, well, yes, but what will actually happen? And you have these people who sort of say, well, there'll be no gatekeepers in this world. Well, Really? I always thought there'd just be new kinds of gatekeepers.
BitClout comes to mind as a model that tried to financialize Twitter in an extremely aggressive way.
Yeah, there was a bunch of stuff wrong with that. One of them is doing the old thing but on the new architecture almost never works. You have to do actually some new thing that takes advantage of the new architecture. The other was, it wasn't It was also the wrong time because Elon hadn't bought Twitter and screwed it up at that point. Yeah. And it was also that it wasn't just that it was on the new architecture. The whole pitch was it's Twitter but with Bitcoin, which reminds me an awful lot of open source projects 20 years ago where they would say it's Microsoft Office but open source. And guess what? But open source is not a selling point to anybody. It's a selling point to engineers. So if you are building engineering infrastructure and it's open source, that's actually a selling point. But if you just want to write letters, the fact that the thing you're using to write letters or do spreadsheets is open source, you don't care. It's not a feature.
Yeah, it is interesting because a lot of people are building Twitter clones in some capacity, especially as you mentioned, now is the time. And Farcaster is one that is effectively a Twitter clone, but with 2 things enabled by crypto. One is that you can't censor someone. 2 is building on top of the platform for developers, whereas Twitter shut down its API.
Yeah, I mean, the censorship thing, again, I think is one of these kind of weird ideological back alleys that if you go and ask 3 billion people who use social networks, what is the censorship problem with social networks, they will tell you the problem is they don't take down enough stuff. The problem is they don't get rid of enough Nazis. It's not that they take down too many Nazis.
And so then it's a question, is the API, is the developer building on top of it a big enough thing?
Yeah.
Well, this goes back to the democracy question, which is, who is it that makes— you have to have somebody who's making decisions. You can't run a country entirely on referenda. You need to have some sort of executive power. And so there has to be somebody that can say, no, yes, we're shipping this. No, we're not shipping that. And I think you could argue this is one of the reasons why open-source projects never manage to produce a breakout consumer product back in the early 2000s. That could be a whole other podcast. But one of them is that open-source projects were systemically extremely bad at user interface. They were very good at building engineering infrastructure. But, you know, desktop Linux was a joke. This was the running joke. This is the year of desktop Linux. And, you know, here we are 20 years later, there is no desktop Linux for consumers. No consumer uses desktop Linux. It's only something that's used by engineers. And then you get the engineers saying, oh, but Android is running it. And yes, but people use Android because it's not— they got rid of all of the stuff, abstracted away all of those UI issues, and Google sat and designed the UI. And so that again applies even more so, I think, with blockchain is how do you abstract away the UI? And this is, like, kind of one of the kind of elemental questions I think we're building something on a blockchain is how do you get to a point that the user doesn't need to know what architecture you've used? Or let me rephrase that. You can do that relatively easily, but if you do that, then what's the point? Because if the point of this is that it's portable, and composable, and addressable, and they have ownership, if you abstract that away so that it becomes easy to use, then what was the point in doing it? And so, there's a kind of a weird, like, there's a interesting product balance to find in that. How do you give people the benefits of ownership and portability without making them like print out their seed phrases? What is the reason why you're not using— just using a database? Can you— those reasons survive if you are not requiring people to install MetaMask and audit smart contracts? You know, what is— is there a path between those two, although, or are they fundamentally in conflict? And I don't think there's much agreement on that at the moment. What you certainly can't do is say, well, everyone's going to have to learn how to use this, because that just doesn't work. I mean, this was sort of, well, everyone will have to learn how to compile their own code and how to install new kernel extensions because Linux is so great. No, no, fuck off. No one will do that. They just want to turn the computer on and have it work. And that was solvable in open source because, like, macOS runs on open source. The user doesn't need to know. But I'm kind of repeating myself here, but a lot of the challenge for Web3 is that the reasons you're using it are things that the user would need to know about. And to your music example, why aren't you running that on a database? Now write down your answers and say, but do the users know about that? Because if the users don't know, then what's the point?
Are you more bullish on reimagining kind of internet applications or Things like Instagram, as the example you brought up, but in a native Web3 way, as opposed to rebuilding the financial system or reimagining the financial system?
So again, several observations here. One of them is, I go back to my point about Netflix, that an awful lot of these questions are basically financial services questions, not tech questions. Second, I think a lot of the discussions here are basically Americans wishing they had stuff that they don't know, maybe don't know, that the rest of the developed world had like 20 years ago. Like, um, you know, I can do a free instant transfer. Yeah, I think I was doing that in like 1998. Yeah, we can get rid of checks. Like, I swear to God, I moved to America, I had to pay my check for the rent. I had to phone my father and say, how does this work? What is this? Because I'd like— I'd done like 2 checks in my entire life. What does it mean if they say they want me to endorse a check? I have literally no idea what that means because that's something my father hasn't done since the '80s. And so there is this whole chunk, and I think you could see this is kind of in PayPal as well. There's this whole chunk of leapfrog the last 40 years of the financial services industry that America kind of just never had, never happened in America. So that's the piece. And so then I think Again, there's a strain of ideology. So I mentioned religion. There's an open source sort of religion of let's get rid of gatekeepers, let's get rid of Google and Facebook, where previous open source people want to get rid of Microsoft. There's also this sort of gold bug, fiat currency inflation, central banks evil, the government might steal my money thing, which again is very ideological. And look, if you live in Turkey or Argentina, then the government might either literally steal my money or deflate away half the value of it this year is a real thing. And keeping your money in gold under the bed is like a rational thing to do. Whereas if you live in Germany in the '20s or whatever it is, and if you were Jewish in the '20s, then having diamonds might be quite good, although sadly it tended not to work. If you're living in Afghanistan, being able to put your money on a USB stick sounds like a great idea. If you live in Britain or France or America or Japan, this just sounds insane. Like, why the fuck would I want to do that? I'm not worried about the government stealing my money. And yes, you say this to people and they think you're an idiot. Most people are not worried about this. Most people do not own gold bars. That's not a use case for most people, which is, again, like being able to compile the source code is not something that most people care about. And open source people could never accept that. Strip all of that away. Yes, there's some interesting things in there somewhere. Quite a lot of them have sort of speedrun the last 100 years of financial services innovation and financial services regulation, or don't understand why we had that. Here we are. Useful book for anybody in DeFi. Whoops, there we are. Reminiscences of a Stock Operator. Nice illustrated edition with the original artwork from the Saturday Evening Post. And this is a biography of, what was his name, Jesse Livermore, who was a famous stock manipulator. And this is in the days when there basically weren't any laws about what you could do. I think an awful lot of people in the last couple of years were basically working their way through that page by page, and they didn't then go and look up Jesse Livermore and find out what happened to him, which is that he went bankrupt 3 times and killed himself. There's a science fiction story about an alien race that, like, the humans encounter an alien race and sort of beam down libraries to them. And one of these sort of alien politicians reads Machiavelli and goes and seizes power and says, I should probably read Julius Caesar next. And of course, gets assassinated.
Yeah.
And so there's a lot of speedrunning Wall Street here. Within that, yes, there's some interesting ideas about, but mostly these are kind of Wall Street conversations. Can you do interesting new stuff? I don't know. Ask a commercial banker.
It is interesting. Science fiction works both ways in terms of it inspires, but it also deflates. I used to be really excited about income share agreements. And then I read this book, The Unincorporated Man. In the book, one corporation owns a stake in everybody that somewhat feels like they're slaves to the corporation. And that got me less excited about the idea of income share agreements for ownership and everyone's upside. A couple of things before we get to the AI portion. There's 10 different ways of describing the metaverse. What is the most accurate way of describing it? Is it the idea that we're going to live more of our— especially as the headsets or tools get better, we're going to live more of our lives on the internet and thus a whole method of interaction and economy?
So I would have agreed with you, except that now I encounter people who think that you'll be able to use the metaverse your smartphone. At which point, to me, I have literally no fucking idea what you're talking about. That's the internet. You don't need a new word to describe the internet. So there's 2 broad— I would say there's 2 buckets to talk about here. One of them is relatively easy to understand, is the idea that some combination of VR and/or AR, meaning these glasses are displays, becomes the next universal device after smartphones. And so VR gets much smaller, gets much lighter, gets much better. We work out the optics to have something that looks like this, maybe powered by something in my pocket, whatever, that can be a display that can place things into the world around me. That becomes a universal device. That's a pretty straightforward, easy-to-understand thesis. Now, the basic arguments against it is, A, the optics will never work for AR, don't work. But for VR, the argument very simply is— If I'd shown you a PlayStation 5 20 years ago, you'd have said, oh my God, this is amazing, this is the future. And guess what? Games console install base is sort of 200 million units, maybe 250 million units. It's not a small market, but it's not a universal device. Most people see a games console game and go, that's very pretty, and walk away. And now there's a rabbit hole we can go down around, yes, there's 3 billion people playing games. Well, no, it's probably about 1.5, and most of that's Candy Crush. And like, what's the— like, how many people are actually playing games? And are they playing games on smartphones? But the core of it is that VR is, by default, this sort of deep, narrow experience. And maybe it will just remain a deep, narrow experience. Maybe it will remain a subset of games consoles. And most people will say that's very pretty, but won't use it, and it won't become a universal device. AR, there's more basic science problems to solve, but we'll see. But that's like a narrow metaverse thesis in which when you say metaverse, you're basically saying it's like saying mobile internet. It's as though we come up with a cooler word, the cooler way of saying mobile internet. Well, this is VR internet. There's a much broader and much more hand-wavy description, which is, again, it's back to going back to talking about 18th century philosophers. It's sort of imagining this imaginary constitution. So the thing 18th century philosophers do is they describe some imaginary country. And Plato does this with the Republic. Like, it creates— it describes an imaginary country which has an imaginary constitution where everything works incredibly well and all the stuff we don't like about our own society society has been fixed. And so this is also part of the problem with Web3. It's like, in Web3, there'll be no gatekeepers because of this, but that doesn't solve the reasons why there were gatekeepers. And in my imaginary kingdom, the judges will all be just and the king will always be honest. And like, yes, but why will they always be that? And the same thing with metaverse is like, All— everything will be interoperable, everything will be portable, you'll be able to move assets between every different thing. Like, why does that make sense? I mean, I always reference that movie Wreck-It Ralph here, if you ever saw that. So, like, video game characters move between games, basically. So, but the kind of the point is, okay, so I'm— so with all my assets and all of these different metaverse environments are all portable and I can take them from one place to another. Well, why? Why is it that companies don't let you do that now? Why would that change? Why would they do that? But more fundamentally, you know, I go into Fortnite and I buy a 15-foot-tall yellow banana character with a gun that shoots exploding tomatoes. Cool. Okay, now I go and open up, um, my Top Gun game and I'm flying my F-14. So what am I What am I sitting in the cockpit of the plane as a giant banana? Like, what does that mean? I go into Call of Duty and I'm sneaking through the forest, like crawling through the mud with the other VR— with my other VR special forces guys, except I'm a fuck-off great yellow banana shooting smoke. Like, it's like the reason the assets aren't portable between the games is kind of because the games are different.
Yeah.
It's partly the business model, yes. And, you know, You can't move assets between Fortnite and PUBG, and maybe that would make sense, but— and they don't want you to. But it's also that they like— they're designing the game, and the people designing the game want to be able to control how all the different components work so that it's a good game. But it's also that, you know, as I say, if, you know, go back— I buy an F-14, you know, I buy an AC-130 gunship that's got a bunch of 30mm cannons and a couple of recoilless rifles on it, and I take that into Call of Duty. Like, what happens? Does everybody else have to go out and buy an F-14 and take that? Like, then it's like it's a whole other game. So, I mean, a lot of that sort of portability interoperability stuff, it just seems to me like kind of vague hand-waving that's completely detached from any discussion of what the actual product would be. And we were talking a while ago about how long this stuff takes as well, and then maybe this is sort of a more fundamental point. You know, presume for the sake of argument that in 5— between 5 and 10 years' time, say in 5 years' time, VR really takes off. before AR really takes off. Apple announces something this summer. Apple announced the iPhone in 2007, sales didn't take off till 2010. So Apple announced something this summer, by 2025 to 2030, you've got hundreds of millions of people using this. Okay, now imagine it's 2000 and you're trying to describe what the smartphone internet's going to look like. You can't. Anything you say would be wrong. You wouldn't even have got the App Store, although there were app stores. You wouldn't have got Instagram, you wouldn't have got Uber, you wouldn't have got TikTok, you wouldn't have got all the— you wouldn't have got Tinder, you wouldn't have got all the stuff that mattered about how it worked. And the same thing now, like, even if you believe that VR is going to be the universal device and 3 billion people will have one in 10 years' time, any prediction you make today about the specific market structure of that and who's going to be in charge and how it will work is going to be wrong. And it's kind of pointless to even attempt to do it.
It's interesting, this idea of gaming companies keep their virtual goods in-game because if you could make it portable, perhaps they'd be less likely to buy. There's this broader question of anytime you provide more consumer surplus, does that just mean the new version of the business that had this sort of moat that is now less powerful because of the new technology? Is that new business just smaller or do they find this new business model that didn't exist before that now enables them to be bigger. Do you have a perspective on the idea of anytime there's more consumer surplus, are the businesses usually bigger or are they—
It's difficult to answer. I mean, on the narrow game point, I think there's a business model point and there's a product point. I mean, in a different context, it's kind of like saying that Photoshop and Excel should be interoperable. What would that mean? The interoperability layer is the file system and the copy paste. But the idea that Excel should be able to open Photoshop files, Well, then it would be photoshoot. Like, what is— I don't even understand what that would mean. And on this kind of surplus point, I mean, we've had this kind of long-term shift away from marginal cost, which is actually one of the things that's interesting about generative machine learning now. It's like, when is the last time it cost money every time a consumer clicked okay in your app? When is the last time, like, your consumer clicks okay and it, like, takes 30 seconds for something to happen. Like dial-up, maybe, but not even dial-up, really. We're kind of in mainframe territory here. We're in 1970s timesharing kind of territory where it actually costs a material amount of money for a consumer to click on something. And one of the kind of elemental questions about LLMs or whatever you want to call them, generative machine learning, is the second question after the AGI question is, what's the cost structure and the size of the models going to look like?
Let's transition into the AI conversation. You've asked, is this the iPhone moment? Is this the internet moment? Is this something way bigger than both of those? Why don't you reflect on the different forks from here and why that's important?
There's an urban legend that I think during the Cuban Missile Crisis, there's a rumor that the missiles have launched on the stock exchange. There's a rumor in the stock exchange that the missiles have launched. And everyone starts selling and somebody, a veteran trader, goes out and starts buying. And he says, well, look, this is binary. Either the rumor's true, in which case we're all dead, or it's not true, in which case the stocks are cheap. And that's kind of how I think about AGI, in that no one knows. It seems deeply unlikely. There's a lot of very kind of bad logic or illogical argument. But basically, people who spent 20 years arguing about this stuff don't know. Well, they certainly don't agree. And I can't arbitrate between Yann LeCun and somebody else about who's quite right about what gradient descent is going to do, because the only gradient descent I know about is going down a hill in second gear. So I don't fucking know. So it's almost like, take the AGI conversation and kind of park it because we don't know. It seems very unlikely, but So then the question is, then you get into like 10 other questions. And I think most people in Silicon Valley, everybody in Silicon Valley, as I say, is kind of walking around with their kind of trying to hold onto the top of their head with both hands to stop it flying off. Because this is clearly like an enormously huge thing. But then the question is, well, okay, but what is it exactly? And then there's like 10 questions about, well, what is it? How does it work? What does this mean? I think the spread of opinions is that there are people who would say, okay, this is a second wave of AI, and that was a big second wave of machine learning, which has been a very big deal for tech in the last 10 years. Or it's more generally, you could say this is sort of equivalent to the iPhone in that it sort of completely resets a huge swathe of how consumer internet experiences will work. That seems a relatively straightforward thing to say. You could push it a little bit further and say this sort of looks a bit like the internet. I think Bill Gates said that it reminded him a lot of Xerox PARC. park and seeing a GUI. Because what a GUI meant was a step change in who could use computers, because you didn't have to learn how to use a command line. You could just click on stuff and you could teach yourself much more. But somebody else had to write the program, still had to write the program, or you had to learn how to program it, or you had to buy the program or something. But the program had to be created for you to be able to use it. Whereas with an LLM, in principle, you can kind of just ask the computer to make the thing for you. And so it's, again, a step change in how much software there can be, what software there can be. how accessible it could be. It's a bit like Excel. Excel is the world's biggest IDE, like something you can just go out and create a program in it. It's like going from Excel to Bloomberg. It's those kinds of step changes. Then there's people who I think are a bit more overexcited. They say, well, this is electricity, this is computers, this is fire. I'd again come back to that sense of how quickly this stuff gets deployed. Cloud is a relatively old and understood technology. And if you're in the tech industry now, it's like, it's boring, it's done, it's happened. It's only sort of 20% of enterprise workflows. Google just moved from Oracle to SAP to process their internal payments. I mean, this is like the joke, the single biggest thing protecting us from AGI is that everybody's using Discord. That slows everything down. So, you've got all of this kind of swirling mass of ideas. And I don't think anyone— we're kind of struggling to kind of pin down to even, okay, what are the half dozen or dozen questions, never mind what are the answers to that?
We were talking to Elad Gil and Sarah Guo from the investor perspective. You're a venture partner at Mosaic, and I'm sure they're asking you, how should we be thinking about that in terms of what are company opportunities that will be durable and defensible and not just taken up by the big the big companies. How do you think about what are the startup opportunities, or how would you approach this as a venture investor?
So some sort of a priori assumptions. One of them is you go and use Midjourney, and the first thing you think is, okay, imagine what this would be if it had some product around it. This is amazing. And then, I mean, obviously using it in Discord is just ridiculous, but Just the principle of just typing stuff into a prompt is not a great consumer UI or great UI for anyone. It's very powerful, but it's kind of, you know, it's not a good user interface. You know, there's clearly better user interface and better product to be built around that. And, you know, in general, like, there isn't one database company. You know, you don't just go and buy Oracle. There's millions, hundreds of thousands of different database products that do different things. And when you do your expenses, you don't say, like, I'm gonna use some database now. It's an expenses product. And so, you kind of need to build the product and go-to-market and the user experience and the customer success and the support around creating a tool to solve some problem within a company. And now that thing may all be running on top of OpenAI. There may be 3 foundations. This is sort of the phrase foundation model, which I hadn't heard until 2 weeks ago. So it's a bit like metaverse. What does foundation model mean? It may be that there's 3 models and everything else plugs onto the top of those as APIs. So that may be one way it works. There's those 3 models and then maybe there's some middleware and then there's the 10,000 companies software companies in the world all plug onto that. That's one path. But even if that's the case, even if there are only those 3 models, when you do the, we need to validate that we've done criminal records checks for people getting hired as security guards in this shopping mall, they're not just going to go to ChatGPT and say, hey, do a criminal records check. That's not how it works. I mean, yes, if we had AGI, maybe it would. If we actually had god-level AGI, then yes, you could just say, hey, God, tell me, does this person have a criminal record? Done. But you need an actual specific product all around that. So I think that's a basic question. It's a basic a priori assumption. Then everything becomes questions. Why is it? At what point does this— how long does this stuff stay this expensive? What does it mean if it does? Are there always only going to be 2 or 3 models, or is it going to be more like machine learning where we got to a point that it's like databases, there's millions of machine learning models? So that's sort of a second-level question. And then maybe the third question would be following on from those. What happened with the last wave of machine learning was that you sort of say, well, how do you have the data that's specific to this How do you get product and customers before— how do you build the product before you've got the data? How do you get the data before you've got customers? So, it's a kind of circular question. You need customers to get data, but you need data to have the product to get to customers. And does the product get better indefinitely if you get more data, or does it reach a point where it kind of works? There's an S-curve. And how much data is that, and how many people could have that? Is that gonna be like 10 people, or could anybody have that So those are just sort of very, very early sort of how does this work kind of questions. I think stepping back, to me, the first is the AGI question. The second question is the cost. And the third is the error rate, which kind of is the AGI question. If the error rate goes to zero, then you've kind of got an AGI. Maybe, maybe not. But right now, if I ask ChatGPT-4 the current one, write a biography of Benedict Evans, hit it reload 4 times, you'll get 4 different biographies of 4 different people. And they overlap quite a lot. But, you know, my first job was investment banking. Yes, but at DKW?
No.
No, it was at McKinsey. No, I worked for The Guardian. And what's happening up here, of course, is it's pattern generation. It's not answering the question. It's not doing a database lookup. It's saying, what would an answer to that question look like? Or even, what do answers to questions like that look like? Which is why it matters how you ask the question. So if you ask the question and start by saying, imagine you're an expert, then you get a different answer because you get the kind of expert answer that an expert would give. By extension, you could kind of say, imagine you're an idiot and don't know anything about this, and then you'll get an even more— So what it's doing is it's matching the pattern of the question with the pattern of an answer. And at the moment, that's like, Pick a number, 60%, 70% accurate. At what point does that get to be 80%, 90% accurate? In what domains does— what accuracy do you need in what domains? And can you see the accuracy? We're still at the stage where it's happening so fast that there's always new questions and new ways of understanding it.
Well, as you were drawing that kind of stack diagram, it brought to mind a theory that I've heard you talk about before, which is that there's 2 ways to make money in software. Bundling and unbundling. So my theory about what AI is starting to do right now that I wanted to bounce off you is I see it unbundling jobs into tasks and thus enabling kind of a re-architecting of how stuff gets done with the task as the fundamental unit of work. And then on the other side, the bundling side, I see things like ChatGPT, which is certainly first incarnation, starting to, with their plugin architecture, bundle all of the apps that we use in life into this kind of new super app where you talk first to a consistent interface and then it goes out and deals with all the APIs and browses the web for you. And at some point you may want to get into that experience, but to the degree that you don't, It can kind of handle it for you. So what do you think about that bundling-unbundling lens on AI?
The metaphor that I always use is I always talk about Jack Lemmon in The Apartment. It's a Billy Wilder film from 1960 where he's a clerk in an insurance company and you've got these long shots of this office floor in an office building in Midtown Manhattan. And everybody has kind of a one-person desk with a typewriter and a Rolodex. and an electromechanical adding machine and an inter— stack of internal mail envelopes. And so what happens is a document arrives, you do the calculations on this adding machine, you type the results with this typewriter, you put it in internal mail, you write somebody's name on it, you give it to a mailboy, he carries it off to another desk somewhere else in the building. And so the whole building is basically an Excel file. Jack Lemmon is a cell in a spreadsheet. The whole building's a spreadsheet. Once a week, someone on the top floor presses F9 and the whole building recalculates from top to bottom and generates new insurance prices. And in— I mean, as you may know, computer originally was a job title. That was a person. He is a computer. That's what his job is. And in 1965 or 1970, they bought a mainframe, and all of that processing, information processing, got moved into a mainframe. If you go and look through old census reports, you'll see these kind of job categories like data keyers. A data keyer is somebody who is holding a piece of paper and is keying the data into a computer. Which is not a job anymore, except for the IRS, obviously. And that's actually not a joke. That's literally what happens. If you kind of think about those cycles over the last 75 years, what would we think is it that's happened? Is it people used to do data entry and input and calculation, and now they do more decision-taking and they do maybe more routing, and there's a lot more sort of, this problem has come in, who needs to send it to, do I need to solve it to, and it goes to that person, and they do these 3 things and then press this. So, you know, if you were sort of more of a sort of a futurist kind of person than me, you might kind of come up with some phrasing here of people used to be tools and now they're routers, or I don't even know what that would be. The way I tend to look at this is that you sort of automate more and more things, and you free people up to do less repetitive tasks and more creative tasks or more tasks that require high-level brain functions in some way. We're going to automate another kind of another class of job. I mean, if you read or listen to Dan Bricklin, who created the first spreadsheet software, VisiCalc, you know how he has these stories? Because, well, again, like, we don't know now, but a spreadsheet was literally a sheet of paper with all the cells printed on it. And so a financial analyst would have like a past stack, like a printed pack of these sheets and use those to do the calculations with a pen and paper, or maybe like a hand calculator. And so all he did was take that and put that onto a computer screen, and he would show this to people, and they would like literally lose control of their bodily functions because this was like years of their life that was being done in seconds. And he has these stories of people who would like— had been given a job, an accounting analysis job that was expected to take like 3 months, and they did it in an afternoon. Literally, from months to afternoons. And what happens as a result of Excel, of spreadsheets and then Excel, is that you have more analysts. It's not that you have fewer analysts. We mentioned Goodhart's Law. The other law here, I think, is Jevons' Paradox. So Jevons is a 19th century economist. And at this time, Britain obviously has by far the largest navy in the world, and it's converted to steamships. and Britain is basically made of coal. But people are saying, well, we're using all this coal really fast, and we might run out, and then we'll have a serious geopolitical problem. And then a bunch of people say, yes, but the steam engines are getting much more efficient every year, so we'll use less coal. And Jevons says, no, no, no, no, no. If you make steam engines more efficient, we'll use more coal because the steam engines will be cheaper. And so when you make the use of the resource more efficient, you use more of the resource, not less of the resource. And so what I think this very obviously applies to computing, that when you make it cheaper and easier to do the analysis, you do more analysis. We don't do less. I mean, you can— we can all just sort of think of examples of this in our lives. You know, you go and spend 3 days in Excel doing analysis. If you'd had to do that by pen and paper, you just wouldn't have done it because it would take you like a month to do what you just did in 10 minutes in Excel. And so I think those are sort of building blocks that I have for thinking about what this does to employment. It's that it will be— I mean, it's a Steve Jobs line that a computer is a bicycle for your mind.
It seems like it's both to me. I certainly see the bicycle for the mind paradigm and it seems like that probably is the dominant benefit for a certain class of people. But then I look out at the world and I'm like, So many of these jobs really are largely routine. And I wonder if the right way to think about this is not so much bicycle for the mind, but dominant technology. You can certainly find plenty of examples of technology changes in the past where it's like, we don't use more steam engines, we just have a better form factor that Serves the vast majority of the use cases.
I mean, there's analysis that people did of motive power in England in the 19th century, of beasts of burden at the beginning of the 19th century, and the aggregate horsepower of all of the steam engines at the end of the 19th century. And I'm sure if you ran that through to today, the number would have increased by many more orders of magnitude. The economics textbook argument here would be that it's easy to see the jobs that any new wave of automation will remove because they're right there. You can go and look at them. You can see those jobs. It's more difficult to imagine the new jobs that will get created because you have to imagine them. They don't exist. And yet those jobs happen. And if we look over the last 200 years, there's been this continuous process of jobs getting automated away and new jobs getting created. That happened with the internet and it happened with PCs, and it happened with GUIs, and it happened with computers. So the default presumption would be that this will just continue. And the fact that we can't imagine what those new jobs will be doesn't tell us there aren't any, because that's been the pattern for the last— nobody else, nobody could have imagined our jobs 20 years ago. I think the counterargument to this is that what you've been doing is sort of moving up the scale of human capability. And so you start with human beings as beasts of burden, I mean, the other image I use— so I use the image of Jack Lemmon in the apartment. I also use this image, which is a painting, 19th century Russian painting by Ilya Repin called Barge Haulers on the Volga. And it is basically a dozen guys pulling a barge upstream by hand, like leaning on a rope, pulling a barge upstream. And just on the horizon, you can see a trail of black smoke from a steamer that tells you how this is all going to change. And nobody is employed to pull barges up the river anymore. And yet those people got other jobs. But what happens is you go from beast of burden to you go to people's arms, you go from legs to arms to hands, so to speak. And you're kind of moving up to the top. And in principle, you could kind of get to the top and there won't be anything else left that you can't automate. The problem with that is, again, you could have said that 50 years ago. You looked at all those people in that building and you said, well, all of those jobs will get automated away. These are all these boring competitive jobs. So what will those people do? Guess what? Insurance companies probably employ just as many people as they do now. Certainly we haven't had a net change in employment. And so there is this sort of, we can't know what the new jobs are problem.
Yeah, I think it's always important, and I always try to keep in mind too, that it's not going to be a binary where it's all one of these effects or all the other. We'll definitely see both. And the question is which one will prevail. predominate. But I wonder if you would venture a prediction when it comes to, say, insurance companies. I would imagine that a lot of that headcount has gone into agent networks and effectively sales and broadening the market. And now I imagine a 2025 world where I go to allstate.com or whatever, and instead of a connect with your local agent, or maybe alongside of a connect with your local agent, there is a talk to your virtual agent. And that virtual agent is certainly going to have some advantages.
But again, this is what's been happening for 100 years or 150 years. I mean, I was talking about this earlier, like we were talking about travel agents. There are no travel agents anymore. Before the internet, if you wanted to book a flight, there were these stores that you went to and they had the flights. They had books with the flights, or maybe they had a terminal that connected to the airline system with the flights and they would make you the tickets. Gone. Those jobs are gone. So what? Yes, it sucked if that was you and that was your business and you'd spent your life building this business. And there's always a frictional short-term pain of these jobs. But the process, the cycle of jobs getting destroyed has been happening continuously since the Industrial Revolution. And so all you've just done is just describe another job that will go. Well, yeah, those jobs will go away. The question is whether there's some a priori reason to think why there won't be any new jobs now, as As opposed to all of the new jobs that got created over and over for the last 200 years.
I think another question too, though, is how fast does the destruction happen? Because when you go back to the Industrial Revolution, you are talking about a multi-generation process from the first piston to a refined steam engine. And here it looks like we go from GPT-2 in 2019 to GPT-4 in 2023, and that's the leap from Basically, Babbel to expert-level human doctor. Our next guest on the Cognitive Revolution is a guy who took GPT-4 on rounds.
If you're in Silicon Valley, cloud is— if you're in the tech industry like all of us, cloud is this old boring thing and it's done. But cloud is only sort of 20% of enterprise workflows. Apple announces the iPhone in 2007. It took until 2010 for the sales to take off and it took until 2015 really for it to really take over the majority of the phone industry. I think we can sort of underestimate how long it takes to get the Scottish police force to swap out the system that they use for doing criminal records checks. They're not going to do that this year, or next year. In fact, the UK sales tax system runs on DEC.
Hmm.
Literally runs on DEC computers. Which DEC hasn't existed. And I think Compaq bought DEC in '99. The company hasn't existed for 25 years. So, you know, we can sort of— I mean, on the one hand, like, this whole thing is moving incredibly fast, partly because there's a sort of standing on the shoulders of giants point that you don't have to wait for people to buy phones. You know, you don't have to wait for everybody in the world to buy a smartphone. It's just everyone's already got We've already got cloud, we've got the web, we've got cloud computing, we've got the GPUs, we've got Stripe, we've got everything. Everything's right there. So you can just add this one idea and kind of, okay, now there's 1,000 generative AI companies and there's like 10 new projects on Product Hunt every hour. So that, yes, on the one side, you've got this very rapid speed of creation. On the other side, You want to go and get Allstate to deploy this system into their call center? Okay. It'll take you 3 months to get a sales meeting. They will not deploy that this year. It will take them 3 years to deploy that, if that. So, like, the entire world doesn't just change its infrastructure in 6 months, even if the product that they could buy is ready in 6 months, which actually it won't be. It will take longer than that. So, this stuff will take 5, 10, 15 years to get deployed. I mean, there's another way of thinking about this maybe is we could have built TikTok in 2010. You argue a little bit about what bandwidth looked like and how smartphones were, but it took 10 years for that idea to happen. Actually, a better example is we're using Riverside. One of the companies I used to like talking about is Frame.io. Which is basically collaborative video, professional video, not actually editing, but everybody who needs to see a piece of professional video. It's all the commenting and the version tracking and the sharing. So the editor can post the latest version and 50 people can see it and you can work on it and comment on it and mark up that bit on that slide. You could have described that in 2005. The technology was probably ready by 2010. It was probably cheap enough by 2015. It got built in 2018, it got bought by Adobe. It takes time both for this stuff actually to get deployed and for people to work out the right instantiation in the right product, in the right go-to-market to get that deployed.
I think that's well put. This has been a fascinating overview on Web3, metaverse, AI. Maybe in closing, as a closing question, We're almost at May 2023, so we're almost halfway through the year. We get back on a call in February of 2024. What can you predict are the questions that you'll be asking in your next presentation?
Well, almost by definition, I can't. Particularly given what's happening at the moment, the questions will change. I hope we'll have a better sense of what the questions are and we'll have a better sense of some of the answers. Nathan, you may disagree, but I don't think we have a consensus on what the models are going to cost and how that's going to evolve. I don't think we have a consensus really on what's going to happen to the error rate and whether we get stuck here and we're at the top of an S-curve, or whether it's going to go all the way up and improve radically. We don't have a good sense of whether we're in a world of 3 or 4 foundation models or 50 foundation models. models, or everybody can have their own. And so those sorts of basic, how the hell does this work? What does the structure of this work like at all? We now have a better sense of what those look like. But at the same time, 2 weeks ago, no one was talking about AutoGPT. You mentioned plugins, which we haven't talked about. I feel ambivalent about ChatGPT plugins because everyone looks at this and goes, oh my God, this is amazing. Now I can do anything. I look at it and think, well, if it was so fucking good, why Why does it need the plugins? And isn't this just basically me using Alexa to talk to Booking.com as built in 1998? If this thing is absolutely fucking amazing and changes the world, how is it that it's using a Web 1.0 API to talk to a Web 1.0 company to answer this question? Well, I could just go to Booking.com and do that. And now all you're doing is giving me a natural language frontend to a Web 1.0 company. 1.0 company, which they could do. Now, AutoGPT in a sense isn't a way of solving that, which is, no, it's not one plugin, it can chain together 15 of them and actually solve a problem. Whether it can do that or not, I think we don't think it actually seems more theoretical than real at the moment. Then one of the things I've talked about as well is this sense of UI. I think we will move on from command line, and I think we'll have a better sense of what the right UI is for some of these things and the right way of presenting them will look like. You have this kind of, you know, there's this phrase prompt engineering. It's natural language, but prompt engineering, which is it? If this is natural language, why do we have prompt engineers? Because of course it's not. And if I actually want to describe a piece of enterprise software, I don't think I could do that in 500 words. What am I actually going to do? I'm going to paste in a 500-page PDF ChatGPT and press go and then it says, great, okay, so what do you want to do? I'm not sure how that's how it works. So there's a lot of hand-waving, which I'm guilty of at the moment as well. And there's a lot of, how do we get from this breakthrough in principle to 10,000 products and the stack that supports that? It's like, kind of look like, it's a bad analogy, but it's like looking at multitasking. And going, okay, what is this? What's this for? How do you use this?
I think radical uncertainty is an idea that we can all resonate with as we try to make sense of what's happening. But if you're listening to this, you're likely a student of technology. And as a student of technology, Benedict Evans' newsletter and podcast is a must. So Benedict, we're grateful for you for spending some time sharing some of your learnings and ideas with us. And look forward to continuing to follow along your writing and podcast and continue the conversation. Thanks so much for joining.
Thanks for having me.
Upstream with Erik Thornberg is a show from Turpentine, the podcast network behind Moment of Zen and Cognitive Revolution. If you like the episode, please leave a review in the Apple Store. Secure Frame is the leading all-in-one platform for security and privacy compliance. Get SOC 2 audit ready in weeks, not months. I believe in SecureFrame so much that I invested in it and I recommend it to all my portfolio companies. Sign up for a free demo and mention Upstream during your demo to get 20% off your first year. Now more than ever, startup founders need a safe place to put their cash. Mercury protects your money and also provides the streamlined user experience that great founders expect. Through partner banks and their sweep networks, Mercury offers up to $5 million in FDIC insurance, which is 20 times the per-bank limit. They also make it easy to invest any cash above the FDIC-insured amount in a money market fund. 100,000 startups trust Mercury with their finances. I've been a happy Mercury customer and have found their team incredibly helpful and responsive. They even got an important wire out of purgatory on Christmas Eve. After all, your Christmas is my opportunity. Visit mercury.com to get started. Mercury is a financial technology company, not a bank. Banking services provided by Choice Financial Group and Evolve Bank and Trust, members FDIC. MarketerHire is one of my favorite resources for growing startups looking to hire marketers. Hiring is hard and the time it takes from founders can be better leveraged elsewhere. MarketerHire pre-vets top-notch marketers across a dozen roles. Whether you need help with growth marketing, SEO, lifecycle, content, or any other aspect of growth marketing strategy. It's free to use and you only pay if you end up hiring someone. If you wanna hire a great marketer the easy way, MarketorHire is offering Upstream listeners a $1,000 credit for first-time customers. Go to marketorhire.com/upstream and use code Upstream to get your $1,000 credit.
More from Erik
Palantir CEO Alex Karp & Rep. Ritchie Torres on Innovation & The West | Hill and Valley Forum 2024
