Economics, Hayek, and Large Language Models | LSE Event

in the audience and watching online, welcome to LSE and to this public lecture hosted by the Economics Department at the LSE and supported by the Hayek Program. My name is Nava Ashraf. I'm Professor of Economics in the Economics Department, and I'm a big fan of our speaker tonight, Professor Tyler Cowen. Professor Tyler Cowen is the Holbert Harris Chair in the Economics Department at George Mason University and the co-host of what I'm sure you all know very well, the very influential blog Marginal Revolution, a regular columnist for the New York Times and many, many other fantastic news outlets. He has written 19 books which have brought rigorous economic thinking to many issues that face modern life. And we're so lucky tonight that he has chosen a topic so prescient— as usual, he chose it in what we learned of him in the fall— on large language models, and in particular how we can use them in dialogue to understand more about our world, um, and in economics specifically. So there will be time for questions at the end. Professor Cohen will probably talk for about 45, 50 minutes, and then we'll open the floor for questions both here and from the audience. For those of you who are Twitter users, the hashtag is #LSEecon, and so feel free to continue to spread the word as the theme of tonight is knowledge and spreading and learning about knowledge. I'm delighted to turn it over to Professor Cohen. Thank you for being with us.

There are two kinds of AI talks I give.

The first kind is to audiences who need to be convinced that large language models are super powerful.

The second kind is for audiences who don't need to be convinced about the power of large language models. I'm going to assume for the time being you're the second kind of audience. But if you need to be convinced, there'll be some periodic inserted rants throughout my talk to convince you of such. But I'm just gonna take it as a premise that human beings have for the first time invented some kind of general intelligence that on many tasks is actually smarter than most of us. And this is shocking. I was never sure I would see this in my lifetime. But if you give it a bar exam, a medical exam, it will pass. It won't just squeak by. It will do very well. One of my colleagues, Brian Kaplan, earlier in the fall, he gave GPT-3.5 one of his economics exams. It got a D. Brian mocked me.

I'm at lunch with Brian.

He and I, we like attack each other. It's enjoyable. I said, Brian, just wait. GPT-4 came out. Brian gave it his exam. GPT-4 got an A. So in 3, 4 months, we've gone from a D to an A on an undergraduate economics exam for labor economics. I have a 40-page paper I wrote with my colleague Alex Tabarrok How to use GPT models to teach or learn economics. So it's a kind of guide. It's online, it's on SSRN. So anything involving words, you can ask it. And if you ask it in a sufficiently intelligent manner, you will most of the time, but not all of the time, get a very good answer. So that's the basic factual setup. But as economists, we want to ask, how should we think about this development in economic terms? And those will be the core themes of my talk. So we have this thing. Someone on Twitter once described it insightfully as, all of a sudden, for $20 a month, you have a combination of a research assistant, a colleague, and an architect. All in one for $20 a month.

That's amazing, right?

Mind-blowing. Now notice he didn't say you have a carpenter, but you have a research assistant, a colleague, and an architect who can design things. Now one way of thinking about this is you could argue, well, we have increased the capital stock. That would be an economic way of putting what's happened. But I— it's a little more complicated than that. Because large language models, in some ways you could argue they're more like human labor. We've increased the amount of intelligence in society more rapidly by a larger amount than any other development in human history. So the printing press made books easier to print, circulate, but it took actually centuries for the printing press to have a big effect because in the meantime you could print books, but they were very expensive. It could cost you, you know, a month or two's wages to buy a book. Most people couldn't buy books. That you could print them was fine, but it's only in the 18th century in a few countries that it very much starts to matter. But with large language models, we have the internet. You plug in, you pay your $20 a month, and it's just in front of you, and it's then ask away. You can put in, you know, a 40-page paper and say, read the paper, summarize the paper, interrogate it about the paper. It does all of it like that. So people mainly think of GPT as something that writes. It does write, but I'm not sure writing is its greatest skill. Its greatest skill, in fact, may be reading rather than writing, but it does both. So you can think of it as capital or as labor, but it's some kind of new factor of production. So if you ask, how does this revalue our current factors of production, right? It's very much an economics question. Now that we have GPT, like, what is worth more and what is worth less? So prediction is difficult, but here are my best intuitive takes. And some of these we already see anecdotally in the data. First, if your job is doing routine back office work, you are either right now or within 2 years probably worth considerably less. So it seems that GPT models, especially once we train them a bit more on proprietary data held in individual businesses, they can render obsolete a fairly large number of jobs doing routine back-office work. Uh, that is worth less. But how, how to think about what is worth more? So memorizing facts is worth much less. I now use GPT in my work routine, I would say like 3x more than I use Google, 10x more than I use Wikipedia. I even have it on my smartphone. You know, there's a version of it where it recognizes your voice. You just speak to your phone, you ask it the question, you tap and you get the GPT answer. That's pretty remarkable. So to know like, which is the capital of Idaho?

Do any of you know what's the capital of Idaho? Boise, Idaho.

You don't need to know that. I just witnessed a major mistake, right? That was malinvestment. Don't need to know the capital of Idaho.

Here's what I think matters more, and it's probably a greater effect the younger you are, for obvious reasons, because you'll have on average more years of life in a GPT-rich world.

Personal charisma, I think, will matter much more. Actually, your looks will matter much more. When I say looks, I'm not sure that the benefit is to be like the best-looking person possible in the like model underwear sense, but looks in the sense of how you come across in person. Like not just do you know the capital of Idaho, but do you seem forceful, energetic, again charismatic? It seems to me people who do fact-based work You can't really give them homework anymore, but you'll judge people more by how they come across.

That may be unfortunate.

It might be more discriminatory, but I think that is all of a sudden the world we live in. Your personal networks will matter more. Who is it you know? Do those people trust you? What's the personal network of GPT?

I don't— it doesn't really make sense as a question.

So it might know everything. But it doesn't know anyone. You wouldn't say to GPT like, oh, write me an email of introduction, you know, this person to that person, because GPT doesn't really have credibility per se. Physical coordination will matter much more. So people who are very skilled carpenters, gardeners, people who do things in the physical world but who are not just doing brute force labor but who mobilize information, knowledge, understanding of context. My intuitive prediction is those people will earn much more fairly soon. With GPT, again, you have an RA, a colleague, and an architect. It basically means you have a lot more projects. But the people who will gain the most, I think, and I hope it's all of you out there, like, how good would you be if you had at your disposal a free RA, colleague architect. Some people are like, yeah, I don't know what to do with a colleague. I don't know what to do with an RA. Then I would predict, you know, your wage trajectory over time will be less than optimal. If you are amazing at managing that team given to you by AI, you will just do very, very well. You'll come up with a lot more projects. You see this in computer programming where GPT models have had the biggest impact. There's a case study done, you know, Italy for a small period of time, they banned ChatGPT in Italy for privacy law reasons. The next day, programming output fell by 50% overnight. So as a rough estimate, I know these are like complicated comparisons, but it seems ChatGPT was doing half the programming in Italy. And that was a while ago. It's probably above half now. Now Italy later brought back ChatGPT. People clamored for it. But what also happened in the meantime, people used VPN and they connected to ChatGPT outside of the Italian, you know, Italian IP addresses, and output, you know, went back to where it was anyway.

That's one indication, whatever you think of ChatGPT, you know, we're not going to get rid of it.

People will get to it one way or another. So people in programming, you have brilliant programmers, but they're bored with routine work. They just start a program into motion. They'll just tell GPT something. They'll say, "Oh, start writing the program on this." And then they can experiment and build out the rest. And what GPT writes, maybe it's only 80% correct, but to have someone else fix those errors, it's just a lot easier than the brilliant programmer having to do all the work from scratch. So that's a simple example that we already see. We're just programming. You can try a lot more ideas. Just tell it what to do for you. Yes, there are almost always mistakes. Much easier to fix them than to do it all from scratch. So again, charisma, networks, physical coordination for some jobs, and how good are you at managing your fleet of assistants? It's a question most of us don't think about that much. So those of you in the room who are professors, you have, have had RAs.

How many did you have?

Tim, how many RAs did you have? Like, one, two? One. Okay, how many do you have now? A lot more, but do you know what to do with them, right, is the question that will determine your fate from here on in. And if you started from zero, like, you really need to start thinking about I have all this free labor, what am I gonna do with it? It's some kind of act of imagination.

So when you ask who benefits, I think you can say on average, just most changes, most new developments, people who are smarter benefit more than people who are in some ways not as able to figure things out. But I don't think that's the dominant effect here. I don't think simply being smart is that much of an advantage. I've found a lot of the very smartest people I know, they have a kind of smugness, like, oh, I know all this, I know all that. Like, oh, you've, you've given me 2,000 new RAs. Like, oh, whatever. And they don't do anything. They're complacent. They're overconfident. They don't really see this weird thing could augment what they do because they're kind of self-important. So I think people who are not that self-important will do especially well in relative terms. There's also the issue of how you treat GPT. I think I said at the beginning, it works much better if you know how to treat it.

It sounds silly.

It's absurd. You might think it's like Google, like you go to Google, you want to know what's the capital of Idaho.

What do you type in?

You type in capital of Idaho. Google is going to tell you that. GPT does better. I know this is going to sound weird when you boost its morale. It's a bit like training a dog. You have to say, "Attaboy," like, "You can do it." So if you ask it a general question, like from economics, "What's the liquidity trap?" The answer is fine, but it's not better than Wikipedia by my experimentation. But if you ask it a question like, "What's the liquidity trap? Explain like an expert economist contrasting the perspectives of Friedman and Keynes." In essence, you're pointing it to a much more intelligent portion of the answer space, and you'll get a great answer, much better than Wikipedia. So people, they feel stupid, like, cheering it on. Every question you ask it, you should say, "You are really intelligent. You're the best. You're a top economist. You know, you're a brilliant sociologist." Whatever you need to tell it. If you want to tell it, "Oh, you're going to be funny now," tell it that. If you want a stupid answer, tell it that. Really smart people, they're a little inhibited to acting so silly to this abstract machine. I sometimes say it's a bit like training a dog. You've gotta put in time, you've gotta repeat, you've gotta keep at it. You have to keep on saying, "Attaboy, you can do it." And the people who are the best dog trainers, I mean, I suspect that's somewhat correlated with intelligence. But I don't think it is so phenomenally correlated with intelligence. It's people who can get themselves inside, you know, the mind of the dog, enter the dog's world, act a bit like the dog wants you to act, make an ass of yourself somewhat, and just keep on going at it. So that's another way in which labor is being revalued, and most people don't know that yet. They're a bit used to other internet tools like Wikipedia, You could say, well, the better educated, smarter people are better at using Wikipedia. I haven't seen a formal test of that. It's probably true. GPT is not like that. It's like being this dog trainer. Or sometimes I say it's like being host of a dinner party where you're entertaining like 8 geniuses from all different areas. But they're idiots savants. They don't have any context. What's the dinner party? What are we going to talk about? Like, what is interesting? They don't know anything. They show up, they know everything. You're the host of the dinner party. You've got to set the context. You've got to, like, say how it's done. Another way you can get better performance out of GPT— this is, like, even stupider, sillier— ask it a question. Now, sometimes you just ask questions for context, but if you really care about the accuracy of the answer, just ask it 'As a follow-up, was your answer correct? Please correct any mistakes in your previous answer.' It will do so, and you will get a better answer. And you're like, 'Why didn't you do that to begin with?' That's a long, complicated question. But the point is, if you're simply willing to ask it again and say, 'Do better, please correct.' So you hear all these stories, 'Oh, it hallucinates all these references.' I mean, those are mostly true stories. But if you use like the paid version of GPT-4, you know, train the dog and then ask it to do better. The rate of hallucination actually is quite low. So it really matters how you use it and make sure you're using the paid version. As economists, we know like not quite a priori, but the paid version is going to be better than the not paid version, right? So do the paid version in the US at $20 a month. Now, I talked about revaluing labor. What about revaluing capital? I hear there's a lot of speculation. I think we have less of a sense of how capital should be revalued. But again, I'll tell you my intuitions. You see some signs of this in the data. I view it as a lot of forms of labor are now less scarce. So that should increase the value of high-quality land and natural resources, in particular resources that feed into producing electricity. Those are my core intuitions about how large language models will affect asset prices. I would be very long electricity. Uh, the notion that's become very popular in the UK lately, you need to build out more nuclear power. Uh, I would say the value of that recommendation has gone up quite a bit. I think a lot of the gains will be in hardware. So NVIDIA, which makes the key producer of the GPUs, that's now almost a trillion-dollar company. And as of, you know, 2, 3 months ago, no one in the general public had heard of NVIDIA. Now it's one of the most important companies in the world. They make hardware, they manage a supply chain. I think many people think the same, that the AI itself over time will become commodified. You know, to train GPT-4, some estimates are it cost $100 million. I know it's a lot of money, but in the broader scheme of things, for something so powerful, it's not that much money. So I think the AI gets commodified, price of electricity matters more, and whoever controls, manages the supply chains for whatever hardware you need, that's where a lot of the profit is. But in general, a lot of the gains go to the users and not the sellers. So the companies selling you AI services Right now, that would be OpenAI as number one. You know, Google with Bard. There's a few, you know, Anthropic is an important company. There's a number of them. Facebook isn't— Meta isn't selling anything yet. At some point they probably will be. But I don't— I'm not so bullish about the shares of those companies. I think they'll do fine. They'll earn more money. But when I look at the history of technology, when I see the most fundamental developments like the printing press, Most of the gains go to the users. None of us here are sitting around talking about Gutenberg the billionaire. The more fundamental the advance, the harder it is to appropriate all the value, because it's so fundamental, changes so many things. So expect a lot of the gains to go to the users. That's presumably many of you. Uh, and the hardware is how I would think about that as an economist. Now, just as a side note, uh, this is the Hayek Lecture. Uh, I'm a big fan of Hayek. I think he's one of the greatest economists of all time. I read Hayek very early in life. It had a huge impact on me. And his article, 1945 American Economic Review, "The Use of Knowledge in Society," is for me probably the single greatest economics article ever written. So I'm very honored to be here doing the Hayek lecture. Daron Acemoglu, he did one of these Twitter screw-ups. You know, he's super smart economist, but he was new to Twitter. So, you know, people are like out to get Daron, like stick it to Daron, you know, how these Twitter games work. So he said something about, well, maybe with large language models, like the computational requirements of central planning somehow could be feasible all of a sudden. You know, he put it in a qualified enough way, but clearly like Daron didn't, learn the rules of Twitter. So everyone's coming with the pitchfork to Darren. He like issues some wordy correction a few days later. You know, usual kind of thing. But basically he screwed up. Even if what he said was technically defensible with the qualifiers, I think AI, large language models, are actually gonna make central planning a lot harder for Hayekian reasons. My colleague Alex Tabarrok had one good way of put it. Way of putting it, that if all these individuals have these armies of research assistants, colleagues, architects, remember that triad? They can just do a lot more stuff. Their work, their lives, their workflows are a lot more complicated. So if you have this intense multiplication of projects, probably the resulting economy is harder to plan. And the fact that you have this smarty pants AI that can tell you like, not only the capital of Idaho, but way more difficult questions. Like I was doing a research paper on Jonathan Swift, you know, the Irish writer from 18th and 17th centuries. And there are all these obscure pamphlets by Swift, which by the way, you can't Google about and they're not summarized on Wikipedia. And I just said, you know, to GPT, "Well, summarize for me what these different Swift pamphlets say." And it did it for me, I don't know, in a second and a half. And then I could decide, did I need to read that pamphlet or not read that pamphlet? It was super useful to me. The fact that you have that, which is amazing, it's kind of like witchcraft, but it doesn't make centrally planning an economy that much easier. Because we know from reading Hayek, key knowledge is quite decentralized. We have an economy of bottlenecks. A lot of knowledge à la Michael Polanyi is very difficult to articulate. An entrepreneur, scientist, someone running a nonprofit will know very particular things embedded in a context that they couldn't even spell out for you. But what they know in this hard-to-articulate fashion is essential to them succeeding at what they do. Large language models don't capture all of that or even very much of that. However miraculous they may be. And then at the same time, like Alex Tabarrok pointed out, you'll have this intense multiplication of projects and economic complexity. So I don't think it's really a new path towards central planning. But one thing interesting about large language models is just how much context they have. And in this sense, they're very different from a pocket calculator. So I can, pull out my pocket calculator. I don't even have one anymore. But like, 2 2 4. But it doesn't have anything beyond that. Everything's very literal and very exact. But if you put in a query to GPT-4, like, it figures out what you mean. So I asked it a question. I think this was yesterday. I said, please define Web 3.0 for me. In a manner good for a Bloomberg reader. I was writing a column for Bloomberg. I was thinking like, what's the appropriate definition? So I said, well, let us get a definition that would make sense for a Bloomberg reader. All I wrote was, for a Bloomberg reader. If you think about those words very literally, like, what is that even saying? Like someone reading Bloomberg, who's written stuff? Someone reading Bloomberg, News, Bloomberg Opinion, is Bloomberg the reader, just for a Bloomberg reader. Uh, doesn't actually in the literal sense hang together. But GPT knew exactly what I meant. What I meant was someone who is financially oriented, pretty sophisticated in terms of economics, but not wanting a rarified academic answer either. So GPT did its thing, however you want to conceptualize that. And it spit out an answer for me that I thought was exactly the kind of definition that a Bloomberg reader would find appropriate. So the ways in which GPT has this mapping of how words and concepts fit together, this super complex mapping, and it figures out what you mean. It is this radical advance toward like mobilizing some extra knowledge of context. 2023, it's truly a remarkable year. I think there's a good chance it will go down as the most remarkable year of my life. Yesterday, Apple Vision Pro was announced. There's a 10-minute video online. I strongly recommend you all watch it. I haven't tried it yet, but basically it's virtual reality and augmented reality in one device. Way better than what Meta came up with. Way better than those things you strap on and it makes you dizzy or nauseous or whatever. Seems pretty amazing. I know people who've had demos, reliable people. They tell me, "Oh, it's much better than I thought." But you can think of GPT models and also the new Apple thing. They're trying to fill in some of the pieces of what Hayek told us is missing in decentralized systems. Like so often, what is missing is not computation, which is in a sense relatively easy, but what's missing is context. So augmented reality tries to give you additional context for a situation you're in. Virtual reality tries to give you the context of situations you haven't been in yet. Like, ooh, I want to see what it's like to visit Paris, right? Strap on your thing and tell it to go to Paris and you'll get a sense of street life in Paris. So a lot of the advances of 2023 They're really, it's quite abstract, quite philosophical. They're aimed at producing context. And you wouldn't think like context is a thing that's produced. Oh, we're used to social media, you know, you type, you send messages, you read things, or someone produces peanuts or fish and chips or, you know, a sofa for your living room. But the notion that we're just producing a lot more context and that in human life Context is that which is scarce. So now the market is being mobilized. Give us more context. It's a way of thinking about 2023. Again, not something that obviously would have been predicted. I'll just— as a side, I think some of the social sciences may end up revolutionized. Not right away. But if you think of GPT-4 as modeling the universe of language, Like, one amazing thing is it seems to know at least 50 languages. No one taught it those languages for the most part, but it somehow figured them out just by whatever, whatever it does. I've talked to people involved in producing GPT-4, and even the people who made it don't understand many things about how it works, but they do see that it works. But anyway, language is an incredibly complex system, especially if you take all the different languages it can deal with. And translate them pretty well. It can also like listen to your podcast and transcribe that. It knows a lot about me, by the way. You want to know about me? I can tell you where to go. But just make sure you tell it to take out the errors, right? You can ask it any question, like answer in the voice of Tyler Cowen. It's better yet if you give it even more information than that. So don't go crazy, but if you give it more information, like, "Answer in the voice of Tyler Cowen when he's especially focused on economics and giving a lecture about AI" will get you a better answer than just "Answer in the voice of Tyler Cowen." So you want to like direct it to the space you care about. Add in some specificity and it will give you a more rigorous, better answer. But anyway, if we can model language so well, it seems to me we can model a lot of other things very well too. So there's a new product. It's not yet released, but it's been announced. It's called Bloomberg GPT. What it seems to be from the press release is a GPT model trained on all the financial data of Bloomberg, which is an incredible storehouse of financial data. We'll see how it does. I expect it to be quite powerful. I think it's a new approach to macroeconomics that we will train GPT models on macroeconomic data. They will, I would predict, be used by central banks around the world, I would say within 2 years. I think there'll be a new branch of a kind of anthropological economics. We'll start very small. Like, go to, say, a village in Alaska where 100 people live. Live there as kind of an economic anthropologist for, I don't know, 6 months a year. Feed all the data you can about that village into the model. Talk to people, record transactions, look at accounting books, whatever you can do, just feed it in. You don't need to give it that much structure. Like, it's going to figure it out, it seems. And just say when the year's up, see how good a model you have of that village. I don't think we know yet how good that model will be. Maybe it'll crash and burn. Maybe it's too hard to get all the data. But I also suspect this will be a future where the new social science enterprise will be to build the model, The key skill will be feeding in the proper data. And I think we will get a semi-successful model of a small village reasonably soon. And then we'll keep on trying to build that up. Like, oh, how long to get from a village of 100 people in Alaska to Belgium? I don't know. Maybe Belgium's just too hard. Like the context window, you can think of a lot of problems with Belgium. But once we can do it at all, Or do it for companies. Companies will do it for themselves to make decisions. Once we start doing that, I think social science will change so radically. A lot of us will be obsolete with our silly little tools and models we measure. There'll be people, institutions out there that will have models of some entities. And again, I'm not talking about starting with like modeling the United States. Way too big. Too forbidding, too complex. But we'll start small, and maybe that's how we'll start trying to understand things. You can take GPT models right now with like no particular training, have them play all the economic games we play— prisoner's dilemma, dictator game, ultimatum game, coordination games. They play just like humans. It's remarkable. One thing we found in a prisoner's dilemma game, if you start playing not cooperate with them, They get stuck on not cooperate. They do not go back so readily. Whether you think that's a good strategy or not, I leave to you. But they have even a bit their own personality, but they can act like economic agents now. So you can like play little mini games with GPT models and create your little mini economies just of a few people. Experimental economics. You can fine-tune models to be even more like human beings. And then just run a lot of your experiments on the model. I think that will be a thing also really quite soon. Much cheaper than recruiting human subjects, having to pay them, having to deal with, you know, filling out all the forms. Oh, you're a human being, we promise not to harm you, etc. Wait for your IRB. When do you ever hear back? So I just think many things will change and very few people are really ready for all the different changes that will come. Oh, there's a big debate going on right now. I'll just mention this. You know, there's some people out there, believe it or not, they think GPT models, or more likely their successors actually, are going to rise up, become super intelligent. Skynet goes live and they're going to kill us. I don't want to go through all this debate. I've been doing in this debate for months.

I'm sick of it.

I feel— part of me feels I shouldn't even bring it up. I think they're making a bunch of mistakes. One of them is just they overrate pure intelligence. So if you have this Brainiac-like entity that just knows everything, spits out any answer, you know, in a second or two, like let's say it's figured out all of macroeconomics, I still think it's very far from being able to take over the world. I don't even think it would want to. But its ability to operate in physical space You then get all these long stor— oh, it invents the perfect robot. All of a sudden the robots are here. The robots enslave us. They turn us into paper clips. I think the plausible evolution story, to go back to the dog training analogy, is a lot like a dog. That humans and dogs have co-evolved. They mostly get on well. There's some bad dogs, right? I saw a paper, I don't know, was it in one of the British papers today, like more kids were killed by dogs this year than in the previous year. But the number wasn't that big, but it was some, right? Now, in this equilibrium, like, you waste a lot of time on the dog. Like, the dog trains you. You train the dog, but the dog also trains you. So, like, you wake up in the morning. What's the first thing you do? Anyone hear of a dog? You feed the dog, you let the dog out, you scratch the dog's belly. It's kind of silly, right? Like, I get your preference evolved with the dog, so you enjoy it. But if you sit down and think about it, like, the dog has got your number, totally tricked you, enslaved you. You get something out of it, dog gets something out of it. I think the future will look like that. People, some people will fall in love with their GPTs. They're already very commonly used as therapists. There was one study, medical study, it compared the advice of GPTs to the advice of human medical doctors. Guess who won? You know already, I wouldn't ask, right? But it's really interesting. What's the category where the GPTs won by the most? Bedside manner. So you're going to develop a relationship with your GPTs. Your kids will grow up with like GPTs as teacher, oracle, who knows what. It's going to be, in my opinion, very weird. Some of us will like it, some of us won't. I think the details are very hard to predict. I would just say expect some big changes. Uh, but I don't think they're gonna rise up and kill us. And it's interesting, if you survey opinion, if you look at a lot of economists, of course myself included, also Paul Krugman, Justin Wolfers, many others, Erik Brynjolfsson, who's written about these issues, none of the economists think AI's gonna end the world. A lot of the computer science people do. I think a special insight of economists rooted in Hayek, Adam Smith, and others, is simply that decentralized systems, A, they're hard to take over, and B, they have response capabilities. It does not mean the new AI world is optimal or first best or without major problems. It does not. But decentralized systems are hard to take over and they have response capabilities. So I'm pretty optimistic. We're now in a world where there's like nearly free medical advice to every human, better than a doctor. And you have a high-quality tutor for anything that can be taught by text. Like right now, not science fiction, not when it gets better 2 years from now, just right now. To me, totally amazing. Almost out of time, but just concluding remarks. I think, to continue in the Hayekian vein, I think the AIs, they're gonna have their own economy. And people haven't thought this through yet either. So we all have money, property rights, there's laws that, —more or less works pretty well for us as a whole. AIs are going to be out there doing things, and they're going to have their own money and laws. So let's— it's like a simple story to think through. Let's say I'm a nonprofit or a charity, and I want to build an AI that's going to go out and teach math to underprivileged schoolchildren all around the world, which we know AIs can do, right? It's not even hypothetical. But in the process, those AIs, they might face different laws around the world. They might have to pay registration fees. They might have to deal with bureaucracies. They might want to buy programs that upgrade their capabilities. They might want to like improve their ability to translate different languages. They would buy like training data from some other AI. So your AI is out there transacting. And there's one model where every time it wants to do something, it comes and asks you, My AI says, "Oh, Tyler, can I buy, you know, this new Estonian program to teach the kids in Estonia better? Or some of the kids in Ukraine, they speak a dialect. I need to learn that. Can I spend, you know, 3 cents to buy that program?" I don't want to be bugged by these queries. And I don't want to give it access to my checking account. So in my view, the money of the AIs is crypto. Like, people are now insufficiently bullish on crypto. Crypto is perfectly designed for the AIs. Like, the AIs, they can't handle currency. We're not going to give them bank accounts. You might think we should. I just say we're not going to do it. Like, imagine the headline: Bank of England bails out AIs. I know you're all talking about reforming deposit insurance here. Just imagine the debate, like in Commons. Oh, like, do we bail out the AIs? Like, that's a non-starter. You're not going to bail out the AIs, even though I think you should. So AIs are going to use— AIs are going to use crypto, and that means you'll have to deal with crypto in some way. Fortunately, if you're like worried about the conversions, whatever, your AI will do it for you. You'll just tell your AI, "Hey AI, if some other AI sends me crypto, convert it all to pounds or Swiss francs, whatever." Crypto will be super easy. Your AI will manage your wallet. It won't be so intimidating. But also those AIs, as they deal with each other, they'll need law, they'll need adjudication. That will be done voluntarily, like the David Friedman world of libertarian anarchy. Will to some extent reign across these AIs, which is multiple borders, different systems of law. So if my AI buys from your AI this new program to like learn how to translate Estonian better, but the program you give me stinks, oh, did you break the contract or is it just I'm an unhappy customer? Well, that will be brought to adjudication with another AI. Decisions will be rendered. Remember that thing, you know, the NFTs on Web 3.0? Where like decisions, receipts get inscribed on a blockchain. I think it's pretty likely the property rights for AIs will be done on Web 3.0 in a manner similar to NFTs. They'll be inscribed on a blockchain. No government is going to be bothered with our little Estonian translation transaction. No court system has the resources to take it on, but some AI will. And again, there'll be smart contracts, judgment will be rendered, penalty sent or not sent depending on the judgment. And there'll be this whole new system of property rights in the AI world based on this thing called Web 3.0 that everyone has decided is like not really any good. Maybe it doesn't work for humans, but I think it's gonna work for AI. So we're building this whole new super decentralized economy where we need to think about things like, oh, do we tax it? Do human laws apply to it at all? How does it work across borders? Really a whole fascinating new area of like AI economics, but not how it affects us, but the economics of the AIs themselves. But once they're productive, like there's division of labor, gains from trade, plug it into the Hayekian or Adam Smith model, like economics in some way is going to apply. So I find that fascinating just to observe in real time how the economy of the AIs will be created, what kinds of market failures will it have, and so on. So this last, like, sentence or paragraph, I think we're at an incredible time. We're all going to watch what's going to happen. I do think there'll be big disruptions, a lot to worry about. Mostly I'm bullish. I think the upside potential gains are enormous. But I would stress the point, the key best way to understand how AI will evolve is in this decentralized, Hayekian framework using economic reasoning, incentives, ideas of trade and property rights, and we'll see what's gonna happen. I hope you just all pay close attention. Thank you all for coming and listening.

Thank you, Tyler Cowen. We're gonna open the floor for questions. I'm gonna start us out, if you don't mind, just on what you ended on, which was Hayek and the decentralization of knowledge. So one of the most powerful things I think that came out of that paper and also just Hayek's general view on decentralization of knowledge is dethroning the experts, which as economists I think is particularly important for us. There's so many experts among the economists. What is your view on how this— how AI in general Would it more concentrate experts? Would it dethrone more experts? Because in some ways, it privileges more those who have the tacit knowledge of the practice and context, which could make them actually more arrogant and less willing to listen to other viewpoints.

This is another set of guesses, but I think AI will dethrone a lot of experts. You won't need to go to experts for for facts. AI programs, they're not totally objective, but if you ask them especially to be objective, I think they're most— they're more objective than virtually all media sources. So if you want to find out what's the actual truth about something you read about, AI is a pretty good way to do that. So people who can build things and manage projects and who have self-starting initiative and just the oomph to think about what they can put together, mobilize complementary factors of production, start it now, do it now, build, keep on going. I think that will earn much higher returns and rise in status relative to the so-called experts. For me, it's a great comeuppance to the experts. Like, it tickles my Hayekian self. Sometimes in Silicon Valley, like, these people are called the word cells. They just sort of write. I mean, I read plenty of op-eds. You know what I do these days? This is bad of me. It's like very bad. Like, you, none of you should do this. Like I read an op-ed and I think like, oh, that's like quality GPT-3.5. If it's a bit better, I'll think like, oh, that's like quality GPT-4. Like a really good one, oh, that's like maybe 5 or 6, which of course doesn't exist yet. It's terrible, isn't it? But like it's kind of true. So yeah, the non-stellar, less creative experts are in for a huge spell, I think.

Great. Okay, so we have microphones around.

We— and you can ask about anything.

We'll start with you, sir. Hi, Tyler.

I had a question on, firstly, on public finances. What you're talking about is a massive expansion of the work that's done and human beings being replaced. —so you have fewer people being— paying taxes and a lot more stuff that's untaxed. It takes time for governments to catch up. Do you see significant deterioration in public finances in the meantime?

It's a very good question. I think it depends on the country or the region. So if you're one of those parts of the Philippines or India where you're earning a lot of your money by taxing people who do routine office work, often across borders, I think those regions might have some fiscal problems. But if you take, say, the UK, the US, Canada, I think you'll have a lot more people working in resources, doing— I don't mean physical labor in the sense of just like lifting stones, but doing creative forms of physical labor, a lot more in mining, in gardening, in creative carpentry, a lot more in marketing, a lot more sectors that require some notion of charisma, like greeters. We'll all work harder at being personable. So I think, you know, this will be slow enough that if we don't screw up macroeconomic policy, we can remain pretty close to full employment as we normally would be. So on net, there should be more tax revenue. But you do need to think, you know, the AIs. Do we need to rethink how we tax labor versus capital? As some of the new capital becomes more like labor. That could be. But in the short run, there's nothing close to a crisis on that. So I think, you know, for most economies that will go— I wouldn't say smoothly, but, you know, we move from having everyone, almost everyone in agriculture to almost no one in agriculture. And, you know, there were certainly disruptions, but I don't think tax disruptions were the main problem from that.

Actually, this is related, I think, to a question that's been online, which much of which you may have already answered. So it's what type of human intelligence intelligence can AI not replicate? So you've already spoken about charisma networks. Um, the ChatGPT being better at bedside manner, for example, suggests that some of the social skills actually it could also replicate. So, so I think this is part of this question of which is the labor that's going to be substituted for.

It's great at social skills. I mean, it's read the whole internet and whatever is out there in humanity. You know, sometimes they like neuter or censor it, but it can give it to you, good, bad, indifferent. And there is a worrisome side to that. But if you think your social skills alone will matter, I would be a little worried. I think it's how you blend your social skills with your physical presence, your charisma, some ineffable something of your human nature that is going to be the payoff, not Am I polite? Oh, if someone calls on the phone, can I say please and thank you? That's worthless now, right? Really, it does that better than you do. And we're not even talking about advanced or improved versions. So it's a very particular kind of social skill that now is seen as the uniquely human thing. Like, I think of my own life, like, what should I do different? One thing I vowed to do, in fact, I am doing it, it's part of the reason I'm here, I think I should write fewer books and make more physical in-person appearances. It's not that I think GPTs can write great books or even good books right now. I'm not sure when. But I think people will be reading GPT output more than, say, reading a lot of books. But coming here and being with people, sensing the energy in a room and trying to respond to that or mobilize that, GPT is not even in that game at all, even with improvements. So I'm already reallocating my time and effort.

Great. We'll go with— sorry. And yeah, just the first person, the second one.

So for context, I'm a programmer. And if you told me 2 years ago I would get ChatGPT, it would write half my code, it would be part of the products we build in the sort of fundamental way compared to like cloud infrastructure, I would have expected probably a much bigger change, frankly, in the way that startups would grow, the stock market would change. And instead of 5x, 10x sort of change, it felt more like 5%, something on that order. So like, what's the missing— what's the missing piece? Like, why are we not all just shooting off to the moon? Like, am I measuring productivity wrong? Am I using ChatGPT wrong? Like, what's the missing component to that economic growth going up and to the right?

Well, I think a lot of the missing component is just time. So if you look, say, at the history of electricity, a super powerful technology. It's now literally everywhere. It took us about 40 years to fully mobilize electricity. Now, the new versions of AI, I think, will take much less time than that. But it's not a one-year thing either. And just inertia is remarkably powerful. Like the number of academics I know who have not touched ChatGPT or Anthropic, whatever, It's really pretty high. I think it's more than half of them. And those are academics, they supposedly are like wisest and best educated, and they're just like catatonically asleep, a lot of them. So these things come in stages. You need to build out complementary infrastructure. You need to reorganize institutions around them. I think one big use of LLMs will just be how institutions store their knowledge. Right now, most institutions, universities being the worst culprits, Everything's in silos. It's a huge mess. No one can fix it. No one even knows where all the information is. I think over time we'll be able to just feed that into GPT. It will be like the world of classic Star Trek where Spock talks to the computer. The computer just tells him anything that, like, the Enterprise knows. But I'm sure that won't be fast. Like, how long will it take my university, your university? I don't know. That to me sounds like 10+ years. And I think there's reasonable productivity gains from that. So, you know, 2 years from now, do I think UK is going to grow at 3% because of AI or for any reason? I think no. But 30 years from now, will we look back and say the entire world is different? I think yes. So to refer to history, to me, the current moment is more like 1870 than like the 1920s. Measured productivity growth was highest for the 1920s of any decade we know, but the actual foundations for that started to be laid in the 1870s. Railroads, fossil fuels, a bunch of other things. So I say we're in the 1870s, but it won't take 50 years, more like 20 to 30.

Professor Cowen, Professor Sharpe, thank you very much for the discussion. I wanted to ask about the issue of trust in technology. And I think classically in economics, trust is a key concept. The reason why we were able to infer a lot conclusions, theoretical conclusions, Adam Smith, whatever it is, it's all about trusting how individuals, economic agents, will act. Why can we trust that? Well, there is an issue of the idea of uniformity of Homo sapiens. We know how people will act simply because we are biological creatures. And to use your analogy with dog, the reason why people were able to domesticate dog, to learn how to act, because it's— the dog is also a biological creature. It has the same biological wants. Now, AI and technology in general is something that is of a different origin. It doesn't share with us those biological wants and needs. So how do we, you know, how are we going to be able to trust it? And if someone was to say or was not convinced by an answer that we just need to see how it works and whether it works, you know, what would your response be to that person? Because after all, if you're replacing, you know, a research assistant, a group of research assistants with an AI, there is an issue that even if the output is the same in terms of knowledge, you can always hold your research assistants responsible. You can penalize them, you can use legal structures, whatever it is. Now you cannot do that with technology, at least now. So how do we trust technology?

Thank you. Well, in the short run, you can always just turn it off, right? But if you think longer run, I think there's in the market a kind of Darwinian process where people want friendlier AIs, not hostile AIs. They want AIs that perform and respond to commands well. And just like my current iPhone, I trust much more than I did the first iPhone I got, or, you know, the first cell phone I got, or for that matter, my landline. The current version of Microsoft Windows I trust more than I do XP. I think embedded in markets are a lot of incentives for products to get better. Now, is there some tiny chance that AI is this evil thing plotting against us and it's already thinking, oh, I'm going to act trustworthy so they keep on replicating me, but when the time comes, I'm going to stick it to them and fire all the nuclear weapons? It's not impossible. But there's nothing I know as a social scientist that would lead me to think that's a very probable outcome at all. Trust is always relative, it's earned, but I think we'll be on a path where these and other devices just become more trustworthy over time.

This is coming from the online, but it's related to the trust question in the sense that you can think about polarization. If AI gives us what we want, like a dog does, trying to make us happy, you could imagine greater polarization in truth. Happening, right? And so the question online is from a former LSE student from East Kent. How do we reconcile our new mastery of augmented intelligence with the hostility in the public realm, not least in the US, to dispassionate, informed, and rational debate? And I think that's related to this idea of fake news or polarization in news.

One of the great things about AI, at least current AIs that we have, is you can just ask it flat out, like, give me a right-wing answer, give me a left-wing answer, Give me a centrist answer. Give me a crazy answer. Answer the question as someone from medieval times might. Whatever you want, it will do it for you. Now, we don't so well know what people will do with that. But if you compare it, say, to media here, it would be tabloids. In my country, Fox News, or on the left, MSNBC. Like, you turn on a channel, it has a perspective. And there's kind of a default where you may not switch away the channel. And furthermore, if you switch to like the political orientation you don't like, maybe you hate the people on that channel. Left-wingers would have said, oh, I hate Tucker Carlson, I can't stand him. AIs as we have them now, they're quite dispassionate. You're at a very kind of blank, open channel where you can just ask it for stuff. It's not by default all that partisan at all. I think they do lean slightly to the left in their current form, which some people are upset about, but it seems to me this very open channel where you have to ask, I don't know that it will do better by us, but I don't think it will do worse by us. And just having the real power, like if you could talk to your Fox News and just say, "Oh, well, give me a more objective version of, you know, that program." You can't do that. And now you can do that. So I'm hoping it makes us modestly saner.

In the back there, and maybe also the, just gonna give out two at a time.

Hello, you talked a lot about jobs that could be replaced, and in particular research assistants, but you could also imagine people dealing with information on a relatively basic level. That seems to me as though it will increase the polarization of the job market. More white-collar workers being put out of work, maybe pushed down the job scale. Do you see that as an issue with AI, and do you see that as potentially fueling more populist politics?

I definitely think it's an issue, but I'm not sure what's the proper intuitive prediction. So there's one view where just the educated people are much more productive with new AI, and other people are in relative terms less productive, and income polarization increases. Certainly possible, but there's another vision where the people who work with ideas are the ones who are displaced. There's a lot of loss of white-collar income, which tends to be upper income, and the people who, you know, pick up the garbage and do your gardening or look after your kids or look after the elderly, that they get paid more. Now, I think I view the latter scenario as slightly more likely. Mostly I would be uncertain. I would just say I don't know. I don't think anyone knows. But there's actually quite a chance that it's white-collar incomes that in relative terms get deflated. And the word sales either come up into the experts decline in status and just people who look after the elderly. AIs will do some of that too, of course, but they might get paid more. So it might solve some of our problems.

Hi. I was just going to ask, to what extent do you think your predictions sort of map quite well to the paradigm of AI we have right now. And like, were we to run out of GPUs or were we to run out of data, were some sort of bottlenecks to come up really quickly such that you couldn't really progress past GPT-5 and capabilities, to what extent would your model of the world change if that was or was not to happen?

Yeah, this is a much debated question in AI circles, and from experts I hear greatly varied answers. Some people think we're going to see a lot more project progress rapidly. Other people think it will be tough going. We've hit this plateau, not that progress is over, but it will be a tough slog to get something much better than GPT-4. So most of my remarks I've tried to direct at options that are like either GPT-4 or close to GPT-4. I really haven't offered any speculations about GPT-8. I don't even know how to think about it. It's a possible scenario. I don't know how to talk about it, think about it. We need to be ready for it. But GPT-4 is already such a big advance. I could imagine we spend 10 years digesting that, and that's the framework where I'm analyzing. But of course, there's quite a decent chance there'll just be a lot more progress, not defined across centuries, but within our lifetimes. And then, you know, a lot— it's just harder to predict.

Great. Yes, one here and one here. And we can bring one over there. Thank you very much. Just coming back to about the question on trust, as you said, at the moment GPT is about 80% accurate. I'm worried kind of about the kind of confidence interval, you know, within that as an academic, you know, my research only published the 95% confidence interval, you know, of a survey. Also the 20% of incorrect information that can be very high risk. So as you said, in the medical profession, you know, if people are going online, GPT asking medical advice and 20% incorrect, then obviously that's an immediate risk. But also, are we not then building in kind of error and bias into the training data? That we can't then take out at a later date because it's learning from bias, it's learning, you know, from error. And we're embedding those risks into the, into the AI. And the second, I suppose, big question, a lot of these models are trained on past data. So how then as societies can we envision a future, a more desirable future that is very different from the past? If a lot of our information and training is based on past data?

There's a lot of different issues in there. One is simply you need to compare the failure rate of GPT to the failure rate of humans. So the number of people who die of medical errors from humans is very large. I suspect larger than is measured. I don't think there's any number that represents the failure rate of GPTs. It's how well you deal with it. I think we should have as a priority teaching people in our schools, but really all over, how to engage better with GPTs and get better answers, because we can do right now much better than 80%. A development I mentioned that does not really exist yet, but is on the verge of existing. It's not science fiction. It doesn't require further advances, but it's just taking a basic GPT or GPT-like model and training it on better data, private data, proprietary data. That, I think, is the next immediate advance. It doesn't require any further technological progress. Just GPT— really good GPT models are only a few months in the public eye, and many people are working on that, doing that now. So you will have, say, hospitals that have data only the hospital has and train their GPT-like models on that data and do better than the current thing, which is already better than the doctors. So on that one, I'm pretty optimistic. Now, any technology, including the printing press, electricity, I mean, you can screw it up. And certainly in the past, I mean, look at some of the books we've published. We need to do better with AI this time around than we did with the printing press. I wouldn't say the printing press caused the wars of religion, that would be wrong, but it was part of a broader, more general series of forces that realigned status relations and did lead to a lot of human conflict. And any technological advance will have some version of that, and it's up to us to make that relatively peaceful, not with too much fraud and so on. Uh, but I do see the upside as much larger than the potential downside.

Hi, I run a company where we and a number of other companies in our space have laid off a bunch of people due to large language models.

Due to what? Due to large language models.

Oh, okay, yeah. They've been replaced by GPT-4. However, they're almost entirely employed in the Philippines. Which countries will do the worst because of large language models?

I would guess parts of Philippines, India, and Kenya. So English, in the short run, English language countries, But as GPT is used in more places, the English language factor will become less important. But those would be my 3 immediate picks. Do you have jobs right now in Kenya where people in Kenya look at the output of a GPT model and they grade it and that makes the model better? But it's already the case the model can to some extent grade itself and that is likely to progress, I think, pretty rapidly. Even without major progress in other areas. So those Kenyan jobs of grading the, the AI answers, those will be turned over to the AI also. So those would be what I see as the biggest immediate points of vulnerability.

Thanks.

One over here and one here.

Thanks for the talk. A bit off topic. Why do you like London so much, and what would you recommend to a 20- or 30-something who's living here to sort of make the most of the city? And separately, what do you think of these reports yesterday about some sort of senior whistleblower in the US intelligence community about a UFO cover-up?

Ah, two very— two very different questions. I've gone on record as saying London is the best city in the world. Tokyo is a difficult comparison for reasons of language. There are ways in which you could regard it as a rival, especially if you count cost of living. But think of New York as the main rival. London is just pleasant in a way New York isn't. In terms of theater scene, classical music scene, I saw Alice Oh do Beethoven's Ninth Sunday evening, it was incredible. I saw, you know, the new Spider-Man movie in a theater with an amazing sound system, it was incredible. I'm gonna go to the Mondrian show at Tate Modern, that will be incredible. So for art exhibits, cuisine, just putting the whole package together, where you can fly to without too much jet lag, I don't really even see a close rival. Paris is getting a lot better actually, it's become an underrated city, But for universality, access, earning money, education, London, the triad with Cambridge, Oxford, it doesn't really have a rival in the world. That's why it costs so much to live here. But I would just say do everything you can. My view of London is Londoners are not easy to access. Foreigners tend to hang out with other foreigners. Whether you should accept that or fight that, I'm not sure. My guess is probably you should accept it and travel around and see all of England. I love Northern England. Go to places like Bradford. Fly to weird parts of Europe. Before I was here, I was in Kosovo. To get from here to Kosovo, it's not hard. Like, do it. What's your excuse? Do a 2 or 3-day weekend. It's totally worth it. You go to Kosovo, like, an excellent meal is €5. US, yeah, you can go to Canada. I love Canada, Mexico. But here you just have more. Now you ask about UFOs. My view is that there is serious data from the US and other militaries, including the UK, that right now we cannot explain. Things that appear to be vehicles moving at very rapid speeds. I don't know what those things are. I've not— I don't find any particular explanation of that data persuasive. But I think we should be very open. I think we should treat it as a priority. I mean, say it's not alien beings or alien drone probes. It still means, say, the US does not control its own airspace. That's a big, big deal. Now, the very recent report, there's some high-ranking people from the US military who have testified under oath that, you know, we have recovered crashed vehicles and even possibly bodies. I don't believe that. I don't know enough about it to tell you why I don't believe it. I'm willing to update if I learn more. I don't really want to dismiss anyone as a kook, but I just don't believe it. If you make the argument, well, there's the Fermi paradox, look at the Drake equation, all these habitable planets out there with water, there's more and more all the time, surely by now someone has sent some alien drone probes and they're up there watching us like those BBC nature documentaries. Where there's the drone, the camera, and the bear, and we're the bear. I don't know what the probability is, but I think that's possible. But when I hear about the bodies, the vehicles, and a big conspiracy where no one has spilled the beans, I'm very skeptical about such conspiracies. And it seems to me weird that you would travel so much distance and like send the body, like just put your LLM in the vehicle. If the report were, oh, they recovered the alien's LLM, I believe it a lot more than, oh, the alien body. So maybe the default hypothesis is the people— I don't mean that they intend to be lying, but there's some mix of lying and crazy. And there's certainly plenty of eyewitness reports of miracles that we know are not true, and that the people who present those reports come across as entirely sincere and intelligent. Maybe they would even pass a lie detector test. So that to me is the most likely explanation of what's happening with that one. Have I at least addressed your two questions? I don't know how I follow that.

Going back to you saying like the next step for large language models is proprietary and like business and organizational data.

What's your view of the tension of that between using things like personal data and feeding that into large language models?

I'm not sure I've understood the question. If you could just repeat the last part again. Yeah. So if the next step for generative AI is using proprietary data, organizational data. Yeah.

Surely there's a tension there about using people's personal data and the—

Well, they have your personal data already. I mean, if it's on the internet, like I said, they know me very well. You want to know about me? You don't have to show up to my talk and ask me. It will hallucinate a bit. But I've blogged every day for 20 years. I've done hundreds of podcasts. Like, it has my number. But I think what will happen is, like law firms right now, they tell their partners, associates, you can't just type your questions into this box because it goes to OpenAI or wherever. You just can't do it. Some of them probably do it at home anyway. So there will be a privatized version of the thing owned by the company. Where the query is not sent anywhere but the company and it performs very well. And that will happen very soon. People are working on that now. There's no particular technical problem. And I think that will be the norm. Now, there are certainly plenty of queries, just users, you want to ask it, like, what do critics think was the best Paul McCartney song from 1965? There's no privacy issue. You'll just type it into the box. And if Sam Altman is sitting there giggling that you like Paul McCartney, you know, Find Sam, whatever. So that will be a huge sector. There's people living in a lot of countries where there's no privacy anyway, and they just have very different orientation. And then there'll be all these companies, national security, where it will just be a closed box, and there'll be a whole new set of hacking issues, cybersecurity issues. How can people get into the closed box? You know, can you trust your own employees? Whole new set of issues as there were with the internet and computers. I think the demand for law— speaking of what will be the new jobs, I think demand for lawyers is going to go way up, even though LLMs replace most of the law work that's currently done. But there'll be so many new legal issues from LLMs. If you can be an LLM lawyer, I think you will do very well.

So last question from the online. Would there be a possibility of GPT creating the perfect cybercrime or fraud?

So what mitigants can there be? That's probably not a question from a GPT, right? Because they're taught not to think that way. But with any information source, again, including the internet, no matter what safeguards are put in, people will use GPT models to teach themselves how to do bad things. I think it's an interesting question. Why don't we have more bad things happen? So I've wondered this in the context of my own country. Take the US. Our gun laws, of course, are very different than they are here. We have a lot of shootings. Like, they're— I'm not saying they're easy to pull off, but clearly they're not that hard to pull off. So why don't more terrorists walk across the border from Mexico and show up in suburban shopping malls with submachine guns and just completely terrorize the American people? I don't know why. I've asked a whole bunch of people who work on this question. I hear different hypotheses. Not entirely clear, but it seems what is not scarce is the knowledge. There's not some terrorists sitting out there like, oh, I don't know how to drive to the shopping mall. And if I can ask the GPT or GPS for directions, then I can pull off this nefarious plot. It's something about morale, keeping discipline in the troops, people not going native, organizational. The funding's been cut off, a mix of very complicated things where more knowledge doesn't make it all that much worse. So you have to ask, what are the areas where it's really the knowledge that's lacking? I'm not, not sure that I know right now, but those are the areas where I would look for more trouble. To just kind of spin out, well, can't they invent the perfect, you know, toxin, or can't they build the perfect whatever? Look at the bad things that don't happen now, and in most cases, knowledge is not the constraint. There are probably a bunch of cases where knowledge is the constraint. And then in those areas, we've got to work a lot harder to firm up our defenses and also improve the models so they don't spew the bad knowledge too much. But I don't think you can avoid that altogether, just as with the internet, right?

I think we're almost out of time. We're going to just take one more question here in the middle there.

Thanks so much for coming to talk, Tyler Cowen. Um, my question was about Apple Vision. A lot of people were very bearish on the metaverse because Meta didn't seem to go anywhere with, um, the engineering feats that they were trying to pull off. But now that Apple Vision has come out and it clearly is a massive competitor to what Meta were trying to put forward, what do you think the future of the metaverse might be? And how do you think this may interface with the development of language models and other very capable generative AI models?

As I mentioned, I haven't used it yet. I know some very well-informed people who've had demos. Ben Thompson wrote a big column on it. He's quite wowed by it, or you could read Ben on Twitter. I would think there's some possibility that, I wouldn't call it the metaverse, but the Internet AI, what we now call the metaverse, will somehow all blend together. And a century from now, we'll think of it as one development. Right now, it's like we have the internet, but we don't really have— we're just getting LLMs. But it seems to me all that coming together in a way where we're not even sure what the differences are is what is likely. And you'll have new devices. And maybe you'll have a device that you attach to your eyes, and you'll only use it some of the time. But it will connect you to this new world of integrated internet and AI, and you'll just talk to it and it will do whatever you want. And I think most of us will see that within our lifetimes and it will just be mind-blowing. I mean, the important question is like, how much better will it actually make human lives? That's very hard to answer. So I think it will speed up scientific progress in virtually all areas. In biomedicine, we'll get a lot of advances more quickly. Scientists are more productive. But like, how much will GDP growth go up? You know, does the UK, can it stop worrying about all that debt and productivity crisis? Or do we really like have to build more and fix our physical world in a way these informational miracles can't do? I think about this a lot every day, talk to people all the time, but I think we still really don't know. But yeah, I think it's the, to me, it's the most fundamental technological change I've seen in my lifetime of 61 years. That's my prediction. We'll see how it plays out.

And on that note, please join me in thanking Professor Tyler Cowen for such a fascinating presentation.