AI And The End Of Progress? Carl Benedikt Frey On The Fragile Future Of Innovation

Welcome to the Future of Business and Technology Podcast. I'm Bernhard Marr, and today I'm joined by Carl Benedikt Frey, Oxford economist and author of the new book, How Progress Ends. Carl's work explores one of the most fascinating questions of our time. Why do societies that make huge leaps forward sometimes stagnate or even collapse? His research connects history, technology, and economics to uncover the fragile nature of progress. In our conversation, we'll dive into what history teaches us about innovation, how AI could either propel us forward or derail us, and what leaders today can do to safeguard progress for the future. Welcome, Carl.

It's a pleasure to be with you, Bernhard. Thanks for having me.

Yeah, so nice to have you, have you back. I think last time we talked about your last book. People might remember you for an influential paper you wrote, now over 10 years ago, predicting what AI might do to the future of work. I think you were predicting that it could, in theory, eradicate 47% of jobs. Before we go into any of the new book and the content, I would love you to give us an overview of your background and how you ended up at Oxford and how you ended up doing the kind of research you're doing today?

Yeah, so I'm an economist, economic historian by background. I got interested in technology back in high school. And, you know, once you start thinking about the question, you know, why some countries are rich and why others or not, it's hard to think about anything else. And technology is, I came to realize back then, at the heart of that story. But it's also a tremendously disruptive process. It creates winners and losers. And what I argued in my previous book, The Technology Trap, is that unless we recognize that technology can also create hardship in the short run, we are actually less likely to see acceptance for technological change. which is crucial for technology to progress. Because before the first industrial revolution, there was a lot of inventive people coming up with brilliant ideas, but there were very few societies that were able to put them into widespread use because of resistance to progress. And that's also a theme I'm revisiting to some degree in my new book, How Progress Ends. But sort of the unifying theme of my research has been the political economy of technological change.

Yeah, and it's a fascinating topic. Just reflecting on my holiday this year, I went to Turkey and I visited Ephesus. And you go back and you visit these amazing cities that thrived over 2,000 years ago when the rest of us still lived in tents and hunted with spears. And I'm always amazed by how these societies emerged. and then almost disappear. So hopefully we can discuss this a little bit. You called your book How Progress Ends. What do you mean by progress ending, and why did you decide to write this now?

So to be clear about progress, I mean technological progress. So the book is about innovation and technology, and it sort of pushes back against the narrative which exists within the AI community, I think in particular, that progress is inevitable. somehow, right? And it's not hard to see why you might think so, right? After all, we've been through 2 centuries of very rapid technological progress and extraordinary transformation of how we live and work. But if progress was inevitable, well, the First Industrial Revolution would have happened a lot earlier in human history. And if progress was inevitable, most countries around the world would be rich and prosperous today because they would have adopted modern technologies to roughly the same degree as high-income countries have. And that's not what we're seeing. So the point I'm trying to make is not to predict the end of progress, but to sort of push it back against this narrative. By making sort of some subtle point that technological change entails 2 very distinct phases. So I'm an academic. When I write a new book, I have an exploration phase where I go out in the world, I talk to people, I read, I try to get inspired and try to figure out what is it that I want to work on next. And at some point when I figure that out, there's always a lot of tweaks and twists here and there, but I enter execution mode and I want to sit quietly in my office and write down what I've intended to do. And I think most processes work like that. If you're an entrepreneur, that process is roughly speaking the same, but you're trying to develop a new service, you're trying to develop a new prototype. But at some point when you've done that exploration, you need to turn that into something that's sort of commercially viable. You need to scale up, you need to build corporate hierarchies and bureaucracies in order to manage that scaling process, and you try to cut costs as much as you can to capture more of the market. And that is a very different phase from the exploration phase. And some institutions are more geared towards exploration than they are— than they nurture exploitation, and vice versa. And the point I make in the book is that you might be very good at one or the other, but if you constantly want to be at the frontier, you need to move between the two. And that transition is never easy because when the returns to something peter out, well, the interests that are tied up in that technology and that activity, they rarely go down without a fight. And they are not very keen on the technology, uh, or, or a competitor, uh, overtaking them. And so, you know, uh, exploration thrives on the decentralization. Exportation thrives under concentration and consolidation, but then to get onto the next cycle, you need decentralization again, and that transition is very hard to make.

Interesting. Do you have some practical examples of organizations that thrive in the exploration phase and in the execution phase and maybe not in the other?

Yeah, so, you know, some First, to stay with the macro picture a little bit, in economics, broadly speaking, there are sort of 3 strands of research around why some places are poor and others are prosperous, and they fall into 3 categories: geography, culture, and institutions. Now, if you take the Soviet Union, for example, it grew very rapidly in the 1950s, it collapsed in the 1990s. The geography of the Soviet Union was essentially the same in the '50s as it was in the '90s, right? And so geography can't explain both the upsurge in growth as well as the decline. Culture, you might argue, changed a little bit because, you know, with radio technology, for example, you know, the transmission of Western ideas could go right through the Iron Curtain, and the Soviet Union got more exposed to those kind of ideas. But if anything, that should have led to an upsurge innovation, not a collapse. And needless to say, Gorbachev tried to modernize the Soviet Union, tried to make it more decentralized. That didn't really work out either. And so, private property, which is often regarded as a cornerstone of technological development, I think it's true to some degree, but it's also true that both China and the Soviet Union managed to grow very rapidly without secure private property rights for several decades. And so the argument that I'm trying to make is that if you want to understand both the rapid growth phase and the collapse, you need to understand how technology interacts with different institutions at different stages of the technology lifecycle. And so the Soviet Union was very good at scaling, right? They were very good at mobilizing resources, but when something new was needed and the benefits to mass production petered out, uh, and the computer revolution, uh, began to take off, well, a more decentralized system that could take different bets, uh, was more suitable in this period of technological turbulence, right? So for example, if you were an aircraft engineer in the Soviet Union and you wanted funding for your project, you could go to the Red Army and ask for funding. If you were turned down, you could have maybe 2 or 3 other places you could go. If you were turned down there as well, well, your idea would die with you. And needless to say, unless your technology spoke to national priorities, it didn't have any chance to get funding from the outset, right? So decentralized exploration, there was very little of it in the Soviet Union. On the other hand, Bessemer Ventures famously declined to invest in Google back in '99. And it's easy to ridicule that with the benefit of hindsight, but obviously at the time AltaVista, Explorer, they were doing pretty well. And it wasn't by any means clear that Google would thrive. And it wasn't by any means a safe bet, but others stepped in, funded Google, and as a result of that, uh, one of the most successful companies in history emerged. And so a decentralized system where you can take different bets is really important, particularly in the early stages of the technology lifecycle.

Yeah, and history is full of examples where societies made breakthroughs and then faltered, and I guess Russia is one of those examples in the Soviet Union. What would you draw out as some of the most important lessons then for today's world?

So a key theme of the book is that institutional flexibility is essential, and that's the key reason why the United States has managed to stay as technology leader for a relatively long time. So back in the late 19th century, it had the most liquid market for technology in the world. Innovation was primarily, almost entirely, driven by solo inventors, and most of them licensed their technologies to emerging corporations. Very few companies had patent and R&D departments, and if they did, they were mainly focused on screening the patent landscape for promising external technologies. And so they focused essentially on buying up ideas from the outside and then converting those into marketable progress, right? So the railroad companies and the telegraphic companies, AT&T, which were built around the telephone, well, you know, these firms did not invent any of these technologies. They emerged later on. And up right up until the 1970s, where, you know, some interruptions of depression and war, American institutions became increasingly geared towards supporting the scaling of these firms. And that happened through highway construction, which was critical for the integration of markets. It happened through institutional innovations like the GI Bill, which provided educational opportunities for veterans coming back from World War II and was essential to building white-collar bureaucracies with white-collar professionals. It consisted of developing new business schools, new schools of engineering, and many other things. But what then in the end made American progress sustainable was the capacity for industrial renewal and its ability to enforce competition when it actually matters. Some may remember the 1980s when people, quite similar to today, worried that Japan was about to overtake the United States and would be the technology leader of the world. Why didn't that happen? Well, in Japan's case, institutions were geared towards technological catch-up. First to Britain in the 19th century, then to America in the post-war period. And what Japanese firms did very well was taking American technology, improving production processes, scaling what American firms did, and doing so more productively than any other place. But the Japanese conglomerates entirely missed the switch to software. They were focusing on improving hardware. And in the United States, 2 pivotal antitrust cases, one against IBM, which opened, which had begun to bundle hardware and software. That lawsuit opened up the market, forced IBM to unbundle software and hardware, and permitted firms like Microsoft to enter the scene. And then you also have the breakup of AT&T, which meant that as the National Science Foundation released ARPANET, which would, you know, was the precursor to the modern internet, to the world, it wasn't bottlenecked in the boardroom of AT&T, but the technology could develop organically by its users. And this sort of switch back to decentralization and competition in the computer era, I think, was the decisive factor, uh, for why America came to lead the way, uh, throughout the computer revolution, um, and not Japan, as many people predicted.

Yeah, interesting. So centralization is good for catching up, decentralization is really good for innovation. How does this tension play out in today's AI race?

That is an excellent question. And so right now, I think it's fair to say that there are several foundation model providers which are in intense competition with each other. I think it's also fair to say that companies like Meta and Google look less safe than they did 5 years ago. And so generative AI has to some degree undermined the business model that Google had. And the researchers that developed the transformer, they actually worked at Google and then ended up leaving the company. And this sort of dynamic that we see numerous times, that firms that have something to lose from creative destruction sometimes unwilling to be at the forefront or at least bring that new technology to market. So we saw that with Kodak, who was at the forefront of digital technology, but then for fear of cannibalizing their old business model, didn't act upon it. And Google was similarly slow when it came to releasing generative AI. Now, I think the question is whether the competition we currently have among foundation model providers is just a blip, or whether we will go back to this sort of more consolidated market structure with less competition that we had before OpenAI released ChatGPT. To the world. The field of AI used to be an anarchy of methods. Right now it's become highly focused on deep learning and large language models in particular, and probably for good reasons, right? It's the technology that has shown that it has productive potential. At the same time, I think it's pretty clear that we need more innovation. So A year or so ago, a lot of people still thought that you could just take existing models and scale them. More compute, more data will get you there. And that might work in a world that is static. If the world was just a static distribution of events, you could essentially brute force it. But it's not. It's changing all the time. And because it's changing all the time, it's hard to know whether the algorithms that work well on yesterday's problem or today's problems will work well on tomorrow's problems. So to give you an example, most people will know of AlphaGo beating the world champion Lee Sedol back in 2016. Few people know that human amateurs with standard laptops recently beat the best Go programs quite easily by exposing them to concepts and positions they would not have encountered in training. And so that sort of raises the question around, you know, even when you achieve superhuman performance, how well do these algorithms actually perform out in the real world when circumstances change? And I think, you know, these algorithms have a real resilience problem, and I think we will need more innovation to solve that. And so, as I mentioned earlier, exploration and decentralization is good for innovation. So we want to have, you know, different firms, different people out there testing different ideas, and we want to encourage that kind of competition. And what we're seeing right now is that though even there is competition, there's There's also a growing concentration whereby OpenAI, together with Microsoft, that partnership has something like 70% of the market. We also see that many of the tech incumbents are actually investors into the new AI startups. So I think that's something to keep an eye on. I think perhaps even more importantly, we see growing concentration in the industries that might use AI. Why does that matter? Well, if you're a large firm, you tend to have scale and you focus on process improvements, automation, driving down the cost of what you already do, right? But that eventually runs into diminishing returns, right? If you want to do something genuinely new, which is what creates most growth, right? So after all, if you If all we'd done since 1800 was automation, we would have achieved textiles and productive agriculture, but that's about it, right? And so most growth, most prosperity come from doing new and previously inconceivable things. And what we do want is that firms use AI to do genuinely new things, not just process improvements. And if we have great market concentration, if we have fewer new firms entering this market, I think that's going to be a real challenge. for not just development of AI itself, but what purposes we deploy it for.

Yeah, let me bring the geopolitical dimension into this as well, because there's a huge race between the US and China. If you look at your model, how do you assess that competition then? Who's best positioned to lead the world in AI, the more centralized China or the more decentralized US?

So the United States is changing as we speak, right? And so I did capture some of these dynamics in the book, but, you know, it was completed, uh, you know, just as the second Trump administration, uh, entered, uh, the White House. And, you know, I did not fully foresee to which degree they would actively seek to undercut America's technological strengths. So, you know, we mentioned ARPANET earlier as the precursor of the modern internet, which was funded by the government back in the 1960s and was built on the back of a decentralized system of universities that were increasingly well-funded through the National Science Foundation and other institutions. So that system is now being entirely undercut. Funding through the NSF cut, I think, with something like 50% the autonomy of American universities. being reduced, clampdown on access to talent, and clampdown on immigration that is— well, I shouldn't say unprecedented, but unprecedented in modern times. And I discussed this in the book, Germany, the case of Germany should serve as a reminder that it's possible to destroy your universities. Germany has decent universities today, to be sure, but they were world-leading before 1930. And I think the destruction of American universities is clearly not yet on that scale, but clearly things are moving in the wrong direction. Not to mention the tariffs that are making American firms less exposed to foreign competition, all sorts of exemptions that are breeding crony capitalism. And so I'm much more worried about America than I was when I wrote the book. And the book is titled How Progress Ends. So I was worried back then too. The only relief maybe for the United States might be that China is not doing particularly well either. And so productivity growth in China since the Great Recession has been disappointing and even negative by some estimates. The real estate sector is around a third of GDP. The political priority is no longer just growth and prosperity, but increasingly national security. And there's a drive to instrumentalise firms more for national objectives, which private sector firms are often not keen to do. And so that leaves state-owned enterprises, which have always been a drag on productivity and innovation, with a growing role in the Chinese economy. But that said, there are bright spots. DeepSeek is one of them. And companies like that can be permitted in the Chinese system to fly under the radar as long as they serve national objectives. The sectors where China is doing relatively well, like EVs, for example, are fiercely competitive. And so the Chinese economy is highly uneven. In some sectors, state-owned enterprises dominate, and they're gaining a more prominent role again, and that's dragging down productivity growth. In other sectors, the fact that China has this major economy with local governments competing against each other for positions within the Chinese Communist Party that creates a lot of competition. And that has been driving some of the innovation in China. But China is also increasingly centralising power at the top as well. And in the absence of a rule of law, you really do have to build political connections in the Chinese system to stay on top when political priorities change, which they frequently do. And so the tragedy of this is not that China has become more than the United States, which many expected as China entered the World Trade Organization back in 2001, But that the United States is actually increasingly looking like China.

Interesting. So many of my listeners lead organizations. What can business leaders learn from past societies about managing innovation and avoiding stagnation?

So if you go back to the computer revolution of the 1990s, you will see that There's an upsurge in productivity, but it's not widely shared. It's mainly driven by US firms, it's mainly driven in Europe by Northern European firms. What these firms have in common is that they are more decentralized, and by decentralization I mean people at the lower levels of the organization having higher decision-making autonomy, and also being rewarded for experimentation. Why is that important? Well, if your listeners know anything about AI or LLMs, I suspect most of what they do know is from using it and experimenting with them themselves. You can read books, and you can listen to podcasts, all of that, you know, I'm all in favor of that, but in the end, if you want to push the frontiers yourself, you have to experiment too. And, you know, the people that do the experimentation, they understand best themselves what the technology can and cannot be used for, right? And so giving them the decision-making autonomy to do that experimentation and to implement what works is absolutely critical if you want to stay at the forefront of a technological wave. And you have to reward that experimentation too, right? So, you know, to give you an example from— I'm an academic, so, you know, I tend to have examples from academic life. But, you know, I can potentially, you know, just do the same papers that I've already done, but for different countries, right? So you mentioned my future of employment paper in the beginning with 47%. I could have just have done that for Germany, for France, for Sweden, for Zimbabwe, for China, et cetera, right? And you know, there would be a lot of papers. It would, you know, give me a long list of publications and I may look very productive, but each output would've been less, less influential than the previous one. Conversely, as an academic, you have a lot of autonomy. So I could just go out in the world and explore for a decade, but at some point people at the university might ask, well, Carl has been traveling around the world, but he hasn't produced anything in 10 years. What is he doing? Maybe that would've given me a Nobel Prize in a couple of decades, You don't know. But if you give people a lot of autonomy to explore, then they can go out and do potentially new things. And obviously you need to find the right balance.

Yeah, very interesting. And we've touched on AI now a number of times, and you actually place AI alongside nuclear weapons as an existential risk. What worries you most about AI?

So I'm actually not most worried about the existential threats, to be honest with you, although I'm concerned about them. But we've been through a pandemic, and what the pandemic taught us is that when something very dangerous or something that threatens a lot of people's lives, but not necessarily the human race as such, we throw everything we have at it. And so, you know, something doesn't have to be existential for us to focus on it. Right now, I'm actually concerned that we're, you know, beginning to stifle the AI revolution before it's already taken off. Right now, I'm not more concerned with the AI we have than I am, you know, about people potentially uploading dangerous information on the internet. To be clear, that might change, and, you know, there are good reasons to have rules and regulations around, you know, what kind of information you can train your algorithms on, and, you know, putting certain guardrails in place for that, just As you can't, you know, post the blueprints on a biological weapon on the internet. But it is also true that the more barriers to entry, the more compliance costs we add, the higher the risk of a few firms or even one firm monopolizing the technology. So take a sort of well-intended piece of regulation like the GDPR in Europe. Good reasons to care about data privacy, yes, but, you know, we have to look at the unintended consequences too. And the fact is that larger firms, larger tech companies, they were essentially able to offset compliance costs by capturing a larger share of the market where some smaller firms struggled to compete. And Europe's AI Act, I think, really risks repeating that mistake. Now, from a risk, pure sort of risk perspective, what kind of market structure you ideally want is somewhat unclear, right? If you have open source, there's obviously a lot of people that can do things. both good and bad. But if you have one firm that we all depend on for AI, that strikes me potentially even more disastrous. Fewer upsides, but not less downside. That firm is going to have a tremendous amount of economic and political power. And much of the populism, much of the polarization, much of the competition between different groups for resources that we see today stem from a zero-sum mindset whereby people think that if somebody's doing well, it must be at your expense. And if an economy is not growing, unfortunately, there might be something true to it. And what we see, and Stephanie Stanscheva and others have shown in their research, is that people that grew up in periods of rapid growth tend to have a more positive-sum mindset, whereas people that grew up in periods of stagnation develop a zero-sum mindset, and that zero-sum mindset can become self-reinforcing. And so, you know, to develop more positive-sum thinking and to solve many of our challenges, whether it's, you know, investment in the green energy transition, whether it's, you know, combating Demographic challenges, whether it is improving social security, you know, all of these things require economic growth and we need that to solve our key challenges. And I think we've— Our main hope for achieving a real boost to growth and productivity in particular is artificial intelligence. And so it would be really a shame if, if we strangle the technology before it has even taken off.

Yeah, so you're arguing for less regulation in order to create this decentralized, thriving economy where AI can be used to, to boost our economy.

Yeah, but to be clear, I think, you know, there needs to be some guardrails in place around, you know, what sort of training data you can use. And, you know, I'm open to the idea that as AI becomes more powerful, we may have to update those guardrails as well. But there's no way that we will know exactly, you know, all the challenges that we will encounter with AI going forward. And so had not OpenAI released ChatGPT, we would probably not even have a discussion around AI regulation, right? And so you need these technologies to actually be released into the world to have an informed debate about regulation to begin with. What I make very clear in the book is that Whatever the AI regulation regimes end up being, we need something which is much leaner than the regulations we have in place around pharmaceuticals, which has created tremendous barriers to entry and essentially means that any biotech firm that wants to develop a new drug needs to partner with a large pharmaceutical company because cost of going through clinical trials and complying with all these regulations are so high. And so that is clearly an outcome that we don't want.

Yeah, and you mentioned the fact that we need AI to boost the economy and grow the economy. What are the biggest opportunities then that you see for AI to improve business and society?

So I think right now, unfortunately, I think we're focusing too much on automation and process improvements. And as I mentioned earlier, that runs into diminishing returns quite quickly, right? And if AI means we do email and spreadsheets and some other things a bit more efficiently and, you know, eases, you know, the way we book travels and things like that, then the transformation is not going to be on par with, you know, let's say electricity and internal combustion engine, which created vast new industries. You know, in research, we've already seen, you know, promising applications in medical research in particular. that is very promising. At the same time, though, I think AI is not yet as transformative for science as the personal computer and the internet was, right? The internet essentially, you know, connected the best minds around the world in ways that was previously inconceivable. Computers enabled people to, you know, run simulations and statistical analysis at speeds and accuracy that wasn't possible before. And yet, you know, the productivity upsurge was relatively short-lived. And so it seems to me that, you know, we've created institutions that have become key bottlenecks even to very promising technologies. And, you know, If you're a scientist and you're spending 50% of your time writing grant applications, well, computers and AI can speed that process up a little bit, but probably not going to hugely improve the quality of ideas unless it means that you have a lot more time to think and drill for new ideas rather than, you know, spending that maybe a little bit of time gained writing another grant proposal or, you know, having that eaten up entirely by, you know, new compliance requirements, which seem to be the direction we're currently heading.

Yeah, and in the epilogue of your book, you use Moravec's paradox that AI is good at hard things but bad, and easy ones to frame the human advantage. What does it mean for the future of work?

So I alluded to it a little bit. I think, you know, humans have a resilience advantage, so we're in a much better place to deal with new novel circumstances as they emerge, and we are much better placed to actually develop and learn a new skill over, let's say, 6 months. If I have an RA that I hire and I pitch that RA against the best LLM, that RA will have better picked up my preferences and improved much more tailored to my wishes than the LLM had. I think a key challenge, which I think possibly you're alluding to as well, is also, you know, we learn more actively. We're out in the world exploring. AIs are not, you know. You can think about how much would you know, how well would you function in the world if all you've done was sitting in a basement for 40 years just reading everything you can? That's a bit of an unfair comparison because AIs read a lot faster and so they know a lot more stuff. But still, children are certainly learning things and interacting. before they're able to speak. And so, you know, a lot of the things that we know doesn't just come from language, it comes from being out in the real world, interacting and exploring, not just in a passive way.

In the book, you also stress the importance of ethics and inclusivity. So what does responsible innovation look like in practice?

Well, you know, first of all, responsible innovation is innovation technology that works, right? And so, you know, if you put benchmarks in place for inclusivity, for example, well, it better lead to better performance too. I think that's just, you know, should be pretty clear, but I think it's often actually missed in the conversation. But yeah, we do need, you know, more people to be inventors and to explore. And so, you know, if you take, for example, the fraction of female versus male inventors in the US, right? The gap is closing, but, you know, at the rate it's currently closing, it would take 100 to 120 years for it to close, right? And so that means that, you know, a lot of the brain power that we have is not being used, and it means that half of our inventive capacity with sometimes different preferences is not included in developing these technologies, and that is a real challenge.

So if you could give one piece of advice to leaders shaping the future of business and technology, What would that be?

Well, on that particular thing, I think role models are really important, right? And so, you know, people have to decide to become inventors at some point in order to become inventors, and that often happens by encountering a role model. People that have parents that work in science and innovation are more likely to become innovators themselves. People that grow up in communities with a higher density of innovators are more likely to become innovators themselves. And, you know, obviously business has a responsibility in creating these role models for different groups in society so that they feel included in that process. But other than that, I come back to the point I made earlier. I really think that if you want to thrive as a business in the AI revolution, you need to give people at lower levels of the organization more decision-making autonomy to actually implement the improvements that they are finding for themselves in the work they do with AI. on a daily basis and encourage that experimentation.

Very good. We're talking about the future of business and technology here on this podcast. I'd love for you to give us a prediction looking 10 years ahead. What do you see?

What has really surprised me, I have to say, is how good these models have become and how little it's changed the world so far. And so my prediction 10 years from now is that these models will become a lot better, but there is a ceiling where people might not even notice. any longer because it's already better than they are in a lot of activities. Unless you're one of the world's leading experts in a subfield of economics or physics or whatever it might be, you are very unlikely to notice improvements when it comes to domain expertise in the models that we already have. And so I think the progress from here is going to be harder in the sense that we need to develop models that are capable of, you know, making things up as they go along, as people do in the real world. And so without substantial progress there, I actually don't think that, you know, the world will look that different to what it is today. But that's also in large part because we just don't notice.

Very good. Thank you so much for your time, Carl. That was super interesting, and I highly encourage anyone to read your book, which is now available. For anyone who ever wants to rewatch this conversation, simply head to my YouTube channel where you can find this conversation and hundreds of others like this. Or if you prefer audio only, you can simply head to any podcast platform. Thank you very much, Carl.

It's been a pleasure. Thanks for having me.

Thank you.