Eric Steinberger and Daniel Gross on the future of AI at Sohn 2024

Hello everyone. Uh, it's a great honor to be following the illustrious Scott Goodwin, um, talking about, uh, the enduring value of a bank, uh, with a panel about AI. Um, I think there'd be quite a spread, uh, and so, um, would recommend everyone sort of buckle their seats and, uh, strap in. I have the great pleasure and honor, uh, of being here with the, uh, genius behind, um, Silicon Valley's latest hottest company, Magic, Eric Steinberger. And what we're going to try and do today is have— ostensibly, this is sort of a fireside chat where I'm interviewing Eric, but we're going to try and have a fairly lively back-and-forth conversation between ourselves about where AI is sort of headed in the future. The world's a little bit different here today in 2024. Uh, you know, with, um, all of these semiconductor stocks at all-time highs, and it's sort of clear that some semblance of AI, uh, is priced in, uh, to the market in some way. Um, but what we're hopefully gonna try and do today is give you guys a view towards where things are headed. Um, and sort of what aspects of AI may not be totally priced in or understood by the market. Um, but before we do all of that, I think we should just set the stage a little bit. Um, And Eric, it would be great if you could share with us a little bit when and how you got into this whole field and what sort of got you down the path of becoming one of the preeminent researchers in the Silicon Valley sort of artificial intelligence world. So tell us a little bit about that.

Thank you for that intro. I hope I can deliver about half of the expectations that you're setting. Thank you all for taking the time to listen to us. Yeah, I suppose like, yeah, 10 years ago I was a teenager and just sort of tried to pick— tried to figure out what's important in my lifetime. And after about a year of contemplating and pondering and looking and reading things, it became quite clear if you could simply build something that is like smarter than I am, it could do all the things I have looked at. And so decision Fatigue, you know, because decision paralysis goes out of the window. And I think it's sort of, you know, now more people believe this, but humans ultimately just use tools and talk to other humans in a sort of very well-coordinated multi-agent system. And there's nothing there that inherently you couldn't do with a very smart computer. And sort of from that very high-level idea, here's what we kind of want to be doing, you know, coming up with a— Technical, very clearly executable plan for how to make it. 10 years ago was not at all obvious. It was very unclear what direction this would come from. It seemed likely that it would come from sort of the reinforcement learning direction, which is sort of closer to AlphaGo, which you might have heard of a few years ago. Now, of course, language models are clearly an important part of it. Yeah, I think I spent a few years working on on reinforcement learning. There are a few types of RL. The thing—

What was sort of one of the first few projects you worked on that used RL that may have gotten you a little bit of infamy and fame?

Yeah, I think one of the most interesting aspects of RL is to sort of— if you take the multi-agent imperfect information, which of course you all being in the financial markets are very familiar with, system, and you try to train something that converges to equilibria in that domain rather than like a simple 2-player perfect information game where strategies are much simpler. simpler shape. So I spent a few years working on algorithms to solve various poker games, and we ended up building what I think is still the best multiplayer no-limit dynamic stack size, basically the full real game agent for poker, which just required a bunch of really cool research. We ended up selling it to a few of the world's— Poker pros who now use it as a training tool. So sort of, I didn't want to work for, you know, the big companies.

Okay, so you built the world's best poker bot, long story short, and then you decided to focus on, you know, sort of making this idea of artificial general intelligence, notably working on code. And so why focus on that? Like, why pick code as the domain to build your AI lab in?

Well, you know, so if you think about like where is AGI going to come from, it is going to be made by AI researchers and AI engineers. So, you know, if we could just build an AI AI researcher and an AI AI engineer, maybe we could, you know, speed that up and improve it and scale it further. So, you know, when other companies are, you know, collecting human feedback data for, you know, law, for, Um, Sora for, for video generation, for, for voice, for images, for, for anything and everything. Um, you know, that can get quite distracting. You need 1,000 people working for you. You have 50 research problems you need to solve. But, um, ultimately there seems to be a pretty straight line if you simply build an AI software engineer and research scientist. Um, first of all, you have, uh, a huge sector of the economy that you can call your market. Um, what I would feel like quite cringy putting these giant TAMs and slide decks for basically everything except the automation of the labor market, where it seems pretty legit. And at the same time, it's just a direct route to AGI. And I feel like everything else is just a byproduct.

OK, so I just want to make sure sort of we all understand this properly. Here is the translational layer from your genius into these people's Bloomberg terminals. And so you have this idea that you're going to be able to, at the bare minimum, monetize it, because maybe coding is a very lucrative market. maybe if you can use the model itself to then make subsequently better and better models. And so you can imagine this self-improving process, um, uh, that comes with that. So, um, that all sounds really nice. I think everyone here has had the experience of using ChatGPT, which is really neat, but at the end of the day, not going to be a great vessel for that. Um, if you refeed the same prompt over to ChatGPT for itself, um, let's just say you don't get an AGI. And so it'd be great if you could explain to us— and if the terms become too technical, I'll stop you— but try and explain to us in sort of simple terms the current state of the art in AI. What is that sort of based off of and built of? And sort of the main constrictions in the industry preventing another leap in intelligence like the one you're describing?

Yeah. Yeah, so maybe 7 years ago, this guy Noam Chazir, absolute genius, invented this thing called the transformer, which, you know, in GPT, the T is just a transformer. And effectively, over the last 7 years, people just made them bigger, and simply throwing more compute at it makes them a lot smarter. That will continue to work, but it's not the only thing you can do, and it seems to only— it is also not sufficient. So I'd say the current frontier and what's necessary to accomplish recursive self-improvement is to build something that instead of being, you know, reading the entire internet and then trying to predict things that sound similar, you need it to really be able to reason at inference time. So what I— okay, I'm gonna explain it. Basically, sorry, when you like—

You rehearsed this a few times.

When you make a decision, you know, for a complex purchase, you know, a complex investment decision you're making, You're going to take into consideration a lot of real-time information. You're going to think about it. You might take a week to coordinate with other folks. So you spend a lot of time that isn't you doing your PhD, you studying for 4 years what the market is up to. This is like you spending time on a specific problem. Or similarly, mathematicians— hard problems that get solved these days in mathematics typically take people years of specialized work to solve. They're not just, you know, I've done my PhD, I'm smart, and now I'm just gonna churn out solutions. But if you use ChatGPT today, it sort of, you know, it costs like 10 cents and it produces a result in 30 seconds. What I would like to be able to do is instead of spending 10 cents and 30 seconds, I would love to be able to put my question into ChatGPT and spend $1,000 and wait for half an hour and get something that's truly amazing out. And the, Reality today is that nobody has an algorithm that allows me to give it $1,000 and produce a smarter result. It's just, this makes, it would make so much intuitive sense that this would work, but there is no, that does not exist. That is not a product anyone is selling. And so that's one of the things we're working on where you say, you know, like, how can I feed a lot of real-time information to the model and have it just do long chains of, of reasoning on top of, uh, on top of it the same way that we would maybe look at a hard math problem.

I sort of, uh, I see. So there's this Transformer paper. I think people have heard about that Transformer thing, um, and that gets us to the state of the art today. Um, and then I guess my question would still sort of be, in, in your vision of the future, where it seems like you're describing a world where you can have a sliding scale of how much compute you want to spend to solve any particular problem, and you can sort of dial that up to infinity for the complexity of the problem. And so maybe figuring out the capital of France, you spend a penny on, but then, you know, creating a new drug, you can, with the same model, spend a million dollars on. What are sort of the breakthroughs that, you know, Magic has, Magic is working on, others in the field are working on, that are required in order to make that happen? I mean, is it just sort of buy more NVIDIA and scale up the transformer further? Is there a fundamental sort of mathematical breakthrough that's required? How should we be thinking of that?

I think there are a number of things that multiply. And that's a really important perspective to have when you look at this field. You can make up for bad algorithms by having 10 times more money to spend on compute. Or you can be smarter and have a better algorithm and then need less money to get to the same result. One way to think about this whole inference time— inference being usage— inference time compute idea that I just sort of shared is that you can shift compute from this massive, you know, $10 billion pre-training run. You could say, well, instead of spending $100 billion on this next giant model, I'm just gonna spend more at inference time. So the idea of shifting compute between training and inference becomes really powerful to sort of see the future, like what happens if we spend 10 times more money on pre-training and then maybe inference becomes cheaper again. Yeah, so what people are working on is basically all of it. They're working on scaling more. They're working on buying more chips. You know, I think we're especially interested in the opportunities of further algorithmic breakthroughs on the inference time reasoning.

Do you think just transformer is enough, or do you think more—

I think the transformer is categorically better than everything that came before, but there are things you can do that are better than the, you know, 2017 ideas.

Do you want to share what those are?

I don't think I should.

OK. We'll leave that for the happy hour.

Not so smart.

But tell us a little bit about the category of what those breakthroughs are, meaning is it a longer context? Is it a different kind of thinking?

Yeah, so I think basically— so the way these models work today, a good intuition for it would be that you take all the data that humanity has ever produced, all scientific research, all documents on the internet, all code, and you compress it into some sort of a massive associative memory that's just a bundle of floating point numbers on some hard drive that you can then throw a question into and get out like the mush of all the understanding, produce an answer. It's like the equivalent of reading the entire internet and then trying to do like association in your brain. That is nice, but it's not quite perfectly the thing we want. What we would really want is we would want to, instead of compressing today's state of the world, we would want to be able to update it dynamically. Say if you had a code base in a software engineering world, I might have tens of thousands of files of code if I'm a large company, as I usually have many millions of lines of code, and they change all the time. So if I had this massive training run that compressed my current state of the code, it's not updatable very easily or at all. Whereas what we'd really want is sort of this very smart reasoning thing that I can dump on a bunch of real-time information, that's sort of right now you might consider it a prompt when you send a prompt to a language model, you'd want to be able to say, hey, here is a mountain of information, please let me spend $1,000 to reason, for you to reason on top of it and come back with the thing I need. And so the transformer, if you take the sort of prompt, the context window, if you made that extremely large and you figure out this inference time compute piece, you can see how that's the right shape. But the transformer can't quite do this yet.

Right. OK, so you're describing 2 things, I think I'm getting out of what you're saying. One is a much larger context window.

Yes.

So for reference, like, how big is Magic's context window?

Well, we announced last year a 5 million token maximum length. We are at tens of millions now. And I don't know that I can announce the next one, but we're going to be at more than tens of millions.

OK, so it seems like one paradigm shift you're describing is the context window gets basically, uh, well, it gets very large. And then the second one is the model is doing active reasoning, a kind of active thinking over that context window.

Yes.

Um, and then what does that lead to? And just to texturize the question, I guess, um, in terms of the economic value of AI and language models today, I mean, it's sort of this work 70% of the time chat product. Um, and so that has some economic impact, but it's— the blast radius of that is obviously limited. Um, How should we be think— like, what's the next product shape we should be thinking about that's different from I go to chat.openai.com?

We prototyped a lot of things, and what seems to be the case is that there is a massive uncanny valley between, you know, a little assistant that effectively, you know, suggests things with very low friction, and often they're wrong, but sometimes they're right, and then it slightly accelerates me, and some— a system that you trust sufficiently much to just let it do the work for you. We're very excited about that latter shape. I think the user interface of something like this, you know, where today you're writing code, you have these code editors, they're made for editing, they're made for writing. I think we'll need user interfaces that are made for understanding. Actually, Bloomberg Terminal, I suppose, is a large piece of like, how do I condense this information into my available field of vision and neural capacity such that I can, I can like take in as much as possible and make good decisions? I think we'll see this for a lot of other jobs where the job of the person is no longer to actually press the buttons and do the work. It's more sort of the orchestration of letting the AI do it, but at the same time still having sufficient understanding of what's going on.

So when you say the orchestration of letting the AI do it, is this a kind of like I ask models to do work for me and then I'm supervising them?

I mean, so, okay, if you run a company, if you're the CEO of a company, and say it's like a pretty large company, you have a CTO, and then there are a bunch of engineering leaders, and they have engineers. In the case of software development, if you were the CTO, you never review pull requests or code changes suggested by another developer. If your team is good, you would have one person who wrote it and maybe one or two who would then review it. And then it goes through this sort of continuous integration and deployment phase where it gets tested and then it gets deployed to production. And the CTO, the only involvement of the CTO was, yeah, it'd be nice to have this feature, or like we should maybe use this database company, and then everything else just happens. So I think in a world where AI systems, through active reasoning and the ability to process information at real time through giant context windows, are reliable enough, they're not like 70% or 90% or 95% accurate, but they're like 99.9-whatever percent accurate, that's when you can trust them more than you trust human engineers today. I think everyone working on software, their job will just become orchestrating AI that does the work.

Wait, so you're describing—

There's this step function change. We can't see it until it like— Until it is that trustworthy, because it goes from this one-to-one relationship of like, I use my AI system to, oh wait, it just does it. And that changes things categorically. Okay.

So today you're describing we may be in this paradigm shift where it's call response, call response, I'm always engaged. And then you're describing a world where the models are much more agentic, where they're doing the work that maybe 1 or 1,000 people would be doing once in concert, and you're supervising their output. Maybe they come back to you a little bit later, it seems like, but their answer is always right.

And they might ask questions.

And they might ask questions.

This is the most interesting part. They might come back and clarify your intent.

So all of this is very exciting. How far away do you think this is?

I think the range would— the nearest term it could happen is very, very soon. And maybe— I mean, I don't see it be further away than 5 years, definitely. I just— so much would have to go wrong.

Not further away than 5 years?

Probably a lot closer than that. Anytime from tomorrow to Yeah, there's some distribution there, you know, but, uh, I think, you know, we'll, I, I reserve some, uh, opportunity to open the door.

This will be on YouTube forever. So. Exactly.

Yeah. It's like 50 years away. Don't worry.

Um, okay. So anytime between tomorrow and 5 years from now, and then how do you see, um, I'm sure people here will be very interested in the sort of macro implications of this, but maybe as a, as a mental model, we can focus on the coding industry as an example. What happens? If everyone has access to these millions of cheap, smart workers that can write code at superhuman capability, how does the world of software engineering change 2.5 years from now?

I mean, the 2 analogies that I feel, feel like you have to combine here are, um, farming and electricity, where in farming, you know, so if people invented technology that allowed you, to operate farms with significantly less human labor and be more productive per unit of land. And now basically no one works in farming, but we have way more food. So that's going to be one aspect. And I think another aspect that's going to be interesting is what happened with electricity when electricity started to become a lot cheaper, where initially electricity was scarce. There were barely enough lamps. And now we're lighting up the outsides of skyscrapers. At night. And so as price goes down, the viability of certain things that you might not do today goes up. Like, I think Slack is a wonderful piece of software, but I would prefer having an internal thing that doesn't send my IP to another company. So right now, totally infeasible. We're a startup. We're not going to make our own Slack. But if we had a system that reliably can generate code at a scale, and it's going to cost us $5,000 to make our own Slack, Um, sure, like we'll have our own Slack. So, so I think a lot more software will be created.

Okay, so you're creating a kind of induced demand theory where the cost of creating software goes down, so maybe there's just more of it. Everyone has their own custom little app for everything. I want to stretch out even further with the limited time we have left on the horizon and say, suppose, uh, for whatever meaningful depiction of AGI that happens, uh, even this decade, would that be a fair supposition?

Yeah.

Okay, so if AGI happens this decade, what does that sort of mean? What are the capabilities of these systems, and how should one be thinking of factoring that into some kind of view on what that means for the economy as a whole?

Traditional moats, as the startup world would say, I think will matter a lot more because right now, and this is just one of the things, if you're smart, you can go and figure something out and build a company on top of it. I, I feel intelligence, the intelligence as a unit of of input will be abundant. I think that's the summary, that just whatever is currently intelligence bottlenecked will go a lot faster.

OK, well, I leave that to the audience to convert to an expression and a trade. That's all the time we have today. Thank you very much for joining us, Daniel. Thank you, Daniel.

Appreciate it.