Can The Current AI Boom Scale To AGI? Reid Hoffman & Open AI's Greg Brockman | Greymatter

This episode of Gray Matter is in partnership with Startup Grind, a global community of entrepreneurs. At the 2019 Global Startup Grind Conference, OpenAI co founder and CTO Greg Brockman and greylock partner and OpenAI investor and board member Reid Hoffman discuss if the current artificial intelligence boom can scale to AGI, also known as artificial general intelligence.

That is great energy for the end of a day. So let's kick it off because I think the subject's super important. OpenAI, why did you help found it? What's the mission? What's the way that people should understand OpenAI?

All right, well, just tell you a little about my personal story. So I first got excited about AI when I read Alan Turing's 1950 paper on the Turing test, right? And I was used to programming as something where you think really hard about a domain, you understand it, you write it down in this obscure way that we call a program, and then suddenly anyone could get the benefit of it. And I always thought that was amazing. And then you read this paper and instead Alan Turing says, we're not going to program an answer to this, we're going to learn an answer. And the idea that the machine itself could figure out how to solve a problem that I could not even describe how to begin to approach, that for me was just the most mind blowing thing ever. So I got very excited. I went to my professors, I said, hey, can I do some NLP research? They said, sure. And they pointed me at some parse tree probabilistic grammars. And I was looking at that, I was like, this is just not going to scale. This is not how we're going to solve this. And I was very sad. And so I decided to work on practical things like building startups. And I spent the next decade trying to do that. I was at Stripe for five years, and when I was leaving, looking around, you know, for me, I think that's.

About when we first met.

That's right, yep. And top of the list was AI. The question for me, you know, it's always been clear that the most important problem that I could hope to contribute to is artificial intelligence and really building machines that are able to do materially useful things in the world and do things that humans cannot and help us, you know, achieve new heights. It's just a question of when. And yes, this was about 2015 and three years into the deep learning revolution, things were starting to work. The default for me would have been start an AI company, try to get some cash flow, and then maybe five years later contribute back to general Intelligence development. And I was talking to Sam Altman, and he said, why not just go right for the general intelligence development? Like, it feels like we're on this exponential curve, and it's hard to know where that's going to top out. And so that's where we started. Right. I think that we brought together a group of people who thought very similarly to me in that story that I just told, where I think that for a lot of the people who were at OpenAI and were there at the founding, that general intelligence has just been the most important problem that they can imagine contributing towards. And it was the right time to come together and to actually have a serious effort to build it and make sure that it went well.

And so what is the founding mission like? If people are trying to understand what is the kind of the moonshot of OpenAI and what is the things that they should watch for from it? What are the things that they should anticipate the work will be heading in the direction of? What would you tell them?

So the mission of OpenAI is to ensure that artificial general intelligence benefits all of humanity. And when we say artificial and general intelligence, we're talking about machines that can outperform humans at most economically valuable work, which is a very, very high bar. I think that, again, it comes down to if you can actually build a system like that, it's clearly going to be transformational, and that there's a question of when and exactly how fast that happens. But I think that making sure that goes well is clearly crucial. Right. And you think about technologies that have come to market over the past couple decades, it's pretty clear that getting those right is hard, even when you have a long time to figure it out. I feel like today we're just starting to see some of the implications of choices that are made very early on in the Internet. With general intelligence, I think things can go even faster. And so we really have to make sure we get these things right ahead of time. And so the main thing that we work on is actually development. We actually are trying to build these systems. We're trying to push forward the state of the art. And we've made a number of breakthroughs in different areas, but we also work on safety. That's making sure that you build powerful AI systems, that they actually do what the designers intend. But there's also this other hard problem, which is, well, what should be the values of these systems and how should they affect society? And we have a policy team whose job it is to work with governments and other stakeholders and try to shape answers to those questions.

And how should people understand the open part of OpenAI? What does open mean in this context?

Yeah, so this is, I think, a misconception, right, That I think that you see open AI and you think, great, it's an open source project. And sadly, I don't think that being an open source project to AGI is the best way to benefit the world. We've actually published a document called the OpenAI charter where we document exactly how we think about what we want to release and how we want to benefit. And one point there is that due to safety and security concerns, that we're going to have to decrease traditional publication in the future. And the thing that really matters to us that is core and invariant, is that mission that we want to make sure that general intelligence benefits everyone. And that is baked very deeply into our structure and our ethos.

And so sketch a little bit the research program. I mean, probably people are fairly familiar with, you get a lot of compute, you get a simulated world, you get self playing agents, you have an early state on set of games. But how does that research program look like? As kind of like this is the vector in which AGI might get constructed. And then what's the theory of intelligence that correlates with that research program?

The way that I always like to think about this is starting with the AI that we all know and love. So for the past seven years now, deep learning has really come on the scene. Deep learning is not a new thing, right? It's been around for 60 years, and it's just now that it's really starting to work. And at the core, what's going on is that we're starting to have massive computational power, that we have an algorithm of a deep neural network which is massively scalable, right? The one thing that we found is just a pattern that just holds across domain after domain is if you scale up a neural network with appropriate data, you tune the architecture in the right way, it will work better. And then there's a third thing, which is the availability of data. Now the thing is that these building blocks of AI are starting to change in a very interesting way. One is data. And so to your point, so if you're doing reinforcement learning, you have a simulator, you have a game like for example, Dota, where we've had some breakthroughs. You're suddenly spending computational power to generate your data, right? The agent is actually playing against itself, is generating data that it can then learn from that you're learning from your own experience in the same way that a human bumbles around the world and sees things and learns from that experience. Now that's actually starting to work very well. I think there are a number of groups that have really amazing reinforcement learning breakthroughs and it's actually been very interesting to see. 2013 is the first time that you started to see exciting things with Deep q learning. And 2016 or so with AlphaGo. In 2018, we have Dota and now StarCraft. And I think that we're really starting to reach the point where if you have a simulator, you can solve that problem in a very meaningful way. But there's something else that humans do that none of that addresses, which is we somehow just look at the world. You look at a baby who just really just experiences and somehow is able to extract meaning from that, and you're somehow able to take all this past experience and apply it to a new problem. That's not something that describes the AI systems that we're used to, but we're starting to see signs of life with AI technology doing this. So we'll actually be releasing some results soon on doing this in language. This is something that actually we and a number of other groups have had nascent results on and we're starting to see that same sort of exponential progress where you just train a model to predict the next word in text and it's able to start learning to generate text. That is materially surprising. And actually you're able to use that past experience to solve normal tasks like classification or question answering that normally you would think you just need a big data set of questions and answers for. And so I think that we're starting to see a shift in terms of the story. Right. That people used to say deep learning, it's just pattern recognition. You just have question answer. I learned to answer new questions like that. But we're starting to see that's not quite true. It's also taking action in the world, and it's also looking at the world and extracting structure and information.

Characterize a little bit about how both the Dota result and the starcraft results were progress on this. Kind of like, what does it look like? Because, you know, obviously a relatively untrue person says, oh, computer, you know, AI stuff with learning algorithms, playing computer games. Okay, right. There's been some intelligence program in these games before from someone who's not up close and personal. What's the thing goes, ah, that's a really interesting set of results.

Yep. So I think it's all about distance from the real world. If you look at something like chess, it used to be that people said, hey, if we solve chess, we've solved intelligence, what else is there? Turns out quite a lot turns out that you could solve chess just by looking through more positions than a human possibly could. And you look at something like go. Which people said, okay, this is the new milestone. We can't just look through all the positions and suddenly you mixed in a neural network, which is basically an artificial intuition with looking through more positions than a human could. Those two things together could solve the problem. But you still have an issue with transferring this to the real world. In the real world, I can't look through positions. What does it even mean? Right? I can't run this talk that we're doing 100,000 times and pick the best one of those. And so you need to somehow have an artificial intuition that can perform at the same level as the best humans in some domain. And what's going on in these competitive esports games is it's exactly that you have these, these very long games that you have to do planning, that you have to be able to manage all these different units, you have different time scales you have to plan at, and you have humans that dedicate their lives to these games. I think that in some ways esports gets a bad rap. But if you actually look at these players, they're actually professionals, that they live in houses together year round training to become as good as possible. And so if you can exceed their capabilities there, you've learned something meaningful.

We'll get to the exponential curve in a second, both on the hardware and the other things, because this is obviously one of the things that's super interesting with the AGI, but what does this architecture of iteratively better games, what does that mean for how you think about what human competencies and human cognitive capabilities are and why? That's an interesting research program for that.

So I think that one thing that's great about games is that human designers have put in a lot of effort to make sure that they're actually challenging, right? That they do exhibit different skills, they're very interpretable, right? That you can look at behavior in a game and you can see, okay, this is what a human would do here. And you can see it's learned this, it's learned, it hasn't learned that. One thing that's also nice is that it's a lot of system one thinking, right? That the system two thinking, I think is another piece that we're missing right now and that it might be that.

Our characters should probably give the Kahna Mon.

System one. System two, you might remember from your college psychology textbook, System one, thinking is kind of reflexes, right? It's the instincts and kind of the emotional lizard brain. And system two is the deep thought. Right. Is, you know, kind of what you do when you're thinking very hard about a problem. And I think that what we have right now, by analogy, looks a lot like almost an insect. Right. That you have. If you think about our Dota agent, it's been trained in the simulation for many, many thousands of years. It's hyper optimized for this one environment, and it can actually respond to things that are totally alien to it in that world. Right. You have a human come in and play. It's the first time it's ever seen a human. It's spent millennia playing against other bots. Who knows what those bots do? They definitely don't do what humans do. And somehow it's able to respond and able to operate well in that environment, but it's never going to teach you calculus. And I think that that's one gap that we need to overcome.

And as we look at the game playing, what should we look at as the future of kind of mixed human and agent in terms of how the humans and agents can play together? I mean, that was one of the things that we were working on on the Dota environment.

That's right. Yeah. I think that we'll actually have some interesting things to show there. And I think that at the end of the day, AI is not about beating humans. Right. It's really about enhancing what humans can do. And so I think that one thing that's, that's very exciting is starting to see not just how can we outplay, but how can we actually make something that's materially different, where you have a collaboration between humans and AI. And I think this is actually a very untapped area. But we're starting to reach the point now where the systems are so competent that they actually have meaningful value to add to the human experience.

Now let's go back to the AGI kind of exponential curve. What's driving the expense? What leads to the. Okay, so we're at insects maybe now, but we are actually, in fact trying to get to human level. What are the drivers of that exponentiation?

Yep. So one core driver that I think was very surprising to us to discover just how fast it's growing is computational power. So we published a study sometime last year where we just looked at the most computationally intensive results from the past six years. And what we found is that there was more than 10x growth in that computation each year. And it wasn't that this was a planned thing, it wasn't that people even knew about it. We published this and everyone was surprised, including the people who had contributed to those results. But it's just the emergent phenomenon of all the money that's pouring in. All these people wiring together systems, people building faster chips and wiring them together in bigger systems. You look at devices like Google's GPUs, we track about 47 different hardware startups that all have a unique approach. And what's going on at a hardware level, at a silicon level, is that we've more or less run out of juice on Moore's Law. It's kind of hard to fit more transistors on there when you're talking on the nanoscale. But what we haven't tapped out is parallelism. And turns out the neural networks, just like the thing up here, massively parallelizable. And so, yeah, you can just wire together all these chips and you're going to get benefit.

So is the 10x increasing compute scale that you're talking about is actually not transistors on the chips, but actually in fact a better ability to get a bunch of the chips working together in a parallel scale?

That's right. So it's kind of the product of a few different phenomena. And one of these is wiring together massive systems. It used to be that in 2012, if you look at the history, there was a group at Google who started out trying to train neural networks on 16,000 CPU cores and were blown out of the water by two grad students on two GPUs. It's a wonderful David Goliath story, but I don't think it'll ever happen again. Because you look in 2014, people figured out how to use eight GPUs in a single server. And then start in 2017, I believe, is when we had a linear scale up to 1024 GPUs. And that's about a million dollars worth of hardware right there. And people are continuing to scale further and further.

So one exponentiation is compute, which is through the parallelization, which works very naturally with parallel distributed processing neural networks. What are the other areas that make the exponential curve?

So one that I think is often very undersung in these conversations is actually algorithmic progress. And it's a very subtle thing because on the one hand, deep learning, you can almost think of as one algorithm Right. It was the late 1950s that Rosenblatt released his Perceptron. And we're basically just playing with advanced Perceptrons right now. But at the same time, there are fundamental advances that people make, and often these fundamental advances, they're one line changes to the existing code. Right. Figuring out which line it is, that's the hard part.

I mean, it's kind of more of a general entrepreneurial audience. But is there a specific line you could call out that would show that illustration?

Well, so one good example is how people learn to scale to the thousand GPUs is basically changing one constant in the code. It's as you grow the number of GPUs, you're growing the amount of data you're processing at one time. And you just have to make sure that your learning rates the. The amount that you learned from each step was growing proportional to that. Interesting.

So now we're on an exponential curve, and there's at least two broadly disparate camps that react to this curve. One says, oh, my gosh, we're going to be able to solve all these amazing, interesting problems. And the other one goes, oh, my God, the robots are coming for us. Where are you on this? And how do you think people should think about this spectrum between these two positions?

I tend to think of myself as right in the middle, but I think that right now we have a lot of optionality as to how these technologies play out. I think that actually one thing is very disempowering as a technologist is to look at the history of technological invention and realize how almost predetermined it feels. You look at the telephone, two people on one day trying to patent it. How does that happen? What does that mean? I think what it means is that everyone's building on the shoulders of the same giants. You can never hope to invent something, something that no one else will. If Einstein hadn't been born, I think we would still have all the advances that we have. Maybe there would have been a time delay. Maybe the best you could hope for is that time delay. But I don't think that's true either. I think that the true impact you can have as the creator of a technology is setting the initial conditions under which it's born. You look at the Internet, there are lots of competitors, but the system that won ended up having its values be something that are reflected in the systems that we're using today. And of course, I think with the Internet, somehow those values don't always end up translating into what the designer intended. And I think that's really crucial, too. And so the way that I think about it is that by building these systems and by being on that cutting edge, you can see the implications before they've actually hit the world. And then you really get to choose, right? Not just as one company, but I think as a world we get to choose how are these technologies going to affect our lives? So I think that we can't hope to be in a world where AI progress stops, and I don't think we should want that. I think there's so many benefits that we can get, but at the same time, I don't think that we should just go and deploy and see what happens later.

So one of the things that's been happening in the last few years, starting with China, now the White House announced an AI initiative. How do you see the nation states playing into this? How within the philanthropic mission that OpenAI has, which is say, let's make sure this technology is for the benefit of humanity, is openly available, isn't just limited within commercial silos, et cetera, how do you see the nation states of both the White House is doing and also what China's doing?

Well, one thing I thought was really interesting in the executive order that came out was basically five different points, and one of those points was about international cooperation. And I think that that is actually a very encouraging situation, right? That at the end of the day, nations around the world are going to be deploying these technologies. And if you think about safety standards around the world, there are lots of different car manufacturers, but everyone has seat belts, right? And everyone wants there to be standardization on how to build a safe car. And that's technology that you should be sharing, you should be coordinating on. And so I think that in this kind of core, most important question of how do we make sure that these systems are built in a way that is safe and that does what the operator intends? I think that that is actually something that everyone's very aligned on. And so the question is just, can you get people into the room? And I think that what we're starting to see is at least there, there's appetite for it. It's still a high bar to really see it turn into something concrete, but. But I actually feel more optimism than I did say a year ago. And I think that the question of when you look at who's investing and it's not just those countries, right? It's, you know, France has a $1.8 billion effort. South Korea has a dollars, billion, billion dollar effort and a number of others. And I think that people are starting to realize that AI is here in a lot of ways and it's going to be affecting people. I think that the question remains for AGI how people will start thinking about that and how we can make sure that as these technologies go on that ramp, that we aren't caught unaware and are actually able to make sure that they benefit.

It's probably worth, for the benefit of the audience and everyone else to describe a little bit more in depth the OpenAI safety effort, because part of the reason, of course, you say, look, we're a public trust. We're trying to build this for the interest of all humanity and, you know, US Government, other governments as well. And so there's a natural place for trust and safety as a repository. So say a little bit more about the OpenAI efforts on this.

Yep. And so I think that you can kind of class both safety and policy efforts into that bucket. On the safety side, we think of that as technical safety. And so that's really solving technological problems for. Well, you think about today's AI systems and the problems that they have in terms of fairness, transparency, bias. Those are really important. You think about what's going to be coming down the pipes in terms of you have a really smart system that's empowered with controlling a power grid or controlling a whole city or an entire nation's economy in some way, and you want to make sure that that's actually doing things in a way that you like. You know, we already have highly autonomous systems that do things that we don't really think are good for society, sometimes called corporations. Right. Imagine a company whose job it is to just maximize profits. That's not really something we should trust. Right.

Or trust within certain boundaries. There's some things you.

That's right. I think you can trust that. Right. But I think that it's important that you set up the system in a way that eliminates the screwcases or that people that you trust are overseeing it. And I think that is how we think about the technical safety side, which is how do we actually make sure that these autonomous systems are not going to go off the rails and do things that we didn't intend. And it turns out, I think a lot of people have thought about this problem and gotten very depressed. Because if you think about it, how are you supposed to write down what humans want? Right. Can you write down what you want? No, I think that is an intractable problem. And so then people say, well, building aligned AGI Systems is an intractable problem. But the thing that misses is that we're already building systems that can do things that we can't really describe the process for. For example, recognizing a cat or a dog in an image. Can't write down the rules for that either.

Or even speaking and communicating with each other.

That's exactly right. And so we are starting to see, and we have results that we published on this, that you can actually have systems that can learn human preferences by looking at data. And I think that this gives us a lot of optimism for if you build systems with the right ethos, with the right, right safety first mindset, you can get all the power and you can also get the associated responsibility or accountability.

So how do you think this is going to play out in the China circumstance? Obviously the White House. This White House has kind of said a lot of confrontational things with China. China has its own AI initiative. What do you think the right way to think about this? Kind of. Because we would prefer, of course, a global pro human perspective. What's the right way that should think about this? The way that OpenAI is thinking about this.

So the term that we like to use is coordinated competition. It's clear that there will be competition across different countries and different companies. But again, there's this core that I think that we can all cooperate on and that's really safety. Right. Making sure these systems are built in a way that is not going to be destructive to the world. It's kind of hard to object to that. I think that it's really important for us. One thing that we say in our charter that we're very concerned about is late stage AGI development turning into an arms race. Right. If you are moving as fast as you can towards a transformative technology, the first thing that's going to go is safety. I think the winner being whoever can build the most unsafe system doesn't sound like a recipe for success. And so I think it's really important that right now, before you're in that world where everyone can feel the stakes, that you're actually extending the olive branch, that you're actually building those roots, relationships and building that trust. I want to be clear that I feel it's important to be a realist on these topics. And it's important that the way that we extend that all of branch is a way that feels both honest but also is prepared for all possibilities. But I think that the core of it really is that as long as we're in this world where we can keep saying safety Safety, safety, that there's real hope for coordination.

So let's talk a little bit about the relationship between AGI and the Narrow Intelligence act applications, right? Because there's like autonomous vehicles, there's stuff in precision medicine, there's all of these different machine learning applications that are going after these major industry transformations. How do you see the relationship between kind of the AGI moonshot up, this exponential curve and all of these specific machine learning applications?

There's a OpenAI micro view and an industry macro view. So as a startup, as a small company, we're about 100 people, which is very small compared to other people in this space. It's hard to work on more than one thing, right? You really got to pick. And for us, we see that exponential curve and we think it's very, very important that we stay on that cutting edge. And so all of the choices that we make in terms of what to prioritize are really about that, about staying focused and not peeling off into applications, which is a really painful thing. It's easy to say that. But when you, when you have some new breakthrough and you're like, I can see all the things this can be used for and not pursue that, it's painful. You feel it inside, especially as an entrepreneur startup. No question, no question. And I think that there is also the macro view here, which is that even if your goal was just to try to productize and make, you know, the best company out of AI, I think that one sad truth about today's AI technology, and I apologize to, to anyone in the audience who this, who this applies to, but I think it is not a good way of building a standalone business. You know, I think that it is a good enhancer to a business, right? If you want to do something that will say, you know, you want to do something in customer service, you still have to build a big customer base. You have to do all the enterprise sales and then you can have your automated customer service agent. But the bulk of that work is still going to be what it's always been in terms of building startups. The contrast is, if you're a big company, this stuff is great, right? You already have this giant user base and you make your photo search 10% better. You're just going to start attracting more users and you're going to see huge amounts of value coming from it right away. So the way they think about today's AI technologies, it's very high leverage, but in very small doses and very, very scalable. And I think that that's really what defines it? And so what that means is that I think that actually the optimal strength strategy, if your goal, which is not our goal, but if your goal was just to pursue profit, would be to stay on that exponential and to just keep going until you reach technologies that are actually valuable on their own. Right.

There's tons of AI and ML projects, applications all over. How many AGI projects do you track?

There's very, very few. Right. I think that AGI projects, first of all, you have to be willing to take on some real uncertainty. Right? It's a problem that I think many, many smart people buy into as being a very important problem. Question of timeline, question of how do you get there, question of how can you really affect it? And it requires pretty massive resources, right? That you need these massive computational machines, that you need to be able to have very, very rare talent. You know, one positive thing is that we find that it's actually really easy to take people who are great software engineers, turn them into great machine learning researchers in, you know, a number of months. So that's one problem that's gotten better. But the result is that there's really only, say, two, maybe three real AGI efforts that are out there right now.

So say you're looking forward five years and say the progress on the exponential curve continues. What do you think you'd say? This is the thing we knew we were going to get right, and we got right, and say the exponential curve wasn't going the way you thought. What would you think the principal thing would be that you didn't get right?

I think that the core really comes down to, well, it's the three things of data, compute, algorithms. And I think that that computation, we see that continuing for the next five years on the basis of today's compute technology. Right. You don't need smaller transistors, you don't need quantum, you don't need optical. You can just wire together more machines and move to smaller precision and do a couple other hardware tricks and you'll get more flops. And so I think that if things go as we expect, it will be that as we continue to scale the hardware, we're able to make use of it. It's one thing to have lots of flops, it's another to actually turn that into a smart machine. I think that maybe that's what we are as AI researchers. We're compilers from flops to systems that can actually be smart. And I think that there's also risk there. Right? It is the case that history has shown that we scale up these models and we're able to find ways of using those flops. But might turn out that these problems are harder than we think. It might turn out that we need not 10 breakthroughs, but we need 100 breakthroughs. It might turn out that whatever is going on inside of our heads, which is running on basically a petaflop machine, is some super hyper optimized process that has evolved over billions of years. And it turns out that you need to run a substantial amount of that kind of evolutionary history in order to replicate it.

So a lot of the AGI vision is a theory of constructing abundance because you can have robots that can do all bunch of different things. Manufacturing, you might be able to solve problems that you couldn't otherwise solve. Whether it's fusion or other things. It's all part of the utopian vision. What do you think in an AGI successful world is scarce?

I think that's a really interesting question. And one slight detour is I think that it's actually very hard. I think that a lot of how we picture technology technologies comes from science fiction. I think that's just always been true. And I think that for AGI there's actually very few positive stories out there. One that I think is there is Banks's Culture series and there it talks about a post scarcity world and turns out that people still have problems. Right. If you're a human and you want to find meaning, turns out that this is a thing that by its very nature requires effort. And so I think that it will always be the case that for us to find meaning is going to be an effort. And I think that again it's almost self fulfilling. There's no way that it could possibly be other. I think that ideally we're in a world where everyone has enough food and has personal health and that we're able to move past a world where you need to struggle in order to survive, I think that that's really crucial. And once you're there then I think the question of do you want to be an artist, do you want to go and see Jupiter? That all of those become possible.

Yep. And actually one of the things that I've actually persuaded some folks to do is help pull together a small working group for utopian science fiction because I actually think we're lacking it. If you look at all the movies, Ex Machina, etc, it's all AI as, as dangerous other versus you know, part of the thing is actually there's a bunch of interesting opportunities here. And by the way Ian Banks, like Player of Games, if you're interested in the culture, would be the. I think the one I would recommend starting with.

That's right.

If there was a science fiction plot that you would want for AGI, showing the utopic possibilities, what would it be?

So I think I'd start modeling after her, which I think actually does a good job of. You have a system who's just trying to help, who's there to benefit humans. And I think that the problem with these plots is you always need the tension, right? You always need the negative side. And I think that actually rolling through the deployment and figuring out how do you actually get to a world, not just a country or a company or a set of people who have these technologies baked into their lives, I think that that should be a pretty core part of it.

Totally agree. All right, well, as you can tell, OpenAI is on a hugely ambitious AGI moonshot project, which is part of the reason why I've been proud to be there from the beginning, helping and backing for the benefit of our humanity. So let's thank Greg and thank you.

Thank you, Reed.