Yann LeCun and Andrew Ng: Why the 6-month AI Pause is a Bad Idea

Sam hi and welcome to this live stream on this proposed six month course on building AI systems beyond a certain scale. I'm Andrew ng, founder of DeepLearning AI and I'm delighted to have you join me here today. And with me also is Yana Kun, who is VP and chief AI scientist of Meta professor at NYU and Turing Award winner. So Yana and I have been friends for a long time and both you and I have thought at length about this six month moratorium proposal and felt it was an important enough topic. I think it would actually cause significant harm if say, a government were to implement it. That Jan and I felt like we wanted to chat about it with you here today. Hey Jan, glad to be chatting with you about this.

Hi Andrew, it's a pleasure to be here.

So, just to summarize the situation, I think over the last decade, or maybe even over the last longer 20, 30 years, we've seen very exciting progress in AI. Deep learning is working well. And then even in the last one or two years feels like maybe there's even a further acceleration of progress in AI with generative AI systems such as ChatGPT, Llama also image generation, mid journey, stable diffusion, Dall E feels like maybe AI's gotten even faster. And associated with that there are people that worry about AI, you know, fairness, bias, social, economic, disability, displacement. They're also the further out speculative worries about AGI, evil sentient killer robots. But I think that there are real worries about harms, possible real harms today and possibly other harms in the future that people worry about. So in this environment, the Future Life Institute put out a call for a six month pause on moratorium on training AI models that are even more powerful than OpenAI's GPT4 model. And a number of people, very smart people, including our friend Joshua Benjo, Stuart Russell, Elon Musk signed on to this proposal. I think Jan and I are both concerned about this proposal. But Jan, why don't you start? Do you want to share your take on this proposal?

Well, I mean, my first reaction to this is that calling for a delay in research and development smacks me of a new wave of obscurantism, essentially like why slow down the progress of knowledge and science? Then there is the question of products. Like, you know, I'm all for regulating products. That gets in the hands of people. I don't see the point of regulating research and development. I don't think this serves any purpose other than reducing the knowledge that we could use to actually make, you know, technology better, safer Also.

Yeah, in fact, I feel like while AI today has some risk of harm, you know, like I think bias, fairness, concentration of power, so those are real issues. I think that is also creating tremendous value, right? I think, you know, with deep learning over the last 10 years and even the last year or so, last many months, technology, the number of generative AI ideas and how to use it for education or healthcare, responsive coaching is incredibly exciting. The value that so many people are creating to help other people using AI. And I think that as amazing as GPT4 is today, building an even better than GPT4 will help all of these applications will help a lot of people. So pausing that progress seems like it would create a lot of harm and slow down the creation of very valuable stuff that will help a lot of people.

Right. And I think there is probably several motivations from the various signatories of that letter. Some of them are perhaps on one extreme are worried about AGI being turned on at one point and then eliminating humanity on short notice. I think few people really believe in this kind of scenario or believe it's a, you know, definite threat that is not, that cannot be stopped. Then there are people, I think who are much more reasonable who think that there is real potential harms and, and danger that needs to be, that need to be dealt with. And I agree with them. There is a lot of issues with AI making AI systems controllable, make, making them factual if they're supposed to provide information, et cetera, making them non toxic. And I think there, there is a bit of a lack of imagination in the sense of it's not like future AI systems would be designed on the same blueprint as current autoregressive LLMs like ChatGPT and GPT4 or other systems before them like Galactica or Bard or whatever. I think that's going to be new ideas that are going to make those systems much more controllable. And so the problem of, then it becomes a problem of designing objectives for those systems that are aligned with human values and qualities. And I think, you know, thinking that somehow we're smart enough to build those systems to be super intelligent and not smart enough to design good objectives so that they behave properly, I think is a very, very strong assumption. That is just not, is very, that's very low probability. And then, you know, there is the question of which is more kind of a political question really is the question of what impact on people and the economy will be due to the fact that, you know, those products at the moment are produced by A very small number of companies that are going to gain power and influence and are motivated by the profit motive, does that have, you know, intrinsic risks? And the answer to this is proper regulation, obviously, but not stopping R and D, it's regulating products.

Yeah. You know about, about aligning AI, I have mixed feelings about that term. But getting AI to behave well, it's actually been interesting to see the rapid progress in the last few years acknowledging the real problems of, you know, AI generated toxic speech. Those are real problems. But I feel like not, not everyone appreciate when we move from base language models like GPT3 to the instruction two models like ChatGPT or GP315 Turbo or whatever, that was real progress. I think the models are now much less toxic, far from perfect and they still make stuff up. But I'm seeing real progress with instruction 2 models and this is why many companies are shifting that direction. But I think that stepping on the gas and dumping down on AI safety and aligning AI, that feels more constructive than proposing a blanket pause. Yeah, actually, yeah. And actually you're using the term AI Dumas. I don't think I've. That's an interesting term actually. I feel like you Yan, for a long time you've been speaking up against AI hype. You know, I think when deep learning was new, a lot of people had unrealistic expectations about what it could and couldn't do. And frankly I think I was over optimistic about self driving cars too. So I made that mistake. But a lot of well meaning people, you know, just overestimated. Right. What it could and could not do and, and unfortunately contributed a little bit to hype. And you and I been speaking against hype and I think this type of AI doom saying, or AI doom as your terminology, I, I think is actually another type of hype that AI could escape and overpower us all. And that type of AI doom saying hype is, is, is I think also harmful because it also creates unrealistic expectations.

I agree. And I think also the fact that now things like ChatGPT and GPT4 have been in the hands of people and then the Microsoft version of them, that people have been playing with it, it gives the impression perhaps that we are closer to human level intelligence because we as humans we're very language oriented. We think that when something is fluent, it's also intelligent, but it's not true. And those systems have a very superficial understanding of reality. They don't have any experience of reality really. They're trained purely from text. I mean GPT4 is trained a little bit with images as well. But mostly their understanding of the world is extremely superficial. And this is one reason why they can essentially produce nonsense that sounds convincing but isn't. And we're not that close to human level intelligence. There is no question in my mind, and this is not hype, there is no question in my mind that sometimes in the next few decades we'll have systems whose intelligence level equals or surpasses human intelligence in all the domains where humans are intelligent. But human intelligence is very specialized. We think we have general intelligence, but we don't. We are incredibly specialized and those systems are going to be much better than us in all kinds of domains. There's no question this will happen. It's not going to happen tomorrow until we have some sort of blueprint of a system that has at least a chance of reaching human intelligence. Discussions on how to properly make them safe and all that stuff is, I think, premature because how can you, I don't know, design, you know, seat belts for a car if the car doesn't exist? How can you design, you know, safe jet engines if you haven't invented the airplane yet? So I think, you know, some of those questions are premature and I think a bit of the sort of panic towards that future is misguided.

So maybe we're on a. We've made one year of wildly exciting progress in AI in the last one year and hopefully we'll keep on making wildly exciting progress the next 30 or 50 years and then maybe we'll have some sort of AGI then. But until we're closer, it's really difficult to know how the. And as we're on that 50 year journey, say, to get to AGI or whatever, a half month course before the next 49 and a half years of work. I'm not really seeing why that's particularly helpful.

Yeah, I mean, clearly we're not anywhere close to human level intelligence, otherwise we would have level 5 autonomous driving, as you were mentioning, and we don't. Right. How is it that a teenager can learn to drive in about 20 hours of training and we don't have self driving cars and we have systems that are fluent in language, but the amount of data they've been trained on, which is on the order of a trillion words, it would take on the order of 22,000 years for a human reading eight hours a day to go through this. I mean, that's just insane. Right? So clearly the kind of intelligence that is formed by this is not the type that we observe in humans. If we did. I mean it's another example of the Moravec paradox. Things that appear sophisticated to us in terms of intelligence, like playing chess and writing text turn out to be relatively simple for machines. Whereas things we take for granted, which you know, that any 10 year old can do, I don't know, clearing out the table, the dinner table and you know, filling up the dishwasher. We don't have robots that can do this yet. You worked on this, Andrew. I remember, on filling up dishwashers. But it's not.

Oh, totally optimistic. Yeah, yeah. And then, and then, let's see. So, yeah, so I think you agree, right? We're very far from AGI. Oh, sorry. I think all of you watching this, the YouTube live stream also has a link for slido content comments. I think Jan and I, we both have access to everyone's questions. We'll jump to that in about 10 minutes or something and take people's questions. So if you pop your questions in the slido Link on the YouTube live stream. So in addition to the problems of the premises of this AI AGI, AI escape, I think one of the challenges as well the proposal is it doesn't seem implementable. I feel like some things aren't implementable. So for example, you know, I think that having, proposing that we do more to research AI safety, you know, maybe more transparency, auditing, let's have more public NSF or other public funding for basic research on AI. Those would be constructive proposals. But the idea of asking AI labs to slow down, especially in this, you know, frankly competitive business environment with labs and countries trying to build advanced technologies is actually creating a lot of value. It just doesn't seem practical implementable to me. And the only thing worse than that would be a government, government steps in to pass legislations to pause AI, which would be really terrible innovation policy. I can't imagine it being a good idea for government to pass laws to slow down products of technology that even the governments and frankly even don't fully understand.

Right. And when we're talking about GPT4 or ChatGPT or whatever OpenAI puts out at the moment, we're not talking about research and development, we're talking about product development. Okay, so OpenAI kind of pivoted from an AI research lab that was relatively open, as the name indicates, into a for profit company and now a kind of a contract research lab for, mostly for Microsoft that is product development and doesn't reveal anything about the product anymore about how they work. So this is product development, it's not AI R&D and when stopping the product development, I think is a question, you want to perhaps regulate products that are made available to the public. If they endanger public safety, obviously that's where government should intervene. That's what it does for drugs and airplanes and cars and just about everything that consumers can put out some level of regulation here, if it's done right, is not necessarily a bad thing. But you don't, you don't regulate the technology, you don't regulate R and D in a technology that's, you know, that's, that's, you know. I'm going to give you an example also. You have to be very careful if you regulate technology that people want and helps. So I've use this example on some of my posts on social networks that there was knee jerk reactions of similar types. For example, when the printing press started popping up, the Catholic Church was extremely resistant to it because they said it would destroy society. And it did because it basically enabled people to read the Bible and it created the Protestant movement and you know, also had bad side effects like religious wars in Europe for a couple hundred years. But it also enabled the dissemination of the Enlightenment and you know, science and rationalism and democracy and which resulted in the creation of the United States by the way. So this had overall a good effect. So what we need to do when a new technology is put in place like this is make sure the benefits, the positive effects are maximized and the negative ones are minimized. But that doesn't necessarily go through stopping it. The Ottoman Empire, when the printing press appeared, stopped it because they were concerned that again, this would kind of get people to think outside of the religious orthodoxy or their political power. And you know, that part of the world basically kind of became an intellectual backwater even though at the time they were pretty dominant in mathematics and science. By stopping a system that amplifies people intelligence, which is what we're talking about here, you're taking a step back in terms of progress of humanity. I mean, AI is going to be an amplification of human intelligence. We might see a new renaissance because of it. A new enlightenment if you want. And why would we want to stop that?

Yeah, and just to add to the idea of regulating technology, I agree with you almost entirely. And there is just one exception. So a lot of the, some of the authors of Petition I think in the FAQ refer to the Asilomar 1975 meeting on recompetent DNA. And I feel like that analogy is not a great analogy in my opinion because when researchers are doing research on DNA recombinant DNA back in the 60s and 70s, their research on monkey viruses, for example. And there was a real risk of creating a new disease that would escape and infect people and harm. And as we've just seen with COVID pandemics are a real thing and they're really, really terrible for society. And so I think back in 1975, the Asylum Conference put in place containment mechanisms for if you're doing certain types of DNA research. And I think that was a good idea. And the reason I find it troubling to make an analogy between that Asinoma conference and what happens in AI, which seems to one of the popular themes out there is AI escape is just not. I don't see any realistic risk of AI escape, unlike escape of infectious diseases. That is a risk. But AI escape would imply not only do we get to AGI, which will take decades probably, but that AGI is so widely and so smart, it outsmarts all of these billions of people that don't want AI to harm us or kill us and that that's just an implausible scenario for decades or maybe centuries or maybe even longer.

Yeah, I mean, I think there are interesting debate, intellectual debates to have about, around this question. You know, some of the ones that you see online are unreasonable, but some of them are reasonable. So there is actually there's been a discussion following an article that I co wrote with Tony Zader that was published in Scientific American four years ago whose title was Don't Fear the Terminator. And basically what we said, you know, those scenarios where AI systems will want to dominate humanity are unrealistic because you need to have a motivation to dominate, to actually dominate. And that motivation exists in humans because we are a social species. It exists in a number of other social species. It does not exist in non social species because they don't have the need for it. And there is no reason it will exist in AI systems that we design. We can design their objectives so that they are non dominant, submissive, or they obey certain rules that are in line with the best interest of humanity as a whole. So as you said, this scenario is, is completely implausible.

Yeah. And there's actually one thing that grossed me, which is I think societal sentiment toward tech. I feel like 10, 20 years ago tech was all good. It's wonderful creating value for everyone. And then I think, you know, over the last five, six, seven years, society realized that there were real problems with tech. And then I feel like this is controversial. I feel like the media sentiment swung appropriately, but swung too far in the tech is bad and all these harms. And if you like a more balanced view, let's acknowledge the wonderful things tech is doing and creating for everyone, even while acknowledging the very real risk of harm. We do have problems with safety, interpretability, transparency, concentration of power, privacy. Those are real things that we're almost getting working on. And then I think this media swing a little bit too far to a negative sentiment on tech, I think feeds into this kind of hype and sentiment that I don't think is, I think actually a more balanced view would be more helpful for moving forward.

I mean, there's clearly some of that, I think this kind of perception, certainly working at Meta and being a bit of a public figure, I've been a bit of the center of that controversy somehow, even though I don't, I'm not involved in Meta's content policy or anything like that, or privacy policy or anything. But what I think people don't realize is that it's not just there is a problem with tech that the public and the press has identified, but the tech industry itself has identified those problems too and has attempted to correct them and mitigate them. So, you know, the side effects of ranking algorithms in social networks, for example, some of it was predictable and sort of fixed before it happened and some of it wasn't and it was fixed when it occurred. In the meantime, there was perhaps bad side effects and some of them are limited by technology. So one thing that people don't realize is that AI progress in AI is part of the solution to some of the problems that social networks have encountered. And I'm not just talking about Facebook or Instagram. That includes YouTube and TikTok and Twitter and everything. So the progress in AI has allowed things like taking down hate speech more efficiently and things like that. And this is due entirely to large language models, because we have large language models, not of the same type, not the autoregressive type that we see in ChatGPT and others, but the bird style. We can do things like do a pretty good job, a better job than we ever could in, you know, detecting hate speech in most languages in the world. That was impossible before that only occurred in the last four or five years. That's due to progress in AI. AI is part of the solution there. It's not the problem, it's part of the solution.

That's a good point. And I feel like the way that there are irresponsible ways to raw products like rolling a self driving Coffee kills people. That would be irresponsible. Well there are irresponsible ways to do that anyway. But then I think a lot of technology, it just isn't perfect the first time out and the practical way to roll it out is to roll it out in a self contained way. Limit harm, limit dangers and increments the rollout so you can better monitor mitigations for the harmful cases and then develop the solutions to it. So pausing progress seems like doesn't seem hopeful. We should release things in a very controlled way and make sure it's not harmful. And I think some of the risk of harm are overblown I guess.

Yeah.

Well we'll talk. Hey just keep on clock should you go and take a look at the. Let's take some of the questions on slido. So thanks everyone for posting questions on slide and voting out. There are over 500 questions. Sorry Yana, I won't really take five questions in seven minutes but the top voted one. What are the conditions? Scenario tipping point that if it happens AI post would be a good idea. Posted by Anonymous Pulse there.

Yeah, sure. So I mean if, if there is, you know, something that is deployed perhaps on a small scale for trials and there is identification of real harm because there are two kinds of harm. There is potential harm and real harm. There's imagined harm and, and you know, if there is real harm, you, you should, you know, stop that Harmony product from being deployed. Now should it mean that you should ban all of AI research? No. So it's very difficult to identify before it occurs. But when it occurs on a small scale you take corrective measures. And that's what's happened in the history of technology. The first cars were very unsafe. They didn't have good brakes, they didn't have safety belts, there were no traffic signals, blah blah blah. And those things were put in place progressively to make it safer. Same for aviation. And eventually you have a regulatory agency that makes sure those, those products are safe. So you know, that's the way it's going to, it's going to work. There is no intrinsic qualitative difference between AI and previous technological advances.

Yeah. And add to that in the case of genetic pathogen research there was a plausible path to harm. You know, pathogens escaping recombinant DNA as low recombinance. That was very difficult to pull back. Even the most powerful governments in the world can't shut off, you know, a pathogen once it's released. And I think the fact that today governments can, you know, pass legislations to cause companies to shut down their service. We do have the option to shut things down relatively quickly if it causes harm. So that, that kind of is another strong defense.

Exactly.

Actually, second question, Jan, this one mentions you. How should we interpret leading specific force, eg, Yoshi Benjamin being signatory while Jan opposes it.

Okay. Joshua is a, is an old friend, a very good friend. And we've known each other since he was a master's student and I was a postdoc. So that goes back a very long time to the 1980s. And you know, people have different opinions and different motivations and he certainly has different, slightly different political opinions than mine. And I think he's motivated by those for signing this. He sees the, the danger of, you know, companies controlling a technology for profit motive as intrinsically bad. I don't believe so. I think it should be controlled, but I don't think it's intrinsically bad necessarily. He is very much against the idea that R and D&AI should be secret, done in secret. And I agree with him. I'm a very, very strong proponent of open research, but again, we're not talking about research here, we're talking about products. I've been a huge proponent of open research and I think it should continue that way. Partly, I think people are unhappy about OpenAI being secretive now because most of the ideas that have been used by them in their products were actually not from them. They were ideas that were published by people at Google and FAIR and various other Academy groups, et cetera. And now they are kind of being kind of under lock and key. But this is not going to last very long in the sense that there's going to be a lot of other products that have very similar capabilities, if not better, within relatively short order. Yeah, OpenAI has a bit of a advance because of the flywheel of data, of having many users for the system that allows them to fine tune it, but it's not going to last.

Yeah. In fact, this is leading to Nick's question as well. Are you concerned that in the near future most advanced AI models we own solely by a few companies can afford the compute cost? I think I'll take a stab at this one. I am concerned a little bit, but not very concerned because the infrastructure layer for LLMs looks hyper competitive right now. I think OpenAI did a great job with scaling up LLMs transformers and really pushing for the instruction. Two models, really at least two major breakthroughs I would attribute to OpenAI. But meta release of Llama was a great thing. I think that Google's Palm model and it's becoming hyper competitive with many large and small organizations releasing different OLMs. So right now it looks very competitive and in the market space I see a lot of excitement then building the application layer on top of the large LLMs.

Absolutely. I think this is going to get democratized actually fairly quickly. So yes, if you want to train the biggest LLM ever, you're going to need a lot of computing resources, but most usage, I mean, there's going to be some sort of pyramid of all kinds of models where the simplest models which are still useful are going to be more widely available. You're going to be able to run them on relatively modest hardware, possibly on mobile devices on short order. And, and you're going to have a lot of such LLMs or systems of that type available with different degrees of openness for either research groups or products on short order. It's competitive, which means there is a lot of motivation for people to put things out there and some of them are going to be more open than others.

Let's take one last question. We'll do it real quick and we'll try to end on time. So, last question. How do you think about the role of AI in engineering? What should the student do to prepare for the future and make good engineering applications in the future?

Okay, go ahead.

Sure. So I think AI is wildly exciting. AI is a general purpose technology, both deep learning, supervised learning and generative AI. These are general purpose technologies, meaning they aren't good just for one thing, they're good for a lot of different things. And so these technological advancements give us opportunities to go around tech, non tech, all corners of the economy to find valuable applications, be it education or healthcare or coaching or lots of things, improving industrial automation, whatever, to find important applications and go and build them in a responsible, ethical, fair way to create value for everyone. So it's a fantastic time to learn about these technologies and go find, use cases and build and not to pause this exciting building.

Right. So that's on the side of engineering and product development and on the side of research. Nobody should get the idea that AI is solved. As I said, we still, we still don't have, you know, domestic robots that can clear the table and fill up the dishwasher. We're missing something big in terms of the learning capabilities of humans and animals that we still cannot reproduce in machines. We can't have systems that can reason and plan. The, the current LLMs cannot really plan at all. They can produce plans that already exist, that they stored in their memory. But they can't really plan. So there's a big, big challenge for the next few years in AI research for machines that can achieve human intelligence. And it's not going to be obtained by simply scanning up autoregressive LLMs and training them with tokenized multimodal data. It's going to take a lot more than that.

Well said. Lots of exciting research to be done still. Lots of exciting product work to be done still. I think it's a good time. It's actually exhilar and frankly, sometimes exhausting to be working AI.

I agree with that.

Hey, I think we're at time. Thanks a lot, Yann. I'm really glad we could jump on this call and chat. I'm really glad that you're watching this, that all of you could join us today too. So I hope this was helpful. Let's keep the conversation going. I think we should keep on pushing forward AI to create value for everyone. So let's keep the conversation going. And thanks. Thank you. Y thank you.

Thank you, everyone.

Thanks.

Take care, everyone.