Artificial Intelligence with Prof Oren Etzioni

This is a global original podcast. Hello, and welcome to the session. The only place to gain legal neuroplasticity via the radio. This is the podcast that lives on the frontier of human ingenuity and endeavour, where our understanding of who we are, how we are and why we are is poked and prodded. This podcast will bend your mind. We seek answers from neuroscientists, philosophers, physicists and engineers about how their understanding of the world around us will change our world. This week we speak to Professor Oranet Zioni, Professor Emeritus of Computer Science at the University of Washington and CEO of the Allen Institute for Artificial Intelligence. If the future really is AI, what future are we hurtling towards? How artificial would that intelligence really be? Could computers even simulate consciousness? But we started with the basics. Where can I find artificial intelligence in my life right now?

When you get a product recommendation from Amazon or Netflix, when you use Google, those are examples of artificial intelligence in practice. When you use an assistant on your phone, like Siri or Alexa, it's AI at your service again.

I often wonder about driving because my car is able to. I can press a button and it can judge the distance between my car and another car and speed it should be going at, and brakes for me and what have you. And I'm often left wondering, am I driving it or is it?

Well, today there are some simple sensors that can measure distances and so on. Think radar or sonar, which we often use in parking, and that's not really very intelligent. We're hoping that in the future the car will have significantly more capabilities, and that could actually serve to reduce accidents, of course.

So what's the difference then between the sonar radar thing that my car is using to judge speed and distance and, say, the. The apparent intelligence of something like Siri or the Amazon recommendation?

Well, intelligence is a matter of degree, right? We. We start with, I don't know, a cockroach. Then we have a dog, maybe. After that comes Boris Johnson, and then here we go.

Here he comes.

Sorry, just trying to fit in with the.

No, it's good.

It's good local mood. So we have degrees. But a key decision is when there's uncertainty, when we're not sure who's going to make a sophisticated decision. When it's a simple thing. Look, the car is one foot away and it's coming at you at 60 miles an hour. You don't have to be a rocket scientist to say we have to hit the brake. But when it's more complex, like in a chess game, or when we're having A conversation like this one, that's when intelligence really comes to the fore.

So let me ask you about that complexity, because one might argue, I mean, it's obviously, it's very good and it works and it does the job. But Amazon forwarding me messages to say, would I like to buy another pair of shoes after I've bought five pairs of shoes isn't all that intelligent.

You're absolutely right. So we have tasks that we define that require intelligence, like making a good recommendation, and then we have programs that try to get better and better at it. So the truth is that the recommendations can still be way off. But if you think about the problem we have with holiday shopping every year, it's quite a tough problem. And. And so these programs are better than they were two years ago, but probably not as good as if you had a human butler.

You mentioned chess. Can you tell us about something called AlphaGo?

Absolutely. So chess was perceived as a major stepping stone towards intelligence for many years. But some years ago, in the early 2000s, a computer program beat the world champion. And then people felt, okay, sure, chess, but that's not going to happen for go, which is an even more complex game with even more trillions and zillions of possibilities. And then the match was arranged between DeepMind, which is based in the UK, and Lee Sedal, then the human world champion. And a lot of people were betting on Lee Sedal to win, the human to win. And in fact, a program called AlphaGo defeated him roundly.

Why is that so significant?

Well, it was a wake up call for people, particularly people in Southeast Asia, who perceive go again as the pinnacle of intelligence. From my point of view, it's actually a misperception for a few reasons. Reason number one, it's not the machine that defeated the human. It's the team of brilliant individuals at DeepMind, or now Google DeepMind, that built the machine and they defeated Lisa Doll. So it was really humans versus. And the computer was only doing the last mile of computation.

Hello?

I'm sorry, that's what's called an Amber Alert in Seattle. I'm just being alerted by the police that. Okay, the alert stopped. It doesn't know that I'm on live radio.

Sorry. No, no. That's amazing. So, sorry. What were the police alerting you to? Is everything all right?

There are two kinds of alerts. I apologize. This will stop in a second. One is. Well, One is a. What is it saying? There are Amber alerts. That's when a child is kidnapped. There are Silver alerts. That's When a. An old person has gone missing and there's a public safety alert, and I just got a public safety alert. But things seem to be stable. So let's. Let's keep going.

It's absolutely terrifying. All right, well, let us know if it. If you get any more alerts. Let us know. So you were talking about AlphaGo and the significance of the computer beating the human, but actually it was only the humans that created the machine to beat the human. And so really, all the computer did, the artificial bit did, was complete the task.

Exactly. And the second point, which I cannot overemphasize, is we in the field of AI are building savants. We're building very narrow programs that are extremely good at a task like playing go. But the program that plays GO can't cross the street, and the robot that can cross the street can't have a conversation with you. And the program like Siri that can have a conversation with you can't chew gum. So we're building very, very narrow systems. And of course, we all know that true intelligence requires a much broader skill set.

The AlphaGo machine can't say, right, go's done. Let's play baseball.

Exactly. Remarkably, also, it's not autonomous. It can't even play another game of Go until a human pushes a button. And that's a very important point, right, Alpha? Because when we talk about the threat of AI, how threatening can AlphaGo be if it can't play bass, and it can't even do anything until you tell it play another game of Go?

So, yeah, tell me, tell me then, what you make of what's been talked up regarding the threats from AI Because Elon Musk and indeed Stephen Hawking have suggested that the buildup of artificial intelligence, machine learning, and we can get into the distinction and why that matters a bit later, but this could be the thing that ends it all for us, right?

So my perspective is that there are many issues having to do with AI, its impact on privacy, on jobs, those lousy recommendations that it's giving you for a pair of shoes. But the one thing that we should not worry about is AI taking over the world. And you're absolutely right that brilliant folks like Elon Musk and Stephen Hawking, the late Stephen Hawking, were concerned about that threat, but for a variety of different reasons. I'm here to tell you and your audience, they're just plain wrong.

What do you know that they don't?

Well, I am in the trenches of this field, working every day on building AI systems, and I Think that the late Stephen Hawking, if you look at his writings and his words, he's a person who thought about the beginning of the universe, the end of the universe. You know, if you tell him, look, you may be off by 500 years, he wouldn't blink. Right. He's a guy who thinks about billions of years. So I would say that for Stephen Hawking, is much smarter man than I am, than I ever will be. It's just he's off by what we call a small constant, a few hundred years. As for Elon Musk, I frankly can't begin to fathom his perspective because he is building autonomous vehicles, and so he should know better. And the only thing I can say is that he's such a visionary that perhaps the difference doesn't matter to him. He predicted several years ago that by now we would have cars driving all the way from Seattle to New York with no human in the loop. And we don't.

But we might.

We will in the future. So now, as the old George Bernard Shaw line goes, now we're just bargaining over the price. Sure. I do believe that artificial intelligence, full artificial intelligence, human level, is possible. Possible. It's a question of when. But it's also beyond the foreseeable horizon. It's more than 20, 30 years out.

Here's Eli, new caller, Ilford. Hi, Eli.

How you doing, Nikki?

Yeah, good. Your question and thought.

Yeah, Professor. Yeah. I disagree with you about AI and the destruction of human beings because you had that experiment with Facebook with the computers, and when the computers couldn't. With this Facebook AI, scientists didn't understand what the computers were talking about, and they unplugged it. You had that. And there's also. You had the Hadron Collider where they said that there would be a mini black hole. Because obviously when you do experiment, the reason you experiment, it's all probabilities. There's that 1% chance you know what's gonna happen. Same with AI as well. And then you've got China itself, China, like quite advance in AI, and they're messing that with biotech.

Anyway.

Just, just before we spin off out into quantum computing. Can you come back to the. The point about the Facebook experiments that you say shows that the professor's wrong?

Oh, yeah. There was a. There was a Facebook experiment where they made a. Where the Facebook AI text. They made a computer with certain AI technology and they get it. Random tasks to do. And after about a month, they didn't understand the computer was saying something, but they didn't understand what basically happened, he reprogrammed itself, and it was communicating with a different computer. They made two computers. So what they done, they unplugged it. So it wasn't like it was. It was quite famous in the media.

So you're saying the computer learned how to speak to another computer to the point where the scientists had no idea what it was saying and what it was doing?

That's correct, yeah. And they unplugged it when they were, when they were asked question, they said, oh, we learned what we needed and that's why we unplugged it.

Okay, wait there, wait there. Let's get the professor's view on it. Professor?

Well, I really appreciate the question, and I think that the information we get about highly technical items is through Hollywood and through the news, and it does have a sensationalized element. So Eli's description of the experiment is actually quite accurate. Right. It was some interesting evolution of the language there, but it was completely non threatening. It's not like the computers would then conspire to take over or to avoid being unplugged. All it is is a natural sequence change in the language, in the words that were used. So I think the place where I respectfully disagree with Eli is around this. What is the chance, right? He's saying, look, these computers are unpredictable. Do we really understand them? Maybe I do a little bit, but who really understands what goes on? Is there a 1% chance that they will run amok? And the answer is no. The example I would give you is if you go to your stereo, when you play music and you start adjusting the dials, you say, okay, let's get the equalizer, let's get the volume up. I want to hear the radio better. What's the chance that by adjusting the dials, you, your radio, your stereo, will turn into the Death Star from Star Wars. And the answer is that's not within the realm of possibility. Even if you don't understand all those little buttons and what they do, it's just not in the realm of possibility. So I submit to you that what these AI techniques and machine learning techniques are doing and what the bots of Facebook were doing, they were adjusting some dials without getting into technical details. They had no risk of running amok.

Eli.

Yeah, but like I said, science is an experiment. So you, as a professor there, the only reason you do experiment, because you don't know what the full consequences of the experiment is going to be. There's always 1 2% chance that something else can happen that you didn't figure out that your probability experiments didn't figure out. So that's one thing. So you can't be sitting there telling that, you know, 100% what's going to happen. That's one point. Second point is this, that, you know, China are ahead of their time in AI technology and they're messing about with quantum computers and they're doing something with biotech and AI, so creating something that's self aware. The difference between AI and us is not biological, but they're messing with biotech and AI to make something. And a lot of religious people say one thing. The religious people say that they will never mess about with AI because they believe that God's created us, you know, perfect. But I believe that I don't know you, Mr. Professor, are you. Do you believe in religion? Could I ask you that? Would you answer that?

I'm happy to answer that. There's so much in your question. But I just want to go back to the first part where you said, hey, you don't know what's going to happen. Let me give another analogy that I think captures the point perfectly. Let's take a die, the kind we roll to play backgammon or some other game. When I roll the die, I. And basically nobody knows what number is going to come up. Is it going to be six or five or three? But I can assure you that it's going to be one of the numbers one to six. So I don't have to have total certainty about the outcome to know what the range of possibilities is. And that is the same point that I'm trying to make. These programs are very limited, okay? And they cannot just radically transform into something else. Now the questions of biotech and quantum and what China is doing, there's a lot more to say there. But I just want to get the point across to you and to the listeners. This isn't something, some primordial ooze out of which who knows what Godzilla will emerge. These are computer programs, zeros and ones. And we have very clear understanding of what the Outer Limits are. The truth of the matter.

Sorry, sorry, Professor. No, no, no, you go. The truth of the matter. Need to get to that.

The truth of the matter is that it's not that we're struggling to contain them, you know, keep them in the lab. Like I'm terrified of viruses escaping from a, from a biology lab. But it's not like we're struggling to keep these AI programs in check. It's quite the opposite. We're struggling to get them to do valuable things. They're so dumb is the problem.

Okay. Hey, Eli, My only contribution to your good questions would be not to worry because they turned the computers off, they just pulled the plugs out. It was fine.

Chidor is though, the problem is I don't want it to be like the Manhattan experiment. And that's the thing, that's the way where we're going to end with scientists, they don't have limits. You need limitations, you know, I mean as human beings we need some limitations. But when you.

What do you mean?

When you've got. I mean, I mean in anything it has to be a certain limit how far we can go, you know, I mean, what we.

That's what the professor is saying, that the, the limitations are like rolling the dice. The dice has numbers between 1 and 6. You're not sure what number it's going to be, but you know, the outer limits of it.

Well, depends on everyone's doing anything different. With AI technology, it's not like nuclear weapons where we don't give it to everyone. You know, we have treaties on it that we say that certain countries can't have it. AI technology, anyone can do that at the moment. Yeah. So anyone in the east or south or the west, anywhere in the world can do it. So you don't know what experiments. There's no limitation, you see, there's no body to limit the experiment. So I mean, they could do where they want to do, you know, I mean, that's the thing. But there has been one, just quick one on a lot of people. Wait, just a quick one. There was this group of AI scientists and they asked themselves this question around the table, that if there was AI technology and they were self aware, what would happen to human beings? And they all argued amongst each other. They greedy one thing, if the AI, if they were self aware, they'll treat us like ants. So basically when you've got antennas, you walk around it, but if you build a house on top of it, you build a house on top of it. So I would like to leave you with that.

Well, I think it's, it's contained in Mark, in Harrow's text, that thing about antsy like, and I think Mark attributes it to Stephen Hawking. In short, the advent of super intelligent AI would be either the best or the worst thing ever to happen to humanity. The real risk with AI isn't malice, but competence. A super intelligent AI would be extremely good at accomplishing its goals. If those goals aren't aligned with ours, we're in trouble. You're probably not an evil ant hater who steps on ants out of malice. But if you're in charge of a hydroelectric green energy project and there's an anthill in the region to be flooded to, too bad for the ants. Let's not place humanity in the position of those ants. Professor.

I do agree actually with a few points here. Perhaps surprisingly, the first one again to the caller is we absolutely need to place limits on science and how it impacts society. So for example, I've argued that all the systems we build should have an off switch and one that's easy to pull the trigger on. Secondly, to the eloquent comments that you read, I do think that if you presuppose super intelligence like in the movies, then the human species, the world, is a very different place than it was. But that presupposes something that is very, very far in the future. We don't know how to achieve it. We have decades, if not centuries to prepare for it. And frankly, I'm a lot more worried about the impact on jobs, right? What is AI doing to transportation workers, to retail workers, to clerical workers, to people in warehouses, to people in the healthcare sector when we have robots that are quite competent, when we have autonomous vehicles? So I just feel we have much more pressing concerns when we think about AI's impact on the world.

Eli, thank you for your questions. 0345-606-0973 Eli mentioned the phrase quantum computing. I've heard of it. I don't know what it means.

Professor well, at the core of computers is zeros and ones. The computer represents everything, it's data, the world, the programs in zeros and ones. And everything is set. It's not probabilistic. Either it's a zero or it's a one and off we go. Somebody had the crazy idea that if we harness the basic ideas of quantum physics, complex ideas that, frankly, I don't fully understand myself, we can build computers that are much, much faster than today's computers. Not twice as fast or 10 times as fast, but ridiculously much faster. For example, these computers could break all of our cod that underlie banking, underlie our spy agencies, and so on. Since that vision was articulated, people all over the world, at startups, at Google, at Microsoft, various government agencies and China have been at work on developing quantum computing. And people disagree on when we're going to have practical computers that can do these super fast computations. That's in a nutshell.

But it's happening, it is coming, it.

Is happening, and it Remains to be seen how practical it is in the short term, but people are actively working on it and there have been some rather impressive demonstrations. I kind of liken quantum computing to a fusion. Right. It's a real phenomenon and the question is, can we harness it in reasonable cost to produce what we want? In one case, cheap clean energy, in an other case, unbelievably fast computational power.

So is there a. Based on the experimentation done so far in quantum computing, can you give us an example of what actually has been achieved? What did an experiment find or show or be able to do?

Yeah, so far, not a whole hell of a lot. There has not been a computer that has really done anything practical with quantum computing. I mean, but that's the nature of these things. Right. If you remember, right. Turing is. Alan Turing, Right, again, of the uk, is venerated as the founder of the field of computer science. But when he was imagining computers, when he was conceptualizing them, they couldn't do much of anything. And there's a famous line from the CEO of IBM, which was at the time right in the typewriter business, it said that the global market for computers may be five or six. Right.

No, get out.

No, no, this is Google. It. It's one of the most remarkable business miscalculations.

That is a mistake, isn't it? I mean, know your industry.

Well, it was nascent at the time. He didn't say this last week, you.

Know, but does that not then go some way to maybe challenging yourself on your. I was going to say assumption, but that's unfair. But at least your assessment that AI is largely going to be fine, someone can turn it off if it's a bit problematic. And at the moment, and it won't be till very far in the future, that any of these quantum computing stuff has any real impact on our daily lives.

So, again, let me go to the die analogy, because it's very concrete. There are certain things that we can be uncertain about, like what's going to be the market for iPhones or computers decades hence. But I am quite sure, right, that when I roll the die, it's going to be 1, 2, 3, 4, 5 or 6. And I think you accepted that. So certain things we can be sure about, certain things we're uncertain about.

Yeah, I do want to. I will come on to challenge that in a bit about potentially what we've seen with biomechanics and, you know, the playing around with viruses and genetic sequences where you would think that they knew that there was a maximum range of 1 to 6. But accidentally, someone created a 7. The basics of quanta is that something can be two different things at the same time. Up can be up and down, one can be one and zero at the same time. It can also mean that you are also currently in another universe right now doing exactly the same thing you are here. Andrew, I have tried with that many worlds multiverse theory stuff. I can't get my head around it. So I choose not to. Thank you for the text. 03456-060973 professor, question from Ali. Could we ever teach AI to have an approximation of human emotions?

That is a fascinating question. And what I would say is that there are two components to emotions. If you think about it, there's a cognitive component where I think, oh, you know, I don't know, my friend got a bigger, bigger raise at work and I'm jealous and say, well, why are you jealous? Well, because he's got a bigger raise and I'm just as competent as he is. And then there's a physiological component where I feel that. That jealousy. I just feel envy. And so we can definitely. I beg your pardon.

Sorry, sorry. You can express that. That emotion.

Exactly, exactly. And so I. We can and are teaching computers to approximate emotions. And actually we've even been working on how to approximate ethical decisions, how the computer will do that. That's a research project at our institute.

Wow.

But the physiological aspect of it is not there. Right. That kind of visceral element.

Were that to be possible, and maybe one day it will be possible to at least give an approximation of the physiological reaction of the human response to guilt or sadness or rage that came from the computer, the machine. Do you think it would matter that you might be able to perceive the robot as being conscious even though it had no inkling of consciousness? All it was doing was displaying the physiological characteristics of a human emotion. It doesn't actually have anything behind it. There's nothing going on behind it. Do you think that would be enough to convince people that the machine was conscious?

Well, first of all, just to be clear, we are now in the realm of philosophy and science fiction. Right? Right. But it is a fascinating realm. And I would say that the question is, at what point do we say the computer is conscious? At what point do we say the computer has its own kind of humanity, if you will, and we need to give it. To give it rights to respect its feelings or.

Because I don't think we'll ever get there. I think, you know, our brains are so complex, we've Got no idea what's happening in our brain. As we've discovered on other sessions, the idea of mapping consciousness onto a machine just feels entirely impossible and out of reach right now. But if the machine were able to simulate the idea of consciousness, that might just be enough. That might be enough to think, oh, I need to take care of this machine. Oh, this machine is my friend. Oh, I could love this machine for humans.

So let me give you two examples of simulation. It turns out that the word simulation is surprisingly ambiguous. Imagine that I were to simulate a storm, a winter storm, you know, with all the air pressure, moisture, an exquisite level of detail. Right. I have a tremendous amount of data, tremendous computational power, and inside the computer, I simulate a winter storm. That simulation will not make the computer wet inside because it's just a simulation. On the other hand, imagine that I simulate in the computer a very complex mathematical computation. Maybe I'm multiplying a million digit number by another million digit number. That simulation is in fact all that's necessary. Right? Simulating the computation, simulating the multiplication results in the multiplication. So the question you're asking can really be reduced to the following question. Is consciousness more like a mathematical calculation or is consciousness more like a storm? And I think that's kind of a difficult question to answer from an engineering.

Perspective, then which do you think? Clearly the mathematical bit would be the easier bit.

Yes, I believe that it's the easier bit. And I do believe that ultimately thought, intelligence, cognition is very much a computational process, albeit a complex and mysterious one. And so I believe that ultimately we will be able to simulate intelligence and then as a result of that, we will have it.

You mentioned some of the experiments that you're doing or the research project that you were doing. Can you expand on that?

Sure. So we asked ourselves, it's one thing as these computers get better and better at product recommendations or at playing chess or go, but what about conundrums having to do with our values? Is it okay to mow your lawn at 3 o' clock in the morning? Is it okay if your boss calls you at home at dinner time to not answer her call? There are many nuanced situations. If I have a choice between, you know, God forbid, killing one person or killing four. Right. What's called the trolley problem. How do I direct my car? And we said, could we teach a computer to make those decisions? And we built actually a computer program called Delphi. And it's available to anyone who wants to try it for free 24, 7 at Delphi. Allenai.org anybody?

D E L P H I E I that's right.

D L P H I dot Allen AI that's a L L E N A I dot org and there were articles about this in the New York Times and in the Guardian. So if you just type in, you know, Delphi Alan AI or Delphi Morality, you'll find that and this program allows you to post to it brief hypothetical snippets like I don't know, what if my mother in law came to visit and I was tired, May I hide in the bedroom? Really on any topic, you know, am I allowed to eat my son's Halloween candy on any topic? And you give it a short snippet like that and it'll come back and say that's acceptable, that's wrong, that's understandable, etc. And the way this program was trained is it was trained on 1.7 million ethical judgments and social norms that we collected and fed to it as its training data. But now that it's been trained, you can ask it anything and it'll give you its best approximation.

And does that. I mean that is absolutely astonishing. 1 so please don't take my question to mean that I'm not impressed, but how does that show intelligence?

Well this goes right back to the question and again, Turing was truly a genius because he foresaw this in the 50s. He says, if we are dealing with a computer program, how do we decide whether it's intelligent or not? And he said one way would be to look under the hood, open up and look at the software. But the truth is that if I looked under the hood in my own head or in yours, what I would see is a bunch of nerve cells connected and chemical processes and light, electrical impulses. I wouldn't find intelligence, I wouldn't find a little box that says that's where the intelligence is. So he said maybe instead we should judge it extern. We don't open the box, we don't open the brain, but we see how it behaves. And so in AI, when we say that a program is exhibiting some level of intelligence, what we say is it behaves intelligently, we give it a test like an IQ test, and we see how well it does. And so the success of our program Delphi on these kind of little ethical snippets, these little situations, is remarkably good.

That does make me think though, going back to the consciousness point, that if the judgment is all our own, this magical thing called consciousness cannot be planted into a machine. And we go, brilliant, that's the consciousness cog off it goes and it whirrs away and you've got. That's consciousness. Then we will always be able to decide for ourselves what we think is conscious or not.

Well, let me put it to you this way. How do I know that you're conscious?

Yeah, yeah, yeah, right.

It's a very tricky thing to determine. I take it on faith that you as a fellow human being are conscious because I'm conscious. But it's very difficult to determine. Imagine brain patients, how do we know kind of what effect has had? So it's a very thorny question.

Ian's in Watford. You have a question, Ian?

Hi.

Well, I'm going to have to sort of end it up with two. In listening to everything, one of the things that bothers me about the artificial intelligence because it can almost become like a religion. So in other words, if everybody becomes dependent on the artificial intelligence, that people will follow it like a religion. Oh, I'll just check with artificial intelligence what I should do, what I should think, how I should think. And of course then, then that is dangerous in the sense that maybe somebody can manipulate it to manipulate people to be thinking in a certain way. So that's what's scary to my mind. And of course the other side, what I was thinking before is how do you set your artificial intelligence at what level of, for instance, risk. You know, the risk factor. We all live by risk. You know, you cross a road, you would take it because you're quicker than somebody else or whatever, and you would take a decision. It. Where would the artificial intelligence if you become dependent on something like that? But the scary bit to me is the fact that if people believe in it in the same way as people believe faithfully in many religions, that to do something in the wrong way.

Yeah.

Will bring grief to them.

So you say that the machine scares me. Sorry, Ian. You say that the machine is intelligent because it displays what you think is intelligence, and then you not just give it the opportunity to come up with what it would suggest, but you allow the machine to decide for you what to do. So using the Delphi program, the machine decides for me that I'm going to swerve this way rather than that way. If it were used as software in a car, how you set the value judgments, how you make sure the dice is one to six. How do you do that?

Well, I really agree with Ian's comments. I think those are very real risks. And the reason I think that is we've already seen them. So let me just mention that quickly we've become very reliant on Google. You know, who remembers facts anymore? I just looked them up on Google, right? And we've become very reliant on our smartphones, Right. I don't remember phone numbers. I used to memorize phone numbers. Now if I lose my phone and I need to call my wife or one of my sons, what is their number? Who remembers these things? So the convenience of technology does become this crutch, which leaves us in a bind if the technology doesn't work. So that is just true, and we need to think about it. As for the second point about the level of risk and kind of the way that people come to believe in it, we have evidence of that, actually, in the American criminal justice system. People in the justice system were using computers to make recommendations about what's called recidivism. Should you give the person parole, let them out of jail, or should they go back and spend more years? And the programs were flawed, and the people using them gave them too much credence. They relied on too heavily on their advice. And that was really a problem. So these, Ian, these are fears that I share with you and that we need to deal with as a society now, Ian.

Yeah, well, that's what I think. That's. I just, you know, if things are packaged, unfortunately, we live in this very clever marketing society which if you package something correctly and you market it and you put some celebrity saying, this is fantastic, people will follow it. And that's what's scary. That's, you know, and then somebody can manipulate it. That's what scares me, Ian.

Great. Call Ray in Orpington before we finish. Hi, Ray.

Hi, Tom. The fear of AI taking over control was, of course, first voiced over 200 years ago in Mary Shelley's Frankenstein. And interesting. Someone brought in religion. The creature in the book is never called a monster. It's not an inarticulate brute. He's learned to speak, presumably from a German translation in Miltonic language. You know, did I solicit the maker from darkness to promote me? And he wonders if now he is self aware, does he have an immortal soul? But I know of one scientist who works. I mean, he's a friend of mine. He worked on the Hadron Collider, and.

He sort of got a lot of.

Abuse because of this threat with the mini black holes. And I actually asked him, was there any risk of us all being sucked into Switzerland?

And he said, there are worse places.

But given the exponential advances in science over such great speed, a computer's Ability to have unlimited access to the Internet, the sum of human knowledge and ethics. Is it truly impossible for it to grow not in a self aware way as we understand it, but to grasp a balance of risk, probabilities, threats even to itself, especially if it is brought into theater of war to combat other computers, you know, lobbing ballistic missiles at each other?

Okay, so there's the idea of self awareness based on machine learning and then being able to act autonomously from us. Professor.

Wow, there is so much there. First of all, just going to Shelley's, Mary Shelley's wonderful novel, which I read as a kid, I do think that this kind of uncertainty about the role of technology and the emergence of robots and our place in the universe is hundreds of years old. It's not occasioned by the latest experiment done by our institute or by Facebook or anybody. And then to the point about what would happen in a theater of war or how do we think about the intelligence of the computers? It is very challenging because the only intelligent being that we're aware of at this time, although I believe there are others in other galaxies that'll require a whole nother.

Oh wow, you mentioned that. With six minutes to go.

Come on man, it's just my ploy to get involved.

Yeah, that's it, that's it. Good tease, good tease.

No, no, but there are other experts on that. But the point I want to make is we really have a hard time imagining how a truly intelligent computer will think. So I can't really speculate about that. It's just so far away. The analogy I like to give sometimes is imagine that electricity was just invented and we're starting to think about radios and maybe about different kinds of motors and so on, and somebody says, yes, but what will happen once we go from electricity to machines to nuclear power to potential pollution. And the only thing I can say is gosh, man, we just invented electricity. I don't know. So it's so hard to speculate about what will be true 50 to 100 years from now with this fast moving technology.

And I've failed miserably. Ray, thank you by the way, always brilliant stuff. I failed miserably to ask you about jobs, which I think if we're honest, is probably the most practical way in which AI is going to affect day to day life very, very quickly. Perhaps that's another reason to have you back on, Professor. But I did want to ask about warfare, the use of drones and the decision making power of robots to decide who to shoot at and when and with what.

So I think that it's very important to distinguish between intelligent weapons. And those are weapons that are highly targeted and potentially could reduce civilian casualties and autonomous weapons. And those are weapons fired with no human in the loop. And I completely agree with. I think it's terrifying to think that a computer could take the decision on its own to take a human life. I do want to point out though, that actually the strongest use of autonomous computing so far has actually been defensive. So the Israeli Iron Dome was shooting mortars and rockets and so on out of the sky. They were aimed at Israeli civilian population centers. So they actually saved a lot of lives. In short, autonomous defensive weapons are good. Autonomous offensive weapons are frightening.

How close do we. Do we know if people are building those?

We do know. Unfortunately, it's filtered out. We are in, by all accounts, a kind of arms race with increasingly powerful autonomous weaponry because nobody wants to be left behind. Right, the U.S. yeah.

And have there been. So are there examples of it? Have we seen the Chinese, you know, unveil the thing that makes you go, oh, my God, what the hell's this?

Well, one of the trickiest things is that with nuclear weapons, we could have treaties and we could monitor compliance with those treaties. Now a lot of this kind of development is happening in secret and we don't really know what our adversaries are doing. I don't really know what the US is doing in a clandestine fashion. And it's really easy to hide. So I'm less worried about something that's been unveiled and I'm a lot more worried about what hasn't been unveiled.

Whoops. Paul's in Wandsworth. Very quickly, Paul, we've got about two minutes.

Thanks ever so much, Tom. Great show as always. Yeah, just in a nutshell then. I'll try and summarize. I think the AI industry has us focused on all these things of can the computer obtain consciousness, all these sci fi sexy elements that we all buy into. I think it's a smoke screen in reality, how they are affecting us today. Well, in many factories, 90% of jobs have already been lost to work. The workforce to automation, machines, computers. I think AI is driven by greed. I don't think that's a good thing to be driving the development of anything or the world. And the World Economic Forum estimate that by 2025, 85 million more jobs will be lost to AI. I think AI today. How does it affect. I think it's dulling our life experience. You go to the shop these days, you're at the till buying your food. You can't even say hello to someone, you have to scan it yourself. I think it takes away from the human experience and it's driven by the greed at the top of very greedy organizations. How can I squeeze more profit out, get rid of more of the workforce, automate and make more money for our shareholders? So I think by 2025 it's believed that 50, 50 combination of humans versus machines doing jobs that can't be good for humanity and not at this rate. You might say, oh, you can retrain. I don't think most people will get to do that. And I think there's going to be a catastrophic cost and already has been to the human condition.

I think Paul just very finally it will be the speed of the replacement of those jobs that will be absolutely critical.

Final word Professor I do very much agree with this concern. I just want to point out about the benefits. So this was a one sided comment. If you're disabled and you need a more intelligent wheelchair, if you are autistic and need to be diagnosed more efficiently, if you have Covid and you need the next level of vaccine to fight Omicron, all those things, AI is on our side to help us. So there's good and there's bad.

Professor Oren Ezioni, Professor Emeritus of Computer Science at the University of Washington and CEO of the Allen Institute for Artificial Intelligence. You can download and subscribe to the session podcast on Global Player.