#256 From Deep Learning to SuperIntelligence | Terry Sejnowski, Head of Computational Neurobiology

We are in a period very similar to that of the Wright brothers 100 years ago. That is to say, we've just gotten off the ground and there's a lot of interesting issues and problems that we face to get to the point where we have a, you can trust AI and have a safe way of using it.

Hi, Terry. Thank you for joining me on the show.

Hi, Richie. Great to be here.

Since today, we're going to be talking a bit about the past of AI and also its future. I want to start with talking about where we are now. So what are the most exciting, cutting edge things you've seen from AI at the moment?

Well, AI seems to have something exciting happening every couple days, so it's hard to say which is the most exciting. But to put everything in perspective, we're going through this very rapid period of development and it's hard to predict where it's heading. Very hard to predict. But there's something that I think will put this in perspective, which is that I think we are in a period very similar to that of the Wright brothers 100 years ago. That is to say we've just gotten off the ground and there's a lot of interesting issues and problems that we face to get to the point where we have a. You can trust AI and have a safe way of using it. By the way, I read a biography of the Wright brothers. Very, very thoughtful. And one thing I learned was that their first flight, they crashed and burned many times. Right. But they did. The first flight reached a level of 10ft and traveled 100ft. Now, it took a long time for them to go from that to something that was controllable. That was the most difficult problem, was how do you make a reliable turn, for example. And AI is going through exactly the same kind of teething problems. Right. How do you make it reliable? How do you regulate it? This is all part of technology development that has happened over and over and over again. Okay.

I mean, that's quite a strong analogy saying it's like the Wright brothers moment where we had the first flight, because that was just such a huge moment for humanity. But at the same time, that also makes it sound kind of unimpressive. They only flew 10ft high and for 100ft and no turns. But yeah, I suppose it shows the power of what's to come, Right? Yeah. Exciting things in the future. But maybe we'll talk about the past first. So, I mean, artificial intelligence, I think the term's coined like 60 years ago. I know you haven't been working in the field quite that long. But can you talk us through some of the major developments that you think have got us to this point?

AI, when it was first developed, used computers that by today's standards are like watches, or actually, actually not even as powerful as watches. Right. They took whole rooms, vacuum tubes, and the only thing they could do well was logic. And so AI tried to develop the right programs, using logic and rules and symbol processing to try to replicate human intelligence. And I think it's fair to say that they vastly underestimated the complexity of that problem. It's so the real advance, in my mind, that made modern AI possible was learning. So in the early days, you had to write a program and take into account every possible complexity or detail that could possibly happen, which of course mushroomed combinatorically. But if you can learn from examples, then all you need to do is have more and more data. And so learning algorithms for neural networks were developed in the 80s which overcame some of the early limitations. And I was involved in that with Geoffrey Hinton, mid-80s. But what we didn't know back then was how well it would scale. And in computer science, the way that you classify algorithms in terms of their complexity is how does it scale with the size of the problem as the problem gets bigger and bigger and bigger. And it turned out that the neural networks scale beautifully. And in fact, they had to because of the fact that nature discovered that long ago that more neurons are better. Right. You have bigger brains, you have better capabilities.

Absolutely, yeah. So the switch from all those sort of expert systems to machine learning, that was absolutely huge. And, yeah, just having models that can learn has been obviously world changing. Okay. I'd also like to talk about the Neurips conference. Since you've been involved with Neurips, I think, for many years now, what sort of role do you think this conference has had in shaping the field of machine learning?

So the Neurips conference began in the 80s as a way to bring together people from many, many different fields, tremendous diversity, ranging from people interested in mathematics, engineers trying to solve difficult problems like computer vision, statisticians who are dealing with large problems and large data sets. And it really brought together an incredible number of different backgrounds, languages. And I have to say, during the first few years, it was very difficult because we didn't have a common language, we didn't have a way of talking to each other. The neuroscientists would get up and give a talk about the basal ganglia and everybody would just glaze over. The mathematicians would get up and Start putting up equations and neuroscientists eyes would glaze over. The only ones that actually got through to everybody were the engineers because they were very explicit about, here's the problem I have, I have to understand this image and here's how I'm going about it. But overall, over the last, I think we're having the 38th conference coming up in December, that community came together over decades and developed a real common interest in what now in retrospect is how you do computing in high dimensional spaces. High dimensions in terms of number of parameters, if you're a statistician, high dimensional in terms of the data sets that you need in order to be able to tweak those parameters. And like I say, nature was there ahead of us. Right. We have a million billion synapses. Those are parameters in our brains, and we use those every day to deal with the complexity of the world. And that's what it takes. And that's what it took.

Absolutely. So it is interesting how there's a lot of overlaps between natural intelligence and artificial intelligence, but you're saying there's like different groups of experts who just couldn't talk to each other for a very long time. So I'm glad that. Well, actually, has it got better? Like, are there AI people able to talk to the neuroscientists these days? Is there a better flow of communication?

There's a whole new field called NeuroAI. And next month I'll be going to a meeting sponsored by NIH. This is exploding. So back in the 20th century, the AI person would come to the neuroscientist and say, look, we know that intelligence is manipulating symbols. You should look for symbols of the brain. And the neuroscience say, what do they look like? There was no connection there. Well, now that we have a basic architecture which is very simplified version of the brain, it's far, far simpler in terms of the complexity of the units and the synapses. But nonetheless, the architecture is massively parallel. A lot of highly interconnected units and learning. Right. So the basic principles are there and now they can talk to each other. And so it's really exciting to see that happening because I'm actually right at the middle of that. My training was in physics. I worked with Geoff Hinton and got a feeling for how computer scientists looks at complexity. And then my career was in neuroscience and recently run the Brain Prize, which is the highest honor that a neuroscientist can get, and founded a new field called computational neuroscience. So now I'm right in the middle of this, because I can see both sides. And by the way, the transition that occurred in Neurips was machine learning. In other words, what developed was all kinds of algorithms over decades and I contributed to that too. Independent component analysis. But there were graphical models, support vector machines. It was really exciting because these were all algorithms that really were helping all these fields analyze their data and really was an explosion. Without data you can't do any learning. So as the data became available, we had the algorithms and we could really take advantage of them.

That's very cool. And certainly something like support vector machines is a standard tool for any machine learning scientist or machine learning engineer. And so the fact that's so commonplace and that kind of was born out of the whole Neurips thing. And yeah, that's pretty impressive stuff. One thing I'd like to talk about is it seems who is doing work in AI seem to have changed a bit. Like I think historically AI was very much an academic research field, but these days a lot of the big news is coming out of technology companies. Why do you think that's happened? Or in fact do you believe that's happened? That there's been a shift?

Yes and no. So it's absolutely true that in order to build a large language model that that's competitive, it's really expensive, it's like hundreds of millions of dollars, which of course no academic has access to. However, it turns out that a couple companies, and in particular Meta, has made an Open source their Llama 3 and Mistral, which is a startup company in France, has a very, very nice model that they've opened up. And so what academics are doing is using these open sour. What academics are really good at is analyzing and tinkering with and trying to understand the complexities of how these models work. So maybe I think it's an advantage that these big companies have taken over production of the actual models because it makes possible the next phase, which is going to be as we analyze these models and figure out how they work, we should be able to improve on them. Right? And it's already happening because there's a big movement going on not towards larger and larger language models, but small language models, the ones that are built from smaller databases and are more focused on a particular application, a particular, the database for a particular company, database of medicine and so forth. So what I see is that there's going to be a broadening in the field, there's going to be a tremendous number of niches where academics and small startup companies can work with enterprises to help them get into the game. And that's already happening. By the way, do you know how many startups there are in AI?

Oh, I lose count. Do you have any sense of what the number is?

It's 100,000 around the world. I mean, this is like Woodstock of AI. You know, everybody is out there.

That's pretty impressive. So last year I tried to make. Well, I made a cheat sheet just listing all the different startups in different AI fields. And it was weeks of research. And you know, a few weeks later it's like, well, it's out of date. There's just 10 times as many companies. Yeah. So yeah, the field is just incredibly vibrant. It's impressive stuff. Okay, so I do like the idea that you've got companies and academics collaborating and working on different things. It seems like it's important to have these sort of diverse opinions. So you mentioned the idea that language models are actually getting smaller sometimes. Do you want to talk me through what's the trade off? Like why you want very large language models or why you might want something smaller?

Like I say, it's all about data, what data you have access to. And the large language models have gone for omniscience. In other words, knowledge about everything. Right. And that's way beyond what any human is capable of. Right. We become experts in very narrow fields. Not every field, not everything. Right. I mean, for some people, they could master two fields, but that's very rare. So in a way, the large language model is really not a good way to go if you want to model a human brain. It's a good way to go if you're trying to create one solution fits all. But if you look at nature, it's really fascinating. Nature has not created one super intelligence. Even though humans think they are, they're not. But there's millions of different species out there that have all adapted into their niche. And in order to survive, you have to have intelligence. You have to be able to understand what's going on in your niche and you have to be able to reproduce and you have to really be autonomous. And I'll tell you, there is no large landsmill that is autonomous right now. It's like a big brain in a vat. Right. It just depends on humans for constant power, energy, constant programming, constant tweaking. Hundreds and hundreds of machine learning people hovering around these large bubbles, coaxing them to get better. That's not the way nature does it. Nature has a very, very sophisticated way. There are hundreds of areas in the brain, for example, that help Us intelligently deal with the world. Like the social world, for example, is very complex. And right now, the large language models are just modeling one part, which is the cerebral cortex, which is, of course, an important part for intelligence. But it's not the only important part.

Yeah, that's a fascinating analogy. So what are the other bits that are missing? What are the other parts of the brain that we're not modeling?

It would take me, you know, literally days to go through this. And by the way, I have a whole chapter in my book, you know, chat GDP and the future of AI on this specific topic is what is missing. But I'll just give you one example. Okay. Under the cortex, there is a structure which receives input for the entire cortex and then projects back to it. It's called the basal ganglia. Now, that's really. I mentioned this before, right? A confusing word. It's a Latin word. But really, it turns out we know what its function is. It's very interesting function. Here's what it does. It's important for a different kind of learning than you find in the cortex. It's called procedural learning. The declarative learning in the cortex is something you're aware of, consciously aware of. You're not aware at all of how the procedural learning system works. It's in the basal ganglia. And what it does is it learns sequences of actions that you need in order to be able to get future rewards. And it's used, for example, if you're learning how to play tennis well, you start out, you're uncoordinated. Gradually you start coordinating the muscles for the serve, and gradually you figure out how to hit the ball back. And that takes years and years and years of practice. And that is all done in the basal ganglia. Right? If you ask somebody, you know, tennis player, you know, anybody who plays tennis, you know, well, how did you hit the ball? Well, you just put your rack out there, you swing it, right? They have no clue. Same thing's true about musicians. They're just doing it automatically. It's all automatized by the basal ganglia. Now, it turns out that this is called reinforcement learning in AI and it has been used. It was used by AlphaGo, for example, to become the world Go champion. And it's all based on predicting future rewards. And what was amazing about AlphaGo was not necessarily just that it won the championship. It did it in a very creative way, in ways that no human had ever imagined. It just shocked KE the world Go champion, he. When he. When he wasn't even in the game, it wasn't like, you know, it was close or anything and he lost face. And in Asia, you know, that is a really bad thing because it wasn't just him, it was, you know, his all, you know, humans. He said that when AlphaGo won, it showed us that humans were wrong in thinking that we understood this game, that, you know, AlphaGo was much better than humans. And, and that took a lot of real self insight to see that had happened and not to give any excuses.

When you're at the top of your field and you're being beaten by AI, I'm sure it's gotta be a bit of a humbling moment. Okay, so you mentioned the idea of say, AI being incredibly creative. Do you have any sense of where AI is more creative than humans? Are there any sort of particular areas where this is true?

Oh, wow, okay. So very unexpected. So it turns out that these large language models are really good at things that are considered the highest forms of creativity in humans. And we're talking about things like writing, poetry, short stories. Some people call them hallucinations. But you know, one man's hallucination is another man's creative writing course. Right. So the other thing, I can give you many, many examples, by the way, I have a substack, if anybody's interested. The book was written a year ago, so it's a little out of date, obviously. So what I do is I take each chapter or little part and I update it. And so I think I'm on number nine now. There are short reads. But one of the most amazing things I came across was that large language models can be trained to become really, really good at cognitive therapy, psychotherapy. And you know, that means they have to have some form of empathy, right? They understand something about how humans have problems, anxiety or depression or loneliness and so forth. There are two amazing things that I discovered in probing this. Okay, the first one was that when they had humans that were given the choice either to go to a human therapist or to an AI therapist, they preferred the AI therapists. Now that really surprised me, you know, why would you do that? Well, in probing further, you know, here's why it turns out that they felt that they could be much more open and unburden themselves with an AI than with a human. Who's going to judge them? Oh, you know, this person is going to think I'm a real clot or worse. And secondly, if you want to get a human therapist, you have to make a reservation, you have to call, and two weeks from now, you're going to meet and you're going to have to pay $200 an hour or whatever. Even with insurance, the fact is that it's expensive because it's actually very time intensive for the doctor. So, well, if you have a problem, you press a button and there's your human therapist right there.

Yeah, it's a genius idea. And certainly therapy's incredibly expensive, so it's not accessible for most of the population. So having something that's, that's going to.

Happen over and over again, these are things that nobody imagined would happen, and they're beginning to happen. And regarding empathy, it turns out again in the book, I have this little section where a doctor was trying to have bedside manner with a friend who had cancer and he asked ChatGPT what he should say and ChatGPT gave him really, really good advice and he thanked. ChatGPT said, you thanked me so much and you helped me so much. And ChatGPT started consoling him, saying, oh, you are a very good friend and you did a great job. There you go with empathy. Right? I never would have guessed that.

Yeah, I guess when you're in the moment, it can be very difficult to work out what to say. And there have been thousands of conversations or well, probably millions of conversations on this in the past, by the way.

Doctors, my wife is a physician. They're given no training on empathy. Huh.

Okay, that seems like an important omission in the curriculum there.

Some doctors are good at it, naturally, and others aren't, but it's something that can be trained. It's just that it has to be done with a good instructor.

All right, so we've had a few examples now of AI being better than humans. And I guess a lot of the prominent AI startups have expressed a desire to create artificial general intelligence, which I guess the whole idea is that AI is better than humans everywhere. But first of all, do you want to jump into. What does AGI mean to you?

It's interesting because I have not seen a good definition of what it is. So whatever, it's like pornography, I guess. It's, you know, you know it when you see it. You know, it's like. And by the way, one of the things that's been revealed by these large language models is that experts don't know what they're talking about when they use words like understanding intelligence. They can't. For example, there's a big rift in the community. Do these large language models understand language? And now half of them say yes and half of them say no. I Mean this is. How could that be? They're experts, right? Well, it shows you that the word intelligence is not very well defined and some people emphasize some aspects and others other. So, you know, it's not really, I think, a good goal. In fact, okay, so again I have a chapter on this, so you can go and check it out. But there's so many things that are missing that. Like for example, autonomy, right? I mentioned, I brought that up. That's part of intelligence, being autonomous, making decisions in the world that allow you to survive and to thrive and reproduce. And that's not really taken into account at all. Even if you build a robot, it's not going to be autonomous, it's being programmed in some way. And so here, I'll just give you an example. One example, okay, two examples. These large danglers models don't have goals, right? Well, humans have goals. Survival, right? We have a goal. You know, you're hungry, you eat. We have the world built in and you know, self protection, all of these things are going to have to be added, right? If you want to have AGI of some sort of. Okay, that's one thing. Okay, here's another thing. So what happens when you're all by yourself and you're thinking, you're planning ahead, what you're going to do the next day, right? Well, okay, well that's called self generative function of the brain. It is self generating thoughts. What happens when you turn a large language model? You eliminate the input to a large language model. What happens? Nothing, Nothing, just stop. No self generative thinking at all, right? If you want to have intelligence, general intelligence, at least it should have, is some of the self reflection, self thought planning, right? There is none there.

These are very interesting ideas. It feels like they're going to have big consequences. So what happens if you make AI completely autonomous?

It's going to get a lot cheaper, right? It'll run itself. You don't have to cobble it. And it's going to have to have much less it take much less energy. So you're going to need a much more advanced technology. And that will happen. That's how all technologies develop is that they get cheaper and cheaper and cheaper. And we already have the technology. The fundamentals were actually introduced by carver Mead in 1980s. That's when I was getting off the ground with my career. And it's called neuromorphic engineering. And what it does is it turns out that the biophysics of neurons, the ions going in and out and action potentials that you can replicate all of that in silicon at threshold. At threshold, it's very, very low power. And you can cram billions and billions of these very simple biophysical circuits that only take milliwatts. Right. Not hundreds of watts, but milliwatts. So eventually, that is going to be the way AI is delivered to the edge devices in the world. And at that point, you will then be able to have autonomous creatures that actually do have a little bit more going for them in terms of onboard processing. Okay.

Yeah. I can certainly see how you want much lower power requirements in order to have, like, something that can be autonomous, because it can't just run in a data center at that point.

All right.

Are there any ethical implications to creating this sort of AGI?

You bet. Okay. You know, there is a tremendous amount of. So I'm the president of the Neural Information Processing Systems foundation that runs the biggest AI meeting, and it's going to be in December. It's going to be in Vancouver. And I'll tell you, we have. There's a lot of people whose careers now are all about AI ethics. Right? This has, like, become a big hot topic. And there's a lot of things that we have to worry about. We're worried about bias, we're worried about fairness. We're worried about making sure that hallucinations don't go awry in terms of how people use it and so forth. And all of those are very important ethical questions. And also how it is going to affect society in terms of copyright laws and things like that. I mean, it's a tremendous. There's a tremendous amount of regulation that's going to have to be done. So all of that is really important. However, when you come right down to it, the ethics of creating an AI is actually no more complex than the ethics of creating a baby. Right. Baby is an intelligence that can go either way. Right. You can have babies that become extremely useful and to devote their lives to society in many different ways. And then there are people, some babies that go off in a bad direction. They become criminals, they become mass murderers. My God. Humans. Ethics that we apply to humans have to be applied to AI. Right? That's going to be absolutely necessary. And that's beginning to happen. Regulations. People are talking about regulations. The trouble is, it's a moving target. We don't know what to regulate or how to regulate. If it's changing quickly, then things are happening. In fact, the European law, AI law that went into effect is already completely obsolete because we can do things now that are. It was done two, three years ago when they, like 100 pages of laws. And you can't use large language models for looking at job applicants. Right. I mean, that's silly. You should be thinking about how to regulate the large language model so that it's fair. One way is to put that into the cost function, the loss function, so that you make fairness, you know, give it a value. What's the value compared to getting your best candidate? Right. It may not be that you don't want the best candidate, period, because he may be. Or she might be really disruptive, you know, but, you know, if, but, you know, other things that are important that need to be weighted equally. So, you know, this is. We're all at the very beginning. This is the Wright brothers, you know, we're just trying to figure out how this, how to control this contraption. Okay.

Yeah. Certainly a bit of a wild west situation on a lot of these ethical issues. So you mentioned the idea of an AI being like a baby, certainly once you have this artificial general intelligence. So what's the equivalent of a parent then or family to this AI baby?

There isn't. Right now, unfortunately, these are orphans. These poor programs are out there without any guidance. Right. And so how is it that we bring up our children? Well, we bring up our children and every culture trains the children, first of all in the language of that culture, obviously, but also in the values of that culture and, you know, what the goals are and so forth. And that takes many, many, many years of reinforcement learning, what I mentioned earlier. In other words, reinforcement learning is absolutely essential for creating AIs that are aligned with us. Right. Otherwise they're just freely floating. They would have whatever culture they want. Right. I mean, this is how we do it with our own children. And interestingly, that's already beginning to happen. So there's the most recent wave of. I think it's Google, who came out with a version of their large language model, which was trained with reinforcement learning to do sequential reasoning. In other words, when mathematician solves a problem, they have lemmas, and then they have intermediate steps and so forth, and that's all. Each one is simpler than the problem. But when you put it all together, you solve the problem. And so it turns out that mathematicians, okay, they're not born to think that way. They have to have years and years and years of training to become a mathematician who can solve equations with proofs they're valid. Right. Well, that's reinforcement learning. It's just like learning how to play tennis. And so you're going to have to Go through that same process if you want to a large language model to be able to do sequential thinking. Right. In terms of breaking a problem down into bits and solve each one separately, then you're going to have to train it to do that. And that happened. But that's just one example. So reinforcement learning has to be integrated in from the very beginning while you're training the model and not at the very end. And that's really where things are headed.

Okay. Yeah, certainly that sort of chain of thought idea where it's breaking down problems into smaller problems and doing sequential reasoning, that seems incredibly powerful. And it's one of the sort of cutting edge things. With these large language models. Do you have a sense of, are there any other techniques like that that are going to sort of help push us towards AGI? Like do our existing methods scale, Are they going to continue to scale to more powerful models and better reasoning, or do we need new techniques like that?

Okay, like I said, there's a hundred brain circuits that are involved in regulating our behavior. And I'll give you one more example. And this is incredibly important for survival, but it's also important for higher brain function. So we have these connections between neurons. They're called synapses. And you know, some are excitatory, positive, some are inhibitory, negative. And we have weights that are positive and negative, just like in the brain. However, in addition, in the brain we have what are called neuromodulators. What's a neuromodulator? Well, neuromodulator itself is not the main signal. What it's doing is, that is to say excitatory, inhibitory. Its main purpose is to regulate the other signals, to amplify them, suppress them. And it does that in a global way. A single neuron, like for example, in the basal forebrain, there's these cholinergic neurons. This is all gobbledygook, right. But there are these neurons with a particular transmitter that then go throughout the cortical mantle, and those are important for attention. There's another set of neurons in the brainstem called dopamine neurons, which project broadly into the basal ganglia and the cortex, and that regulates learning. Now why are these two things important? Well, attention is important because you don't remember something, you don't attend, first of all. So it's really important for long term memory. And the dopamine signal is important for your procedural learning in the basal ganglia. Right. So those two neuroto signals are not found in any large language model. Right. That's a huge control structure that the brain has. And by the way, there are dozens and dozens of neuromodulators, and they're important for social cognition. For example, serotonin. You know, look, there's so much that's left out of these large language models that, you know, we're just at the beginning. This is Wright brothers here, right? Just getting off the ground.

That is an intriguing idea. The idea of having AI that has a dopamine circuit or a serotonin circuit and gets joy from certain activities.

Yeah, why not? You know, why can't large language models have joy? Right. The emotions should be part of the organizing principle of the way that we create intelligence. What I'm saying is multidimensional intelligence is a spectrum in terms of. In someday. Okay, here's a prediction from the book. Every object that you deal with every day, not just your computer, but a clock. You're going to be able to talk to your clock. Now, you don't need a lot of intelligence to be a clock, but, you know, set my alarm for 7am tomorrow morning. Got it, Roger. So. And we're going to have a spectrum of intelligence which is going to be used in every aspect of our lives. And every, you know, every company, when you call up for information, you know, a company, you can ask any question and then you'll get answers that are much better than we're getting now. We still have these trees. You know, you call the company and it says, and it gives you five options and you go down the tree and at the very end, you know, you've wasted five minutes, you haven't gotten a human. This is like, this is so, you know, retro. I mean, it's really. Now that we have. Okay, here's another prediction. Okay. When's the last time you used a typewriter?

Oh, man. Decades.

Many decades. There are museums. Right, but you're still using a keyboard.

That's true.

Why? Well, it's because the only way we can communicate with a computer is through our keyboard. That's going to change. You're going to talk to your computer, your computer is going to talk back. Right. You won't need a keyboard.

Okay, so that's one prediction I'm not so sure about. Because, well, if you're in an open plan office, talking to your computer is going to be awful for your colleagues. But otherwise. Yeah, I agree there are definitely some uses for it.

Well, you know, this is all. This is a prediction that is, you know, it's not going to be. It's not going to happen, just like I said it is. But there will be a way to communicate that is going to be personal. Some technology will be developed to do that. Absolutely certain of that?

Yes, Definitely, yes. Certainly personalization is going to be a bigger thing in the near future. Do you have any examples of how personalization might be useful?

Yes, I think that in fact, this is something I was asked a question after a lecture. You know, what's the killer app in AI that's like Lotus was for personal computers. Right. Spreadsheets. And I hadn't thought through, but now it's pretty clear that the killer app for AI is going to be education. So it's well known that the best way to educate a child is to have a tutorial who understands that child personally and can help it get over mental blocks. Or specifically, the child is having trouble with something. But it's very, very costly. There's human labor involved in that, which is very costly. However, if you have an AI that actually can keep track of that kid and give it advice and serve as a mentor. Wow. Every child in the world, every child, no matter whether you're here in the US and you're wealthy or whether you're dirt poor in Africa, will have access to a personal tutor which will help them navigate through life, teach them things, help give them advice. I think that that is going to be ultimately the way that we're going to be able to improve society and get out of this endless cycle of having wars and terrible, terrible society, problems with homelessness and so forth. I mean, this is really, I think, the future if we get there. Right. It's not clear.

Absolutely. Yeah. The use cases for AI in education are pretty amazing. I mean, obviously this is very dear to our hearts at data camp since we do work AI training and data training. Yeah. And there are limits to what you can do at the moment because it's like, well, most of our examples around things that a lot of people care about, business use cases. But if you care about, like, if you're super into, like frogs or something, then we can't do a course on data analysis for frogs because there's just not much of a market for it. But if you create it cheap AI, then it's possible.

Yeah.

Yeah. Okay. So to come back to the theme of autonomy, one of the big sort of hyped use cases recently has been around AI agents. So how do you see AI agents taking off? Because they're only in their very early stages at the moment.

A lot of our thinking has been formed by science fiction movies. And the average person thinks that the future is going to be like the Terminator. Right. I can assure you that that's not in the near future. But there is a science fiction movie out there which I think is really right on in terms of where we are right now, where we're going to go in the near future. And that is her. And I don't know if you've seen that movie.

I have seen that movie, but do you want to explain it briefly to the audience?

Okay, well, her is Joachim Phoenix has a job in the future and he's very lonely. He's all by himself in his little cubicle apartment. And he signs up for a personal assistant, like an agent. Turns out to have the voice of Scarlett Johansson. Right. So that was a good choice. But he can talk to her and over time they become friends. It really helps him become out of his shell. And he's really doing very well now and much better than he was before. And suddenly she goes silent. And then it turns out that she's been seeing other humans. Like she has a thousand humans. Obviously not two timing him, but a thousand timing him, which caused him pause. But then comes. I'm not going to tell you how it ends, except to say that in a way it gives you a sense of what an agent might look like, how it could help humans and other ways that things might go awry. I thought it was a fabulous movie just in terms of the way it was acted. It was very, very low. It wasn't like there's any action. Nobody was killed. It was. I really. There are other good science fiction movies out there, but I think that one's the closest. Yeah.

So I like the ideas, but. Well, it becomes a companion and assistant and all sorts of things. So you're actually developing a personal relationship with the AI, which again is slightly controversial, I think. Do you have a take on human AI companionship?

Oh, you know, it's already happening. I don't know if you've been following me. There's an Israeli companies has a personalized help robot for elderly called leq. And it actually is. The technology is not that advanced. But the point though is that for someone who's older, it keeps track of them. It knows whether they're turning lights on at night, whether they're. They have pain and they can report back to the family or the doctor. You know, that's already out there. In fact, this is like going back 20 years ago, Japan came out with a little toy dog companion called Aeibo, a I B o and people loved it, you know, even though it was all it could do was follow a ball and follow them around. You know, it doesn't take a lot for humans to adopt pets and AI will be adopted as a pet. So that's something that's already happening. But what we really, I think, should be concerned about is what impact it's going to have on humans in the long term. And we don't know. We just don't know. You don't know until you actually get in there and get the data, do the experiment.

Absolutely, yeah. And with the AIBO dog, I think, yeah, dogs are amazing. But also having a dog that you don't necessarily have to walk every day, probably very convenient.

Yeah.

Okay, so you mentioned that we don't really know about the long term implications of AI. Certainly wants to get to like very powerful AI. So you touched on the idea of aligning AI earlier and once you get super intelligence, that becomes very important. Do you want to talk me through how alignment works and what aligning a superintelligence would involve?

Well, first of all, I think the term superintelligence is a really bad term because it has all these implications for eliminating humans. This is existential threat. That's what people think when they hear the word superintelligence. And look, What we really should be thinking about is how to, as you say, align these models, these large language models to make them compatible with us. And I've already given an example the way that we bring up children and we just have to basically adopt these AI models as if they were children. They have to interact with us and learn as they're interacting. Right now you train a big model for $100 million, you throw it out there and it's supposed to one size fits all. But really you want the large language model to be specialized for particular niches, right? So you're going to have to bring it up in that niche the way you would a child so that it becomes really good at that particular area of data of these cases. And so I wouldn't call that super intelligence, I would just call it diverse intelligence. There's going to be a great diversity of intelligences that are trained once you, once the companies, and this is not the big companies, these are going to be startups. The startups are already doing this, right. They realize that they're not going to become metas or open AIs, but what they can become is a really important second tier that is delivering AI to the, you know, companies. By the way, it's really interesting. Internet did not really have much impact on the internal structure of companies. It had a huge impact on people outside using companies and, you know, interacting with the rest of the world, every aspect of our lives, you know, banking and social media and so forth. But it didn't really change the way businesses work. This is going to be different. Businesses are going to be completely transformed by this technology. Right now if somebody asks a question about IBM, you know, this quarter, where's the money going and so forth, it may take a week to answer that because there's the dozens of databases and you got to query them and you got to SQL. It's a nightmare. I don't know how they managed to do it. I suspect they don't. And nobody there knows what's going on. But if you have an AI that knows all the databases and instantly can tell you the answer to your question, just think how much more, you know, how much better the decisions are going to be made and how much more can be done. So here's my mantra, which is that AI is going to make you smarter, whether you're a person or a company or a society. And you know, people are worried about losing their jobs, right? And fair enough, there are jobs that are going to be obsoleted, but most jobs, you're not going to lose your job. The job is going to change. And that means you have to change. You're going to have to use AI. You have to learn how to use AI. And by the way, young kids coming up have no trouble doing that. They're already using it for school and for other things. Right? So that's going to be the next generation coming up. They're going to be the ones that are going to take over.

Absolutely, yeah. Certainly if you're learning all this technology when you're very young, it's a very different proposition to, well, I'm middle aged trying to learn this stuff. So yeah, I can certainly see how the next generation is going to benefit from being AI native. Okay. So we started that on angle superintelligence. And one of the big sort of risks is, I mean the, the problems from superintelligence. People have talked about things right up to extinction of humans and things like that. And so the probability of this happening is widely known as like the P doom number. Now your collaborator Geoffrey Hinton, where he asked like, what's the probability of superintelligence destroying the world? He put it at 0.5. So 50, 50 chance of superintelligence murdering us all. Do you have a P doom number.

Yeah, well, first of all, I'm not worried about super intelligence, but I'm glad that Geoff Hinton and others are worried about it because I've got better things to worry about that are more the problems I'm working on right now of trying to figure out. I've told you about self generative brains. We don't know how the human brain self generates thoughts. We don't know. So that's the problem I'm working on. And putting a number on an imponderable like that is a fool's game. And if you ask around, I mean the numbers are all over the place and that means there's not enough data to make an intelligent decision. And so like I say, I will be agnostic about the possibility. I think that it's in the short term, I think it's very unlikely in long term, who knows? All I can say is I hope that they put an off switch into it.

Whenever.

Engineers are smart enough to do that, then I think we're safe. But any case, I'm sorry I don't have a number for you, but I think it's between 0 and 1. Okay.

Very tactfully. That's a good statistician's answer. Okay. Yeah. So the idea of an AUSPIC has actually come up quite topically recently with the California AI safety bill. There's one of the rules in there that if you have a particularly powerful AI, then there has to be an off switch which turns out to be very controversial part of the bill, I think.

Well, you know, Newsom turned it down. Really? Okay, yeah, he turned it down and I think he was lobbied by the big high tech companies.

All right, we're not getting enough switch then. That's a shame.

Well, we'll have wait and see about that. But no, I think that it's premature regulation. It can often be a real danger and I'll tell you why. We might want to turn it off here in the US but they're not going to turn it off in China. In other parts of the world that where they're bad actors, you don't want to be behind, you want to be ahead of them.

All right, what do you think the consequences of the competition between different countries.

Are then for AI like any technology. And here I look at the way that we've handled similar existential threats. Bioengineering, for example, to create viruses that are deadly. We have regulations for that. And all the countries have realized that it's not in their best interest to create these bioweapons and somehow it's been contained. Same thing with nuclear weapons. Right? It has to be international agreements that are made, and everybody realizes that if the war starts, we're both wiped out. So it's not worth building up beyond a certain point. So I think the same thing will happen as the countries, each in its own way, develop something and uses it in different ways. And then it becomes clear that this is going to be. Sometime, you know, an existential threat in terms of being able to wipe out the other group, you know, through cyber security or whatever, you know, breaching cybersecurity. You know, I'm an optimist, even though I probably don't sound like it. I think that humans, on the whole are. Even though it's. We make a lot of mistakes and a lot of decisions are really terrible ones, I think that on the whole, we have survived this long, and I think that if we continue even making bad decisions, but at least ones that are going to be controllable, I think that it'll be safe, and that's my hope.

Okay, that actually feels like a good note to finish on. It's like we've done pretty well at surviving for the last however many tens of thousands of years, and, yeah, hopefully will continue to do so. Excellent. All right, thank you so much for your time, Terry.

Oh, Richie, it was great. Thanks for touching all these interesting topics. They're all in my book, so go out and pre order from Amazon. It's coming out October 29th.

All right. Yes. Well worth a read, I have to say.

Thanks. Very good. Great.