Leslie Valiant : The Importance of Being Educable

Foreign.

Kawasaki. And this is remarkable, people. We are on a mission to make you remarkable. Helping me in this episode is Leslie Valiant. He is a professor of computer science and applied mathematics at Harvard. Valiant received a BA in mathematics from King's College, a Diploma of Imperial College from Imperial College. That makes Sense. And a PhD in Computer Science from the University of Warwick. Prior to Joining Harvard in 1982, he held faculty positions at Carnegie Mellon University, the University of Leeds, and the University of Edinburgh. Valiant has made foundational remarkable contributions to computer science, including the theory of probably approximately correct learning, the concept of P completeness and complexity theory, and the bulk synchronous parallel processing model. And if you believe I understand all of what I just said, you are in for a big surprise. Valiant is the recipient of the Turing Award, the highest honor in computer science. The award committee said this about him. Rarely does one see such a striking combination of depth and breadth as in Valiant's work. He is truly a heroic figure in theoretical computer science and a role model for his courage and creativity in addressing some of the deepest unsolved problems in science. Valiant's book, the Importance of Being Educable explores the unique ability of humans to absorb and apply knowledge. He argues that understanding our educability is crucial for safeguarding our future, especially with the rise of artificial intelligence systems. I'm Guy Kawasaki. This is remarkable, people. And now here is the remarkable Leslie Valiant. What exactly is wrong with the term intelligence?

It is that we don't know what it means. It's got no definition. No one can tell you exactly how to recognize an intelligent person. What's the behavior you're supposed to look for? 1 may be a trivial symptom of this, which may not be that important, but just as a symptom. So there's much discussion about the SAT tests, which many American students do, and I was surprised to find out recently what the letter A stands for. So certainly S stands for standard and T for test. But what does the A stand for? So apparently the A used to stand for aptitude. Standard aptitude test. They changed that. Then it was standard assessment Test. They changed that, Then it was SAT reasoning, they changed that. So now officially, the A stands for nothing, just a brand. So whatever it tests for, the society is buying without any label on what it's meant to do. So if you buy food in a store, wow. It says more than that. There's a correlation between you being less hungry afterwards and eating this. There's a correlation between these SAT tests and something. And that's all that's promised. So the problem with intelligence is that it's widely used, but it's got no definition.

Isn't that fixable? Can't we get MacArthur Fellows and Turing Award winners and all you guys together say, can you just please define intelligence? How hard could that be?

So in the book, I quote some reports of the American Psychological association some time ago where they report on some other groups saying that they asked 24 experts in intelligence and they all gave somewhat different answers. So there's some concepts for which we have a word, but we don't really know what it means.

Seeing as how you're saying that there's no sort of agreed upon, consistent definition of the word, how are we seemingly freely talking about AI so much right now? How can we talk about artificial intelligence if we don't even know what intelligence is?

Exactly. So, in fact, I think using word intelligence didn't help the AI field to progress because there was no goal there. In my view, it progressed because AI adopted a more specialized definition, or rather used one. And this is learning. So learning is something which you can define much better. So learning from examples, after you've seen some examples, you can classify future examples better than you could before. So learning from examples is something. It's a very specific ability. It can be defined. The computer programs should do quite well. And in practice, these large language models do exactly that. Okay, so there's some evidence that they do something interesting. So AI, in my view, has progressed by trying to realize by computer tasks for which there is a definition.

In a sense, what you're saying is that intelligence is such a quagmire that you're just going to go outside the box and you're going to create this new standard.

I'm not saying I want to create a new standard or that's not how I start. I'm saying how I phrase it in the book is that in some tent. It's a very old question, of course, is what's the intellectual capability which humans have, which other species on Earth do not, and the capability which enabled us to create our complex civilization, science and everything else. And although this may seem overly ambitious question, in some sense, it's not. Because in evolutionary terms, we evolved pretty fast. In the short term, this must have evolved. So I'm trying to write down this into this capability. And the first part of this is exactly what I was describing with learning from examples. So it's something which we understand well in computers. Everyone sees how it works with these large language models. In the last year or so, and then I add a few things on. So I think I'm doing natural science. I'm trying to explain human capabilities. And I think it's something real which has a definition. Whether it turns out to be useful, that remains to be seen. In some sense it is a replacement for intelligence, but in a special way.

So I'm curious that in the creation of your book and the title and really the writing of the entire thing, did you ever think about the difference between educatable versus educable? Did you ever think of the nuances between those two words? Or now in modern use they're equivalent, so it doesn't matter.

I'm not quite sure how you mean any of those. So at one point these terms were used in the sense of yes, no, so which children could be educated in a regular school as opposed to having to be sent out. So I think that's the use these terms had maybe 50 years ago. And maybe that's the way in which you mean educatable. Right. But what I'm trying to capture is to define more, ground up the capability which I think all of us have to some extent. And I'm not trying to distinguish whether some of us have more of it than others, just a general issue. We're trying to distinguish us from other species. So it's not to be confused with the other senses in which this word may have been used in the past, like it's new technical term, if you like.

Okay, now, from your point of view, is this level of educableness, if that's the word, is it a person's or a machine's innate ability, or is it also subject to the methods and resources and environments around that person? Or is it just the person, so.

The capability itself somewhere there at the beginning, the question is whether it can be enhanced by some sort of life experience. I don't know. Okay, so I think that's a good subject for investigation. Certainly if you can enhance the rate at which you can learn, for example, that would be fantastic for society. But I think that's a subject for research. I should add that most capabilities can be enhanced by some sort of training and the right environment. So the default assumption has to be that this also can benefit from suitable interventions.

But I don't know, you're saying theoretically, if we could do this controlled experiment and we took you, or we took Stephen Wolfram, or we took Neil Degrasse Tyson, or, I don't know, Freeman Dyson or something, and we stuck you in the middle of the Amazon jungle with no Internet, no books, no nothing. Clearly you're still the same person. If we go back 20 years later, are you running the jungle? Are you the leaders there? How much did where you are in modern society play into the fact that you're such overachievers?

Well, first, I'm not equating. So let's suppose that you can measure educability. Different people have different levels of educability. Let us assume that. I'm not even certain of that, but let's assume that what it correlates with in society, again, I don't know. It's not obvious. The most educable people may not be the ones who are most successful in politics or academia or whatever podcasted. And I'm not sure I don't want to speculate, but it would be if you put someone in the jungle, then educability would cause one to how fast they can learn what's going on there relative to what they know. Obviously it may be that they know so little that's relevant to how to survive there that the educability isn't enough to survive. Okay, measure of how fast you can learn and understand what's going on.

If I had to put my money on you or Stephen Wolfram to figure out curare before the random person in the jungle, I think that's a pretty good bet.

So I don't want to speculate, but certainly there's some people, it's maybe the entrepreneurs who are very good at picking up lots of information, sifting through lots of information, understanding what's going on fast. Some are quite sure which profession exhibits educability the most. For example, scientists obviously need need it. But they may not be the most extreme because they're also good at concentrating for a long time to pursue a direction. You could be maybe more where you keep finding out new things all the time and relating everything to each other and running with.

Are you familiar with the work of Carol Dweck and the growth mindset?

No.

She's a professor at Stanford and she in the early 2000s, pioneered this dichotomy between the growth mindset and the fixed mindset. The fixed mindset basically says that you think you are what you are, you cannot be anymore, you cannot learn new skills. It also means you think you cannot deteriorate. So if you're a genius, you believe you'll always be a genius. And Carol Dwecks the that the growth mindset means you can learn things, you can do things. And it seems to me there's a great deal of parallel between your theory of educable and the growth mindset. So I thought that maybe great minds thought alike and you knew each other.

No, but I mean that theory may be more about one's attitude but I think this educability is something which kind of everyone has. So the fact that most people are gathering information all the time, they're watching movies, for example, they're reading novels, they're looking at their cell phones, they're all soaking in information. Much of the information is of no relevance to them, it's got no benefit to them. But that's what they do, they soak in information. So I think it's this distinction whether you regard your ability to soak in information, whether you're going to do it usefully to yourself to benefit yourself to find a new profession. I think that's not exactly the same thing. I think what I'm describing is something which we freely all have quite generously.

I think in your book you mentioned this example of chimps escaping from their cage in a zoo using a fallen tree to jump over the wall. And then the other chimps watched that and learned and also jumped over the wall. And so are you saying then that in this case these animals are educable? Is that the conclusion to draw? That they passed the test for educable?

No, I'm saying that they look pretty smart and they can solve problems and they can obviously the first one to jump over the wall figured out what to do, assuming that hadn't seen something like that being done and the rest could copy. But so by educable described in great length in the book. So it's a combination of learning from experience, being able to chain together what you've learned and some kind of reasoning. And then the third part is being able to learn from someone else describing to explicitly what should be done. So in some sense copying behavior is like that. But in humans it's much more general that we sit in. If you go to college, you sit in a lecture room and someone tells you the laws of quantum mechanics and they sit there for 30 lectures and at the end you can do a lot of stuff. So I think it's only humans who have this ability to be able to soak in a lot of information given explicitly. So it's like other people having had the experience, other people having done the experiments, but they can tell you the conclusions and you can internalize it yourself and use it as if you had the experience yourself.

Have you made observations yet about the presence of educableness? In life, is it normally distributed, is it distributed differently to genders? Does your educableness, does it grow or decay chronologically through life? Have we have some longitudinal studies and cross cultural studies, cross gender studies to understand more about this concept?

No, we haven't had any studies. It's a new concept. We haven't had any studies. So to answer any of the questions, you'd have to develop some test which you possibly think does measure educability. And these tests would have to be of the nature that you're like. If you do a one hour test, in principle, you could have a test which is like, like a one semester course, or you could have a one hour test, but what you'd be measuring is how much you've learned during that one hour. So you're given some questions where it won't help you to have previous knowledge to answer the questions. So it's related to current IQ tests, but it would be, have a, have a different emphasis. It's a new kind of question. But the only later point I'd make is that if one accept this notion that what Kascha has assumed is its educability, which means an extreme ability to learn and soak up information, then this also illustrates why it's very difficult to answer the questions you ask. I mean, to try to prove differences between groups in abilities and other abilities is difficult just because if you make some measurement of the performance of a group today, somewhere else, at a different time, the outcome may be different because the group may have had different experiences. So we are being so subject, so extremely subject to outside influences throughout educability that we're the least promising objects of scientific study, if you like, by doing surveys on humans in different places, you shouldn't be surprised that these results aren't transferable, generalizable, from different places, different times. Just because we're so prone to change. The point of education is that we're so prone to change, so prone to be influenced by our environment that it's very hard to answer any of the questions you ask.

If you think about how would you possibly control all the variables in an experiment like that, Right? Although I gotta tell you, if I had to guess, I would tell you that women are more educable than men. There's no doubt in my mind, but I don't have any objective proof for that, just my life experiences. So if we're at such like the starting point of all of this, let's say somebody's listening to this and say, yeah, you know what? Really I Want myself, I want my kids. I want my company to be more educable. What can I do now? Give me some tactical and practical stuff. Leslie, help me out here.

I don't think I can. I'm sorry. I think the only rational thing is to do research and develop some tests which plausibly measure this kind of thing. So if we can't measure it, then there's not much we can say about it. So I think it's a long term project. The question of whether we can improve our educability. I think it's a long term project. Of course, many people have ideas about learning to learn and teaching to learn. But to validate any of these things is difficult without a measure of when you can declare yourself to be successful in having enhanced learning.

If you were to just take a social welfare perspective and step back, it's hard to imagine more things that could be more important than this. This could be the key to the survival of mankind to figure out how to make people more educable.

Personally, I agree. I do. So I think that's. Yeah, yeah, I think exactly. I think it's a public issue for discussion.

Listen, let me give you some shallow thinking. Let me see if this metaphor will work for you. Is it fair to say that intelligence, even though we can't really define it, intelligence is like the chip speed and educableness is programmability. Is that a fair statement?

No, I don't think so. I think educability is. If supposing. Let's talk about a computer rather than a human is like a discussion of what the capabilities are of this thing. And intelligence, I don't know what that is. I really don't know what intelligence is. Some people say intelligence is what a standard intelligence test measures. Okay. Which is a bit circular, but again, I don't quite know what that is. Many cognitive capabilities are correlated. If you're good at one thing, often you're good at something else. Often a weak correlation, strong correlation. So these intelligence tests are just something maybe arbitrary which is correlated with many things with. So colleges may want to find this out because it may be correlated with how well students do, but many other tests would also. So I really don't know what intelligence is. I'm personally, I'm really so I don't think I go around and saying, oh this person, my neighbor's intelligent. I don't think I would naturally say that.

Orthogonal to this is. Is there no concept that we need to take in account something like morality.

Right.

So that morality you could be the most educable person in the world. Not that I believe this, but let's just suppose for a second that we think Donald Trump is educable, but I would say he has zero morality. Right. You, you could take an educable person who could learn from others and extrapolate and do all these great things. But what if that person is fundamentally evil? Then what?

Then it's unfortunate. But certainly educability only defines exactly, as we say, one's basic cognitive capabilities. And it's about how I face in the book that it's the knowledge I talk about beliefs that you learn. Beliefs, other people's beliefs, they tell you their beliefs. And so in that sense, the theory is about capabilities. So in one sense it's morally totally neutral. It doesn't discuss morality because as you say, the beliefs could be good or could be bad. But what one's reaction to this should be is it shouldn't be moral neutrality. As you say, some people have different beliefs and some we think are evil and some we think are good. And just because our capabilities are neutral to morality, it doesn't mean that we should be. We should still certainly fight for what you believe is right and against things which you believe are evil. But the fact that because our basic capability is kind of neutral on morals, totally bad beliefs can spread through society, we don't seem to have any good defense against that.

If people are listening to this podcast and they're struggling with this educable concept, do you have any people that might be well known in the valiant hall of fame of educableness that you can say, oh, I understand that now he cites this person as educable. Is there anybody like that you can say, think of him or think of her? When you think about educability, you mean.

Someone who shows educability in the extreme? Yeah.

Who's your hero? Who's in your hall of fame of educableness?

I don't know. But I sincerely believe that this is something which we all have. I'm not trying to define something which will separate us. I'm really trying to define something which we all have and which unifies us. So I think the educability is very important, otherwise I wouldn't have spent so much time on it. But I don't know, as we said before, what this correlates with exactly what human performance correlates best with what kind of people exhibited the most. Yeah. So at this point, I just don't have a good answer to your question. It's a natural Question, but it's not the direction in which I'm looking.

Okay, if we can shift gears slightly towards artificial intelligence. I know I just used the I word, but that's what everybody use, right? At this point. When people say artificial intelligence, do you think it means that it refers to what a machine can do like a human, or what a machine cannot do like a human? Like what is artificial anymore?

I think the meaning of the term has changed in history, but at the moment in the media, it clearly means the kind of things which current machine systems can do. Okay, so by AI, people seem to mean large language models and things for which you can download some software which they can do. So it's in the era of machine capabilities in manipulating language pictures. So I think that's what it means now in the media.

So it's artificial in the sense that it's a machine doing what humans can do and that's what makes it artificial?

Yeah, the artificial part always means that it's machines doing it.

Now, in your opinion, is ChatGPT educable?

No, no. Basically ChatGPT does the one of the three requirements of educability, which is learning from examples. So it's trained to predict the next syllable of text. That's what it's trained for doing. Basically, it predicts syllable by syllable texts from being given billions of examples.

Now, I understand at a technical standpoint, but from the outside looking in, if you just give it a series of prompts, I don't think most people would look at it like, oh, here comes syllable after syllable very rapidly. It looks very cogent and salient to me. What's going on there? How does that thing work then?

It's trained on very large number of sentences, so the next syllable will be very likely to come from some sentence or some phrase which has been uttered before many times. And also it works on very large windows of text, so it predicts the next thing on many hundreds of hundreds of characters. So it does give the impression of some sort of stream of consciousness, as if it can remember what it's talking about for a while because it's got this very large bit of text from which it's predicting, but it just uses the fact that it's got vast numbers of sentences stored which it can use. And so some of the mystery is that with machine learning in general, intuitively, what happens if you've got billions of examples is something which is really counterintuitive. So if you train on these number examples people never looked at before, then the phenomena are almost different. In kind, it's okay. Being impressed. I mean, it's impressive. Certainly the smoothness of the sentences is amazing. But I don't think one should assume that these things are providing you more than what it's doing, predicting the next syllables. Certainly you shouldn't take its advice, for example, for some important decision you have to make.

But if you were to take a Turing Test orientation, and now, okay, you're going to have to correct me if I get this wrong, but the Turing Test is if you're interacting with this thing and you don't know if it's a human or a machine, but you can't tell the difference between a human and a machine. Machine. Isn't it a human?

Yeah. Okay. So Turing wrote this paper in 1950. We discussed the word thinking and also intelligence, and basically said that, yes, if you can't tell the difference between a human and a machine, then you can't say that the human is thinking and the machine isn't thinking. So this is like almost a definition of thinking. So in some sense, it's the opposite of what I'm trying to do. I'm saying that you should define what thinking is and what intelligence is. So Turing didn't do this. He said it is what it is. If it looks like thinking, you know it's thinking. So Turing Test is where the machine can effectively impersonate a person. But again, I don't know how far that takes us. So, for example, if you look at these large language models, in some sense it looks like a human. But. So the Turing thing is that you do some 20 questions with it. You ask questions and see what the answers. And, for example, you'll find that large language models don't know about yesterday's news because it was trained a long time ago. Okay, so clearly it doesn't pass the Turing Test technically.

Okay.

But the point is that the Turing Test philosophically, is, of course, very important. It's intrigued people for decades. But by itself, it doesn't tell you what to do to make an intelligent machine. It just says that you shouldn't quibble philosophically.

So the fact that ChatGPT is not educable, does that make you more comfortable with everybody jumping on large language models, or does it scare you more? Is it a good thing or a bad thing that it isn't educable at this point?

One way of saying it is that with educability at all levels, there are some choices to be made, and it's not clear what the best choice is. With a large language model with just learning from examples, as we all know now, we always did. The training set is all important. For example, these large language models can have a political orientation. Depending on what text it's been trained on, they could have a bias. Just depends on what it's trained on. Even at this level, it's not clear what you should be doing right? I mean, depending on your political orientation, you may want to train it a different way. Already we're disagreeing on what the ideal large language model is. If you make things educable, then the problems pile up because like if you just tell your system beliefs, then it's pretty important for the question of what beliefs you give it. So maybe telling it evil beliefs is such a good idea. So although we're wonderful, we think and educable, there are all kinds of problems with reducing our educability. No one knows what the best method of education is. No one knows what the best beliefs are to instill in people. No one knows what right beliefs are and wrong beliefs are. So if we made machines with the same capabilities as humans, it wouldn't solve anything because we don't quite know what the best thing would be is to educate humans. Okay, so if you have an educable machine, you have to educate them. And many choices to be made in education.

What do you think poses a greater existential threat to the survival of mankind? Mankind or artificial intelligence?

Mankind. So the dangers of misusing artificial intelligence or having accidents and things like that are obviously present as they are with any other dangerous technology, chemistry, nuclear weapons, whatever. But I think the extreme viewpoint that somehow machines will take over because they become so intelligent that they'll control the world. I think that misplaced fear. So certainly we have to distinguish control from other capabilities. So we certainly shouldn't give control to a machine if we're not sure what the machine is going to do to us. That's absurd. Okay, so we try to not give it control, but if we don't give it control, then what it'll do will be things, I think which we kind of understand what's going on a bit. Like with chemistry, we understand some chemistry, not all of chemistry. So we have to treat it as any other, a powerful technology. But I don't think by its nature it's different from other technologies. So I don't think some monster is going to emerge from some machine somewhere which we don't understand. I'm not worried about that.

Okay, so is it a fair summary to say that we're at the starting line of educableness and we are not sure how to foster it, how to grow it and all this. And so there's so much research to be done. But it offers great potential to make the world a better place. If we can just figure out how we can get people to learn more from each other, learn more from what happened before. And is that kind of the picture you're trying to paint?

Yes, that's a very nice, generous description. Yes, I think that's a very nice way of putting it. Yes.

And in a sense I'm asking the same question twice. But if I'm listening to this and I'm buying into this, just like people listen to Carol Dweck and buy into the growth mindset. So now what can I do? I'm looking for some tips that I can use for my teenage son to make him more educable.

I'm not sure whether I can give a tip. I think possibly being aware of this dimension. So being aware to the extent to which we're soaking up information, we easily absorb ideas which we hear. So one of the points I make in the book is that our educability is very strong. We soak up information, we can relate information, use information, but evaluating information is much more difficult. So if we hear a theory, it's evaluating whether it's true or not isn't part of our basic cognitive capabilities. And probably because it's impossible, if you hear some report of something happening on the other side of the world, you can't go and look at it. You have to see whom you believe. So I think appreciating that we are so much subject to what we hear and see, and we can soak it up and agree with it, maybe. So one parameter which is important psychologists look at is what we said before is that with you hear all these theories, but which ones do we believe? So what's our criterion for whom to believe? So if we can internalize all these theories, we have to choose which ones too, because they may be contradictory. So what are our criteria? And so psychologists of course look at this, that we prefer theories which agree with what we already believe, theorists which agree with our friends, etc. So this educability I think is also a weakness. It shows that all the time we're prone to some new idea which we believe, but we shouldn't really, we may be tricked into it. It's our weakness to we have some policy for agreeing with things with which they already have a sympathy but may be wrong. So I think it's a viewpoint which is new to me. Just showing how vulnerable we are to. To the ideas which are swirling around us. So I think being aware of this educability notion may help people in understanding what's going on.

Listen, fundamentally, I am a marketing person. So I was chief evangelist of Apple. I'm chief evangelist of an online graphics design service called Canva. So I'm all about sales and marketing and evangelism. And I'm telling you from the outside looking in, you, you are holding in your hand a golden opportunity to change the world. If you could just get people off this kind of sat, IQ test, Mensa memberships and all that, and then just point out to them that it's not about this score. It's about how you can adapt and learn and coexist with other people. That's much more important than your GPA and your sat. And I'm serious, I think you may be holding the future of humanity in your hand here, Leslie. You got a big responsibility here.

Thank you for the comments. I think there are many ways this can go and I certainly need the help of lots of people to actually go in any of those ways. But yeah. So I think there's a new idea here which has a very general relevance to us.

I think there's no doubt in my mind that Leslie is onto something with this concept of educable. I think it is just the flip side of Carol Dweck's growth mindset. And you know how much I love Carol Dweck's growth mindset. Carol Dweck and Leslie Valiant. That would be a remarkable combination. Maybe I'll send this episode to Carol and see if she's interested in meeting Leslie. The world would shake if this happened. Anyway, I'm Guy Kawasaki. This is Remarkable People. Once again, I'm going to remind you, please read our new book, Think Remarkable. It will help you make a difference and change the world and be remarkable. Now, speaking of educable, we have a particularly educable group of people on the Think Remarkable team. They are, of course, Madison Nismer, producer Tessa Knysmer, researcher Jeff C. And Shannon Hernandez on sound design. And our best buddies, Alexis Nishimura, now at Santa Clara University, and Luis Shortboard, Magana. And finally, Fallon Yates. This is the Remarkable People team. We're on a mission to make you remarkable. Until next time, mahalo and aloha. This is remarkable people.