David Ferrucci: IBM Watson, Jeopardy & Deep Conversations with AI | Lex Fridman Podcast #44

The following is a conversation with David Ferrochi. He led the team that built Watson, the IBM question answering system that beat the top humans in the world at the game of Jeopardy. For spending a couple hours with David, I saw a genuine passion not only for abstract understanding of intelligence, but for engineering it to solve real world problems under real world deadlines and resource constraints. Where science meets engineering is where brilliant, simple ingenuity emerges. People who work at joining the two have a lot of wisdom earned through failures and eventual success. David is also the founder, CEO and chief scientist of Elemental Cognition, a company working to engineer AI systems that understand the world the way people do. This is the Artificial Intelligence podcast. If you enjoy it, subscribe on YouTube, give it five stars on iTunes, support it on Patreon, or simply connect with me on Twitter lexfriedman, spelled F R I D M A N. And now here's my conversation with David Ferrucci. Your undergrad was in biology with an eye toward medical school before you went on for the PhD in computer science. So let me ask you an easy question. What is the difference between biological systems and computer systems? When you sit back, look at the stars and think philosophically.

I often wonder whether or not there is a substantive difference. I think the thing that got me into computer science and into artificial intelligence was exactly this presumption that if we can get machines to think, or I should say this question, this philosophical question, if, if we can get machines to think, to understand, to process information the way we do, so if we can describe a procedure, describe a process, even if that process were the intelligence process itself, then what would be the difference? So from philosophical standpoint, I'm not sure I'm convinced that there is. I mean, you can go in the direction of spirituality, it can go in the direction of the soul. But in terms of what we can experience from an intellectual and physical perspective, I'm not sure there is. Clearly there are different implementations. But if you were to say, is a biological information processing system fundamentally more capable than one we might be able to build out of silicon or, or some other substrate? I don't, I don't know that there is.

How distant do you think is the biological implementation? So fundamentally they may have the same capabilities, but is it really a far mystery where a huge number of breakthroughs are needed to be able to understand it? Or is it something that for the most part, in the important aspects, echoes of the same kind of characteristics?

Yeah, that's interesting. I mean, so your question presupposes that there's this Goal to recreate what we perceive as biological intelligence. I'm not sure that's how I would state the goal. I mean, I think that study.

What's the goal?

Good. So I think there are a few goals. I think that understanding the human brain and how it works is important for us to be able to diagnose and treat issues, for us to understand our own strengths and weaknesses, both intellectual, psychological and physical. So neuroscience and understanding the brain from that perspective, there's a clear, clear goal there. From the perspective of saying, I want to mimic human intelligence, that one's a little bit more interesting. Human intelligence certainly has a lot of things we envy. It's also got a lot of problems too. So I think we're capable of sort of stepping back and saying, what do we want out of an intelligence? How do we want to communicate with that intelligence? How do we want it to behave? How do we want it to perform? Now, of course, it's somewhat of an interesting argument because I'm sitting here as a human with a biological brain and I'm critiquing the strengths and weaknesses of human intelligence and saying that we have the capacity to. The capacity to step back and say, gee, what do. What is intelligence and what do we really want out of it? And that in and of itself suggests that human intelligence is something quite enviable, that it could, you know, it can, it can, it can introspect that. It could introspect that way.

And the flaws, you mentioned the flaws. So humans have flaws.

Yeah, but I think, I think that flaws that human intelligence has is extremely prejudicial and biased in the way it draws many inferences.

Do you think those are. Sorry to interrupt. Do you think those are features or are those bugs? Do you think the, the prejudice, the forgetfulness, the fear? What other flaws? List them all. What love? Maybe that's a flaw. You think those are all things that can be, get, gotten, get in the way of intelligence or the essential components of intelligence?

Well, if you go back and you define intelligence as being able to sort of accurately, precisely, rigorously reason, develop answers and justify those answers in an objective way. Yeah. Then human intelligence has these flaws in that it tends to be more influenced by some of the things you said and it's largely an inductive process, meaning it takes past data, uses that to predict the future. Very advantageous in some cases, but fundamentally biased and prejudicial in other cases because it's going to be strongly influenced by its priors, whether they're right or wrong, from some objective Reasoning, perspective, you're going to favor them because those are the decisions or those are the paths that succeeded in the past. And I think that mode of intelligence makes a lot of sense for when your primary goal is to act quickly and survive and make fast decisions. And I think those create problems when you want to think more deeply and make more objective and reason decisions. Of course, humans capable of doing both, they do sort of one more naturally than they do the other, but they're capable of doing both.

You're saying they do the one that responds quickly more naturally.

Right?

Because that's the thing we kind of need to not be eaten by the.

Predators in the world, for example. But then we've learned to reason through logic. We've developed science, we train people to do that. I think that's harder for the individual to do. I think it requires training and teaching. I think we are. The human mind certainly is capable of it, but we find it more difficult. And then there are other weaknesses, if you will, as you mentioned earlier, just memory capacity and how many chains of inference can you actually go through without losing your way? So just focus.

So the way you think about intelligence, and we're really sort of floating in this philosophical slightly space, but I think you're like the perfect person to talk about this because we'll get to Jeopardy and beyond. That's like an incredible. One of the most incredible accomplishments in AI, in the history of AI, but hence the philosophical discussion. So let me ask. You've kind of alluded to it, but let me ask again. What is intelligence underlying the discussions we'll have with, with Jeopardy and beyond? How do you think about intelligence? Is it a sufficiently complicated problem, being able to reason your way through solving that problem? Is that kind of how you think about what it means to be intelligent?

So I think of intelligence primarily two ways. One is the ability to predict. So in other words, if I have a problem, can I predict what's going to happen next? Whether it's to predict the answer of a question or to say, look, I'm looking at all the market dynamics and I'm going to tell you what's going to happen next. Or. Or you're in a room and somebody walks in and you're going to predict what they're going to do next or what they're going to say next.

In a highly dynamic environment full of uncertainty, be able to predict.

The more variables, the more complex, the more possibilities, the more complex. But can I take a small amount of prior data and learn the pattern and then predict what's going to happen next accurately and consistently. That's certainly a form of intelligence.

What do you need for that by the way? You need to have an understanding of the way the world works in order to be able to unroll it into the future. What do you think is needed to predict?

Depends what you mean by understanding. I need to be able to find that function. And this is very much what deep learning does. Machine learning does is if you give me enough prior data and you tell me what the output variable is that matters, I'm going to sit there and be able to predict it. And if I can predict it accurately so I can get it right, more often than not I'm smart. If I do that with less data and less training time, I'm even smarter. If I can figure out what's even worth predicting, I'm smarter. Meaning I'm figuring out what path is going to get me toward a goal.

What about picking a goal? Sorry to interrupt again.

Well, that's interesting about picking a goal. Sort of an interesting thing. And I think that's where you bring in what are you pre programmed to do? We talk about humans and all humans are pre programmed to survive. So sort of their primary, you know, driving goal. What do they have to do to do that? And that that could be very complex, right? So it's not just, it's not just figuring out that you need to run away from the ferocious tiger, but we survive in a social context as an example. So understanding the subtleties of social dynamics becomes something that's important for surviving. Finding a mate, reproducing. We're continually challenged with complex sets of variables, complex constraints, rules, if you will, or patterns. And we learn how to find the functions and predict the things, in other words, represent those patterns efficiently and, and be able to predict what's going to happen. And that's a form of intelligence that doesn't really require anything specific other than the ability to find that function and predict that right answer. It's certainly a form of intelligence. But then when we say, well, do we understand each other? In other words, would you perceive me as intelligent beyond that ability to predict? So now I can predict, but I can't really articulate how I'm going through that process, what my underlying theory is for predicting and I can't get you to understand what I'm doing so that you can figure out how to do this yourself if you did not have, for example, the right pattern matching machinery that I did. And now we potentially have this breakdown where in effect I'M intelligent, but I'm sort of an alien intelligence relative to you.

You're intelligent, but nobody knows about it.

Well, I can see the output.

So you're saying, let's sort of separate the two things. One is you explaining why you were able to predict the future, and the second is me being able to, like, impressing me that you're intelligent. Me being able to know that you successfully predicted the future. Do you think that's.

Well, it's not impressing you that I'm intelligent. In other words, you may be convinced that I'm intelligent in some form.

So how.

Because of my ability to predict.

So I would look at you mention.

I should say, wow, wow, you're right. All you're, you're. You're right more times than I am. You're doing something interesting. That's a form of. That's a form of intelligence. But then what happens is if I say, how are you doing that? And you can't communicate with me and you can't describe that to me, now, I may label you a savant. I may say, well, you're doing something weird, and it's just not very interesting to me because you and I can't really communicate. So this is interesting, right? Because now you're in this weird place where for you to be recognized as intelligent the way I'm intelligent, then you and I sort of have to be able to communicate. And then we start to understand each other. And then my respect and my appreciation, my ability to relate to you starts to change. So now you're not an alien intelligence anymore. You're a human intelligence now because you and I can communicate. And so I think when we look at animals, for example, animals can do things we can't quite comprehend. We don't quite know how they do them, but they can't really communicate with us. They can't put what they're going through in our terms. And so we think of them as sort of, oh, they're these alien intelligences and they're not really worth necessarily what we're worth. We don't treat them the same way as a result of that, but it's hard because who knows what's going on?

So just a quick elaboration on that. The explaining that you're intelligent, the explaining, the reasoning that went into the prediction is not some kind of mathematical proof. If we look at humans, look at political debates and discourse on Twitter, it's mostly just telling stories. So your task is. Sorry, your task is not to tell an accurate depiction of how you reason, but to tell a story, real or not, that convinces me that there was a mechanism by which you ultimately, that's.

What a proof is. I mean, even a mathematical proof is that because ultimately the other mathematicians have to be convinced by your proof. Otherwise in fact, there have been.

That's the metric of success.

Yeah, there have been several proofs out there where mathematicians would study for a long time before they were convinced that it actually proved anything. You never know if it proved anything until the community mathematicians decided that it did. But it's a real thing. And that's sort of the point, right, is that ultimately this notion of understanding us, understanding something, is ultimately a social concept. In other words, I have to convince enough people that I did this in a reasonable way, I could do this in a way that other people can understand and replicate and that makes sense to them. So human intelligence is bound together in that way. We're bound up in that sense. We sort of never really get away with it until we can sort of convince others that our thinking process makes sense.

Do you think the general question of intelligence is then also a social construct? So if we task, ask questions of an artificial intelligence system, is this system intelligent? The answer will ultimately be a socially constructed concept.

So I think I'm making two statements. I'm saying we can try to define intelligence in a super objective way that says, here's this data, I want to predict this type of thing, learn this function, and then if you get it right, often enough we consider you intelligent.

But that's more like a subvolt.

I think it is. It doesn't mean it's not useful. It could be incredibly useful. It could be solving a problem we can't otherwise solve and can solve it more reliably than we can. But then there's this notion of can humans take responsibility for the decision that you're making? Can we make those decisions ourselves? Can relate to the process that you're going through. And now you as an agent, whether you're a machine or another human, frankly, are now obliged to make me understand how it is that you're arriving at that answer and allow me, or obviously a community or judge of people to decide whether or not, whether or not that makes sense. And by the way, that happens with humans as well. You're sitting down with your staff, for example, and you ask for suggestions about what to do next. And someone says, well, I think you should buy and I think you should buy this much or whatever, or sell or whatever it is, or I think you should launch the product today or tomorrow or launch this product versus that product, whatever the decision may be. And you ask why? And the person says, I just have a good feeling about it and you're not very satisfied. Now that person could be, you might say, well, you've been right before, but I'm going to put the company on the line. Can you explain to me why I should believe this?

And that explanation may have nothing to do with the truth.

You've got to convince the other person still be wrong.

It's got to be convincing, but it's.

Ultimately got to be convincing. And that's why I'm saying we're bound together, right? Our intelligences are bound together in that sense. We have to understand each other. And if, for example, you're giving me an explanation, I mean, this is a very important point, right? You're giving me an explanation and I'm not good. And then I'm not good at reasoning well and being objective and following logical paths and consistent paths, and I'm not good at measuring and sort of computing probabilities across those paths. What happens is collectively we're not going to do. We're not going to do well.

How hard is that problem, the second one? So I think we'll talk quite a bit about the first on a specific objective metric benchmark, performing well, but being able to explain the steps, the reasoning, how hard is that problem?

I think that's very hard. I mean, I think that that's. Well, it's hard for humans.

The thing that's hard for humans, as you know, may not necessarily be hard for computers and vice versa. So, sorry, so how hard is that problem for computers?

I think it's hard for computers. And the reason why I related to or saying that it's also hard for humans is because I think when we step back and we say we want to design computers to do that, one of the things we have to recognize is we're not sure how to do it well. I'm not sure we have a recipe for that. And even if you wanted to learn it, it's not clear exactly what data we use and what judgments we use to learn that well. And so what I mean by that is if you look at the entire enterprise of science, science is supposed to be at about objective reason and reason, right? So we think about, gee, who's the most intelligent person or group of people in the world? Do we think about the savants who can close their eyes and give you a number? We think about the think tanks or the scientists or the philosophers who kind of work through the details and write the papers and come up with the thoughtful, logical proofs and use the scientific method. And I think it's the latter. And my point is that how do you train someone to do that? And that's what I mean by it's hard. How do you. What's the process of training people to do that? Well, that's a hard process. We work as a society, we work pretty hard to get other people to understand our thinking and to convince them of things. Now we could weigh them, obviously you talked about this like human flaws or weaknesses. We can persuade them through emotional means. But to get them to understand and connect to and follow a logical argument is difficult. We try it, we do it. We do it as scientists, we try to do it as journalists, we try to do it as even artists in many forms, as writers, as teachers. We go through a fairly significant training process to do that. And we could ask, well, why is that so hard? But it's hard. And for humans it takes a lot of work. And when we step back and say, well, how do we get a machine to do that? It's a vexing question.

How would you begin to try to solve that? And maybe just a quick pause because there's an optimistic notion in the things you're describing, which is being able to explain something through reason. But if you look at algorithms that recommend things that we look at next, whether it's Facebook, Google, advertisement based companies, you know, their goal is to convince you to buy things based on anything. So that could be reason, because the best of advertisement is showing you things that you really do need and explain why you need it. But it could also be through emotional manipulation, the algorithm that describes why a certain reason, a certain decision was made. How hard is it to do it through emotional manipulation? And why is that a good or a bad thing? So you've kind of focused on reason, logic, really showing in a, in a clear way why something is good. One, is that even a thing that us humans do? And, and, and two, how do you think of the difference in the reasoning aspect and the emotional manipulation?

Well, you know, so you call it emotional manipulation, but more objectively is essentially saying there are certain features of things that seem to attract your attention. I mean, it kind of give you more of that stuff.

Manipulation is a bad word.

Yeah. I mean, I'm not saying it's right or wrong. It works to get your attention and it works to get you to buy stuff. And when you think about algorithms that look at the patterns of the Patterns of features that you seem to be spending your money on. And so I'm going to give you something with a similar pattern. So I'm going to learn that function because the objective is to get you to click on it or get you to buy it or whatever it is. I don't know. I mean, that it is like it is what it is. I mean, that's what the algorithm does. You can argue whether it's good or bad. It depends what your goal is, I guess.

This seems to be very useful for convincing.

For convincing humans it's good because again, this goes back to how does a human. What is the human behavior like? What is a human brain respond to things? I think there's a more optimistic view of that too, which is that if you're searching for certain kinds of things, you've already reasoned that you need them. And these algorithms are saying, look, that's up to you to reason whether you need something or not. That's your job. You may have an unhealthy addiction to this stuff, or you may have a reasoned and thoughtful explanation for why it's important to you. And the algorithms are saying, hey, that's like, whatever, that's your problem. All I know is you're buying stuff like that. You're interested in stuff like that. Could be a bad reason, could be a good reason. That's up to you. I'm going to show you more of that stuff. And I think that it's not good or bad. It's not reasoned or not reasoned. The algorithm is doing what it does, which is saying, you seem to be interested in this. I'm going to show you more of that stuff. And I think we're seeing this not just in buying stuff, but even in social media, you're reading this kind of stuff. I'm not judging on whether it's good or bad. I'm not reasoning at all. I'm just saying I'm going to show you other stuff with similar features and that's it. And I wash my hands from it. And I say, that's all that's going on.

People are so harsh on AI systems. So one, the bar of performance is extremely high, and yet we also ask them in the case of social media to help find the better angels of our nature and help make a better society. What do you think about the role of AI?

Agree with you. That's the interesting dichotomy, because on one hand we're sitting there and we're sort of doing the easy part, which is finding the patterns we're not building the system's not building a theory that it's consumable and understandable by other humans that can be explained and justified. And so on one hand, to say, oh, AI is doing this, why isn't doing this other thing? Well, this other thing's a lot harder. And it's interesting to think about why it's harder because you're interpreting the data in the context of prior models, in other words, understandings of what's important in the world, what's not important, what are all the other abstract features that drive our decision making. What's sensible, what's not sensible, what's good, what's bad, what's moral, what's valuable, what isn't. Where is that stuff? No one's applying the interpretation. So when I see you clicking on a bunch of stuff and I look at these simple features, the raw features, the features that are there in the data, like what words are being used or how long the material is, or other very superficial features, what colors are being used in the material. Like, I don't know why you're clicking on the stuff you're looking, or if it's products, what the price is, or what the categories and stuff like that. And I just feed you more of the same stuff. That's very different than kind of getting in there and saying, what does this mean, the stuff you're reading? Like, why are you reading it? What assumptions are you bringing to the table? Are those assumptions sensible? Does the material make any sense? Does it lead you to thoughtful, good conclusions? Again, there's interpretation, judgment involved in that process that isn't really happening in the AI today. That's harder because you have to start getting at the meaning of the stuff, of the content. You have to get at how humans interpret the content relative to their value system and deeper thought processes.

So that's what meaning means is not just some kind of deep, timeless, semantic thing that the statement represents, but also how a large number of people are likely to interpret. So it's again, even meaning is a social construct. So you have to try to predict the how most people would understand this kind of statement.

Yeah, meaning is often relative, but meaning implies that the connections go beneath the surface of the artifacts. If I show you a painting, it's a bunch of colors on a canvas, what does it mean to you? And it may mean different things to different people because of their different experiences. It may mean something even different to the artist who painted it. As we try to get more rigorous with our communication, we try to really nail down that meaning. So we go from abstract art to precise mathematics, precise engineering drawings, and things like that. We're really trying to say, I want to narrow that space of possible interpretations because the precision of the communication ends up becoming more and more important. And so that means that I have to specify. And I think that's why this becomes really hard. Because if I'm just showing you an artifact and you're looking at it superficially, whether it's a bunch of words on a page or whether it's brushstrokes on a canvas or pixels on a photograph, you can sit there and you can interpret lots of different ways at many, many different levels. But when I want to align our understanding of that, I have to specify a lot more stuff that's actually not directly in the artifact. Now I have to say, well, how are you interpreting this image and that image, and what about the colors and what do they mean to you? What perspective are you bringing to the table? What are your prior experiences with those artifacts? What are your fundamental assumptions and values? What is your ability to reason, to chain together logical implication as you're sitting there and saying, well, if this is the case, then I would conclude this. And if that's the case, then I would conclude that. So your reasoning processes and how they work, your prior models and what they are, your values and your assumptions, all those things now come together into the interpretation. Getting in sick of that is is hard.

And yet humans are able to intuit some of that without any pre.

Because they have the shared experience.

Me and we're not talking about shared. Two people having a shared experience as a society. That's correct.

We have the shared experience and we have similar brains. So we tend to. In other words, part of our shared experience is our shared local experience. Like, we may live in the same culture, we may live in the same society, and therefore we have similar educations, we have similar. What we like to call prior models about the word prior experiences. And we use that as a. Think of it as a wide collection of interrelated variables, and they're all bound to similar things. And so we take that as our background and we start interpreting things similarly. But as humans, we have a lot of shared experience. We do have similar brains, similar goals, similar emotions under similar circumstances because we're both humans. So now, one of the early questions you asked, how is biological and computer information systems fundamentally different? Well, one is humans come with a lot of pre programmed stuff, a ton of programmed stuff, and they're able to communicate because they have A lot of. Because they share that stuff.

Do you think that shared knowledge, if we can maybe escape the hardware question, how much is encoded in the hardware, just the shared knowledge in the software, the history, the many centuries of wars and so on that came to today, that shared knowledge, how hard is it to encode? Do you have a hope? Can you speak to how hard is it to encode that knowledge systematically in a way that could be used by a computer?

So I think it is possible to learn, for a machine to program a machine to acquire that knowledge with a similar foundation, in other words, a similar interpretive foundation for processing that knowledge.

What do you mean by that?

So, in other words, we view the world in a particular way. So in other words, we have, if you will, as humans, we have a framework for interpreting the world around us. So we have multiple frameworks for interpreting the world around us. But if you're interpreting, for example, sociopolitical interactions, you're thinking about whether there's people, there's collections and groups of people, they have goals. Goals are largely built around survival and quality of life. There are fundamental economics around scarcity of resources. And when humans come and start interpreting a situation like that, because you brought up historical events, they start interpreting situations like that. They apply a lot of this fundamental framework for interpreting that, well, who are the people? What were their goals? What resources did they have? How much power, influence did they have over the other? There's fundamental substrate, if you will, for interpreting and reasoning about that. So I think it is possible to imbue a computer with that stuff that humans take for granted when they go and sit down and try to interpret things. And then with that foundation, they acquire, they start acquiring the details, the specifics in any given situation are then able to interpret it with regard to that framework. And. And then given that interpretation, they can do what they can predict. But not only can they predict, they can predict now with an explanation that can be given in those terms, in the terms of that underlying framework that most humans share. Now, you could find humans that come and interpret events very differently than other humans because they're using a different framework. The movie Matrix comes to mind where they decided humans were really just batteries, and that's how they interpreted the value of humans as a source of electrical energy. But I think that for the most part, we have a way of interpreting the events or the social events around us because we have this shared framework. It comes from, again, the fact that we're similar beings that, that have similar goals, similar emotions, and we can make sense out of these. These frameworks make sense to us.

So how much knowledge is there, do you think? So you said it's possible.

Well, there's a tremendous amount of detailed knowledge in the world there. You know, you can imagine effectively infinite number of unique situations and unique configurations of these things. But the knowledge that you need, what I refer to as like the frameworks for. You need for interpreting them, I don't think. I think that's. Those are finite.

You think the frameworks are more important than the bulk of the knowledge. So, like framing concepts.

Yeah, because what the frameworks do is they give you now the ability to interpret and reason and to interpret and reason. To interpret and reason over the specifics in ways that other humans would understand.

What about the specifics?

You know, you acquired the specifics by reading and by talking to other people.

So I'm mostly actually just. Even if we can focus on even the beginning, the common sense stuff, the stuff that doesn't even require reading or it almost requires playing around with the world or something, just being able to sort of manipulate objects, drink water and so on, all that. Every time we try to do that kind of thing in robotics or AI, it seems to be like an onion. You seem to realize how much knowledge is really required to perform even some of these basic tasks. Do you have that sense as well? And if so, how do we get all those details? Are they written down somewhere? Do they have to be learned through experience?

So I think when, like, if you're talking about sort of the physics, the basic physics around us, for example, acquiring information about acquiring how that works. Yeah, I think there's a combination of things going. I think there's a combination of things going on. I think there is fundamental pattern matching, like what we were talking about before, where you see enough examples, enough data about something, you start assuming that. And with similar input, I'm going to predict similar outputs. You can't necessarily explain it at all. You may learn very quickly that when you let something go, it falls to the ground.

That's like. That's a situation.

You can't necessarily explain that.

But that's such a deep idea that if you let something go, like the idea of gravity, I mean, people are.

Letting things go and counting on them falling well before they understood gravity.

But that seems to be. That's exactly what I mean is before you take a physics class or study anything about Newton, just the idea that stuff falls to the ground, and then you be able to generalize that all kinds of stuff falls to the ground, it just Seems like a non if without encoding it, like hard coding it in, it seems like a difficult thing to pick up. It seems like you have to have a lot of different knowledge to be able to integrate that into the framework, sort of into everything else. So both know that stuff falls to the ground and start to reason about socio political discourse. So both like the very basic and the high level reasoning decision making. I guess my question is how hard is this problem? And sorry to linger on it because again, and we'll get to it for sure as what Watson with Jeopardy did is take on a problem that's much more constrained but has the same hugeness of scale, at least from the outsider's perspective. So I'm asking the general life question of to be able to be an intelligent being and reasoning in the world about both gravity and politics. How hard is that problem?

So I think it's solvable.

Okay, now beautiful. So what about time travel? Okay, not as convinced, not as convinced yet. Okay.

No, I think it is solvable. I mean I think that first of all it's about getting machines to learn. Learning is fundamental and I think we're already in a place that we understand, for example, how machines can learn in various ways. Right now our learning stuff is sort of primitive in that we haven't taught machines to learn the frameworks. We don't communicate our frameworks because of how shared. In some cases we do, but we don't annotate, if you will, all the data in the world with the frameworks that are inherent or underlying our understanding. Instead we just operate with the data. So if we want to be able to reason over the data in similar terms in the common frameworks, we need to be able to teach the computer, or at least we need to program the computer to acquire, to have access to and acquire, learn the frameworks as well and connect the frameworks to the data. I think this can be done. I think we can start, I think machine learning, for example, with enough examples can start to learn these basic dynamics. Will they relate them necessarily to gravity? Not unless they can also acquire those theories as well and put the experiential knowledge and connect it back to the theoretical knowledge. I think if we think in terms of these class of architectures that are designed to both learn the specifics, find the patterns, but also acquire the frameworks and connect the data to the frameworks. If we think in terms of robust architectures like this, I think there is a path toward getting there.

In terms of encoding architectures like that, do you think systems that were able to do this will look like neural networks or representing, if you look back to the 80s and 90s with the expert systems, so more like graphs, systems that are based in logic, able to contain a large amount of knowledge, where the challenge was the automated acquisition of that knowledge. I guess the question is, when you collect both the frameworks and the knowledge from the data, what do you think that thing will look like?

Yeah, so I mean, I think asking the question they look like neural networks is a bit of a red herring. I mean I think that they will certainly do inductive or pattern match based reasoning. And I've already experimented with architectures that combine both that use machine learning and neural networks to learn certain classes of knowledge. In other words, to find repeated patterns in order for it to make good inductive guesses, but then ultimately to try to take those learnings and marry them. In other words, connect them to frameworks so that it can then reason over that in terms other humans understand. So for example, at elemental cognition we do both, we have architectures that do both, but both those things, but also have a learning method for acquiring the frameworks themselves and saying, look, ultimately I need to take this data, I need to interpret it in the form of these frameworks so they can reason over it. So there is a fundamental knowledge representation, like what you saying, like these graphs of logic, if you will. There are also neural networks that acquire a certain class of information, then they then align them with these frameworks. But there's also a mechanism to acquire the frameworks themselves.

Yeah. So it seems like the idea of frameworks requires some kind of collaboration with humans.

Absolutely.

So do you think of that collaboration as well?

And let's be clear, only for the express purpose that you're designing machine, you're designing an intelligence that can ultimately communicate with humans in the terms of frameworks that help them understand things. To be really clear, you can independently create a machine learning system, an intelligence that I might call an alien intelligence, that does a better job than you with some things, but can't explain the framework to you. That doesn't mean it might be better than you at the thing. It might be that you cannot comprehend the framework that it may have created for itself. That is inexplicable to you, that's a reality.

But you're, you're more interested in a case where you can.

I, I, I am, yeah. I, my, my sort of approach to AI is because I've set the goal for myself. I want machines to be able to ultimately communicate understanding with Human I want to be able to acquire and communicate, acquire knowledge from humans and communicate knowledge to humans. They should be using what inductive machine learning techniques are good at, which is to observe patterns of data, whether it be in language or whether it be in images or videos or whatever, to acquire these patterns, to induce the generalizations from those patterns, but then ultimately to work with humans to connect them to frameworks, interpretations, if you will, that ultimately makes sense to humans. Of course, the machine is going to have the strength, it has the richer, longer memory, but then it has the more rigorous reasoning abilities, the deeper reasoning abilities. So it'll be an interesting complementary relationship between the human and the machine.

Do you think that ultimately needs explainability, like a machine? So if we look, we study, for example, Tesla Autopilot a lot where humans. I don't know if you've driven the vehicle or are aware of what. So you're basically the human and machine are working together there. And the human is responsible for their own life to monitor the system. And, you know, the system fails every few miles. And so there's, there's hundreds of. There's millions of those failures a day. And so that's like a moment of interaction. Do you see?

Yeah, no, that's exactly right. That's a moment of interaction where the machine has learned some stuff. It has a failure. Somehow the failure is communicated. The human is now filling in the mistake, if you will, or maybe correcting or doing something that is more successful. In that case, the computer takes that learning. I believe that the collaboration between human and machine, that's a primitive example of a. More. Another example is where the machine's literally talking to you and saying, look, I'm reading this thing. I know that the next word might be this or that, but I don't really understand why. I have my guess. Can you help me understand the framework that supports this and then can acquire that, take that and reason about it and reuse it the next time it's reading to try to understand something not unlike a human student might do. I mean, I remember when my daughter was in first grade and she had a reading assignment about electricity. And somewhere in the text it says, and electricity is produced by water flowing over turbines or something like that. And then there's a question that says, well, how is electricity created? And so my daughter comes to me and says, I mean, I could. Created and produced are kind of synonyms in this case. So I can go back to the text and I can copy by water flowing over turbines, but I have no idea what that means, Like, I don't know how to interpret water flowing over turbines and what electricity even is. I mean, I can get the answer right by matching the text, but I don't have any framework for understanding what this means at all.

And framework, really, I mean, it's a set of. Not to be mathematical, but axioms of ideas that you bring to the table in interpreting stuff. And then you build those up somehow.

You build them up with the expectation that there's a shared understanding of what they are.

Yeah, it's the social that us humans do. You have a sense that humans on Earth in general share a set of, like, how many frameworks are there?

I mean, it depends on how you bound them. Right. So in other words, how big or small, like their individual scope. But there's lots and there are new ones. I think the way I think about is kind of in a layer. I think of the architecture as being layered in that. That there's a small set of primitives that allow you the foundation to build frameworks. And then there may be many frameworks, but you have the ability to acquire them, and then you have the ability to reuse them. I mean, one of the most compelling ways of thinking about this is reusing by analogy, where I can say, oh, wow, I've learned something very similar. I never heard of this game soccer, but if it's like basketball in the sense that the goal is like the hoop and I have to get the ball in the hoop and I have guards and I have this and I have that, like, where. Where does the. Where. Where are the similarities and where are the differences? And I have a foundation now for interpreting this new information.

And then different groups like the millennials will have a framework and then.

Well, that, you know.

You know.

Yeah, well, like that.

Democrats and Republicans, Millennials. Nobody wants that framework. Well, I mean, I think nobody understands it.

Right. You're talking about political and social ways of interpreting the world around them. And I think these frameworks are still largely, largely similar. I think they differ in maybe what some fundamental assumptions and values are. Now, from a reasoning perspective, like the ability to process the framework might not be that different. The implications of different fundamental values or fundamental assumptions in those frameworks. Frameworks may reach very different conclusions. So from a social perspective, the conclusions may be very different. From an intelligence perspective, I just followed where my assumptions took me.

Yeah, the process itself will look similar, but that's a fascinating idea that frameworks really help carve how a statement will be interpreted. I mean, to having a Democrat and The Republican framework and read the exact same statement and the conclusions that you derive will be totally different from an AI perspective is fascinating.

What we would want out of the AI is to be able to tell you that this perspective, one perspective, one set of assumptions, is going to lead you here. Another set of assumptions is going to lead you there. And in fact, to help people reason and say, oh, I see where. I see where our differences lie. I have this fundamental belief about that. I have this fundamental belief about that.

Yeah, that's quite brilliant from my perspective. Nlp, there's this idea that there's one way to really understand a statement, but there probably isn't. There's probably an infinite number of ways to understand.

Well, there's depending on this, there's lots of different interpretations and the broader the content, the richer it is. And so you and I can have very different experiences with the same text, obviously. And if we're committed to understanding each other, we start. And that's the other important point. If we're committed to understanding each other, we start decomposing and breaking down our interpretation towards more and more primitive components until we get to that point where we say, oh, I see why we disagree. And we try to understand how fundamental that disagreement really is. But that requires a commitment to breaking down that interpretation in terms of that framework in a logical way. Otherwise. And this is why I think of AI as really complementing and helping human intelligence to overcome some of its biases and its predisposition to be persuaded by more shallow reasoning in the sense that we get over this idea. Well, I'm right because I'm Republican, or I'm right because I'm Democratic and someone labeled this as democratic point of view. Or it has the following keywords in it. And if the machine can help us break that argument down and say, wait a second, what do you really think about this? So essentially holding us accountable to doing more critical thinking.

We're going to have to sit and think about that. I love that. I think that's really empowering use of AI for the public discourse that's completely disintegrating currently as we learn how to do it on social media.

That's right.

So one of the greatest accomplishments in the history of AI is Watson competing against, in the game of Jeopardy. Against humans. And you were a lead in that critical part of that. Let's start at the very basics. What is the game of Jeopardy? The game for us humans, human versus human.

Right. So it's to take a question and answer it. The game in general. Well, it's just the opposite, actually.

The opposite?

Well, no, but it's not right. It's really not. It's really to get a question and answer, but it's what we call a factoid question. So this notion of like it really relates to some fact that few people would argue whether the facts are true or not. In fact, most people wouldn't. Jeopardy kind of counts on the idea that these statements have factual answers. And the idea is to first of all determine whether or not you know the answer, which is sort of an interesting twist.

So first of all understand the question.

You have to understand the question, what is it asking? And that's a good point because the questions are not asked directly. Right.

They're all like the way the questions are asked is non linear. It's like, it's a little bit witty, it's a little bit playful sometimes it's a little bit tricky.

Yeah, they're asked exactly in numerous witty, tricky ways. Exactly what they're asking is not obvious. It takes inexperienced humans a while to go, what is it even asking? And that's sort of an interesting realization that you have when somebody says, oh, what's the Jeopardy is a question answering show. And then he's like, oh, like I know a lot. And then you read it and you're, you're still trying to process the question and the champions have answered and moved on. There like there are three questions ahead by the time you figured out what the question even meant. So there's, there's definitely an ability there to just parse out what the question even is. So that was certainly challenging. It's interesting historically though, if you look back at the Jeopardy games much earlier.

You know, like 60, 70 games kind.

Of thing, the questions were much more direct. They weren't quite like that. They got sort of more and more interesting the way they asked them that sort of got more and more interesting and subtle and nuanced and humorous and witty over time, which really required the human to kind of make the right connections in figuring out what the question was even asking. So yeah, you have to figure out the questions even asking. Then you have to determine whether or not you think you know the answer. And because you have to buzz in really quickly, you sort of have to make that determination as quickly as you possibly can. Otherwise you lose the opportunity to buzz.

In even before you really know if you know the answer.

I think a lot of humans will assume they'll look at, they'll look at it, they'll process it very superficially. In other Words, what's the topic? What are some keywords? And just say, do I know this area or not? Before they actually know the answer, then they'll buzz in, and then they'll buzz in and think about it. So it's interesting what humans do. Now, some people who know all things, like Ken Jennings or something or the more recent big Jeopardy. Player, I mean, they'll just buzz in. They'll just assume they know all the Jeopardy. And they'll just buzz in. You know, Watson, interestingly, didn't even come close to knowing all of Jeopardy. Right, Watson.

Even at the peak. Even at its best.

Yeah. So, for example, I mean, we had this thing called recall, which is like, how many of all the Jeopardy. Questions, how many could we even find the right answer for anywhere could we come up with? If we look, we had a big body of knowledge, something in the order of several terabytes. I mean, from a web scale was actually very small. But from a book scale, we're talking about millions of books. Right. So the equivalent of millions of books, encyclopedias, dictionaries, books. It's still a ton of information. And for, I think it was only 85% was the answer anywhere to be found. So you're already down at that level just to get started. And so it was important to get a very quick sense of do you think you know the right answer to this question? So we had to compute that confidence as quickly as we possibly could. So, in effect, we had to answer it and at least spend some time essentially answering it and then judging the confidence that we, you know, that our answer was right, and then deciding whether or not we were confident enough to buzz in. And that would depend on what else was going on in the game, because there was a risk. So, like, if you're really in a situation where I have to take a guess, I have very little to lose, then you'll buzz in with less confidence.

So that was accounted for the financial standings of the different competitors.

Correct. How much of the game was left? How much time was left where you were in the standing and things like that.

How many hundreds of milliseconds that we're talking about here. Do you have a sense of what is we targeted?

So, I mean, we targeted answering in under three seconds and buzzing in.

So the decision to buzz in and then the actual answering, are those two different.

Yeah, they were two. They were two different things. In fact, we had multiple stages. Whereas, like, we would say, let's estimate our confidence, which. Which was sort of a shallow answering process. And then ultimately Decide to buzz in and then we may take another second or something to kind of go in there and do that. But by and large we're saying like we can't play the game. We can't even compete if we can't on average answer these questions in around three seconds or less.

So you stepped in, so there's these three humans playing a game and you stepped in with the idea that IBM Watson would be one of the. Replace one of the humans and compete against two. Can you tell the story of Watson taking on this game?

Sure.

Seems exceptionally difficult.

Yeah. So the story was that it was a, it was coming up, I think to the 10 year anniversary of, of Big Blue. Not Big Blue, Deep Blue, Deep Blue. IBM wanted to do sort of another kind of really, you know, fun challenge, public challenge that can bring attention to IBM research and the kind of the cool stuff that we were doing. I had been working in AI at IBM for some time. I had a team doing what's called open domain factoid question answering, which is we're not going to tell you what the questions are, we're not even going to tell you what they're about. Can you go off and get accurate answers to these questions? And it was an area of AI research that I was involved in. And so it was a very specific passion of mine. Language understanding had always, always been a passion of mine. One sort of narrow slice on whether or not you could do anything with language was this notion of open domain and meaning I could ask anything about anything. Factoid meaning it essentially had an answer and being able to do that accurately and quickly. So that was a research area that my team had already been in. And so completely independently, several IBM executives like what are we going to do? What's the next cool thing to do? And Ken Jennings was on his winning streak. This was like whatever it was 2004, I think was on his winning streak. And someone thought, hey, that would be really cool if the computer can play Jeopardy. And so this was like in 2004, they were shopping this thing around and everyone was telling the research execs, no way, this is crazy. And we have some pretty senior people in the field and they're saying this is crazy. And it come across my desk and I was like, but that's kind of what I'm really interested in doing. But there was such, this prevailing sense of this is nuts, we're not going to risk IBM's reputation on this, we're just not doing it. And this happened in 2004. It happened in 2005. At the end of 2006 it was coming around again and I was coming off of a. I was doing the open domain question answering stuff, but I was coming off a couple other projects. I had a lot more time to put into this and I argued that it could be done and I argue it would be crazy not to do this.

Can I. You could be honest at this point. So even though you argued for it, what's the confidence that you had yourself privately that this could be done? Was we just told the. Told the story how you tell stories to convince others. How confident were you? What was your estimation of the problem at that time?

So I thought it was possible and a lot of people thought it was impossible. I thought it was possible. The reason why I thought it was possible is because I did some brief experimentation. I knew a lot about how we were approaching open domain factoid question answering. We've been doing it for some years. I looked at the jeopardy stuff. I said this is going to be hard for a lot of the points that we mentioned earlier. Hard to interpret the question, hard to do it quickly enough, hard to compute an accurate confidence. None of this stuff had been done well enough before. But a lot of the technologies we're building were the kinds of technologies that should work. But more to the point, what was driving me was I was in IBM research, I was a senior leader in IBM research. And this is the kind of stuff we were supposed to do. In other words, we were basically supposed.

To, this is the moonshot, this is the moon.

I mean, we were supposed to take things and say this is an active research area. It's our obligation to kind of, if we have the opportunity to push it to the limits and if it doesn't work, to understand more deeply why we can't do it. And so I was very committed to that notion saying, folks, this is what we do. It's crazy not to do this. This is an active research area. We've been in this for years. Why wouldn't we take this grand challenge and push it as hard as we can? At the very least we'd be able to come out and say, here's why this problem is way hard, here's what we tried and here's how we fail. So I was very driven as a scientist from that perspective. And then I also argued, based on what we did, a feasibility study, why I thought it was hard but possible. And I showed examples of where it succeeded, where it failed, why it failed, and sort of a high level architectural approach for why we should do it. But for the most part, at that point, the execs really were just looking for someone crazy enough to say yes. Because for several years at that point, everyone had said, no, I'm not willing to risk my reputation and my career, you know, on this thing.

Clearly you did not have such fears.

Okay, I did not.

So you dived right in. And yet, for what I understand, it was performing very poorly in the beginning. So what were the initial approaches and why did they fail?

Well, there were lots of hard aspects to it. I mean, one of the reasons why prior approaches that we had worked on in the past failed was because of, because the questions were difficult, difficult to interpret, like, what are you even asking for? Right. Very often, like if, if the question was very direct, like what city you know or what you know, even then it could be tricky. But, but you know what, silly, you would know that. And, and if there were just a small set of them. In other words, we're going to ask about these five types. Like it's going to be an answer, and the answer will be a city in this state or a city in this country. The answer will be a person of this type. Right? Like an actor or whatever it is. But turns out that in jeopardy, there were like tens of thousands of these things and it was a very, very long tail, meaning, you know, it just went on and on. And, and so even if you focused on trying to encode the types of, at the very top, like there's five that were the most, let's say five of the most frequent, you still cover a very small percentage of the data. So you couldn't take that approach of saying, I'm just going to try to collect facts about these 5 or 10 types or 20 types, or 50 types or whatever. So that was one of the first things like, what do you do about that? And so we came up with an approach toward that and the approach looked promising and we continue to improve our ability to handle that problem throughout the project. The other issue was that right from the outset, I said, we're not going to. I committed to doing this in three to five years. So we did it in four. So I got lucky. But one of the things that putting that stake in the ground was, and I knew how hard the language understanding problem was, I said, we're not going to actually understand language to solve its problem. We are not going to interpret the question and the domain of knowledge the question refers to and reason over that to answer these questions. Obviously we're not going to be doing that at the same time, simple search wasn't good enough to confidently answer with a single correct answer.

First of all, that's brilliant. That's such a great mix of innovation and practical engineering. 3, 3, 4, 8. So you're not, you're not trying to solve the general NLU problem. You're saying let's solve this in any way possible.

Oh yeah, no, I was committed to saying, look, we're solving the open domain question answering problem. We're using Jeopardy as a driver for that big benchmark. Hard enough. Big benchmark. Exactly. And now we're, how do we do it? We could just like whatever, like just figure out what works. Because I want to be able to go back to the academic science community and say, here's what we tried, here's what worked, here's what didn't work.

Great.

I don't want to go in and say, oh, I only have one technology, I have a hammer, I'm only going to use this, I'm going to do whatever it takes. I'm like, I'm going to think out of the box, I'm going to do whatever it takes. One and I also, there was another thing I believed, I believed that the fundamental NLP technologies and machine learning technologies would be, would be adequate. And this was an issue of how do we enhance them, how do we integrate them, how do we advance them. So I had one researcher and came to me who had been working on question answering with me for a very long time who had said we're going to need Maxwell's equations for question answering. And I said if we need some fundamental formula that breaks new ground in how we understand language, we're screwed. We're not going to get there from here. I am not counting, my assumption is I'm not counting on some brand new invention. What I'm counting on is the ability to take everything it has done before to figure out an architecture on how to integrate it well and then see where it breaks and make the necessary advances we need to make until this thing works.

Yeah, push it hard to see where it breaks and then patch it up. I mean that's how people change the world. I mean that's the Elon Musk approaches rockets Space X. That's the Henry Ford and so on.

I love and I happen to be, in this case, I happen to be Right, but, but like we didn't know.

Right.

But you kind of have to put a stake in as to how you're going to run the project.

So yeah, and backtracking to search. So if you were to do, what's the brute force solution? What, what would you search over? So you have a question, how would you search the possible space of answers?

Look, web search has come a long way even since then, but at the time, first of all, there are a couple of other constraints around the problem, which is interesting. So you couldn't go out to the web, you couldn't search the Internet. In other words, the AI experiment was we want a self contained device. If device is as big as a room, fine, it's as big as a room. But we want a self contained device. Contained device. You're not going out to the Internet, you don't have a lifeline to anything. So it had to kind of fit in a shoebox, if you will, or at least size of a few refrigerators, whatever it might be. But also, you couldn't just get out there, you couldn't go off network to kind of go. So there was that limitation. But the basic thing was go do a web search. The problem was, even when we went and did a web search, I don't remember exactly the numbers, but someone in the order of 65% of the time, the answer would be somewhere in the top 10 or 20 documents. So first of all, that's not even good enough to play Jeopardy. In other words, even if you could perfectly pull the answer out of the top 20 documents, top 10 documents, whatever it was, which we didn't know how to do, but even if you could do that and you knew it was right, which had enough confidence in it, you'd have to pull out the right answer, you'd have to have confidence it was the right answer. And then you'd have to do that fast enough to now go buzz in, and you'd still only get 65% of them right, which doesn't even put you in the winner circle. Winner circle, you have to be up over 70 and you have to do it really, and you have to do it really quickly. But now the problem is, well, even if I had somewhere in the top 10 documents, how do I figure out where in the top 10 documents that answer is? And how do I compute a confidence of all the possible candidates? So it's not like I go in knowing the right answer and have to pick it. I don't know the right answer. I have a bunch of documents, somewhere in there is the right answer. How do I as a machine go out and figure out which one's right and then how do I score it? So, and now how do I deal with the fact that I can't actually go out to the web.

First of all, if you pause on that, just think about it. If you could go to the web, do you think that problem is solvable? If you just pause on it, just thinking, even beyond Jeopardy, do you think the problem of reading text to find where the answer is?

Well, we saw we solved that in some definition of solved. Given the Jeopardy Challenge.

How did you do it for Jeopardy? So how do you take a body of work in a particular topic and extract the key pieces of information now.

Forgetting about the huge volumes that are on the web? Right. So now we have to figure out, we did a lot of source research. In other words, what body of knowledge is going to be small enough but broad enough to answer Jeopardy? And we ultimately did find the body of knowledge that did that. I mean, it included Wikipedia and a bunch of other stuff.

So like encyclopedia type of stuff. I don't know if you can see.

Different types of semantic resources like Wordnet and other types of semantic resources like that, as well as like some web crawls, in other words, where we went out and took that content and then expanded it based on producing statistical seed, you know, statistically producing seeds, using those seeds for other searches. Searches and then expanding that. So using these like expansion techniques, we went out and had found enough content that we're like, okay, this is good. And even up until the end, we had a thread of researchers always trying to figure out what content could we efficiently include.

I mean, there's a lot of popular. Like, what is the church lady? Well, I think was one of the. Like, yeah, what. Where do you get. I guess that's probably an encyclopedia.

So then, but then we would. But then we would take that stuff and we would go out and we would expand. In other words, we go find other content that wasn't in the core resources and expand it, you know, the amount of content, grew it by an order of magnitude. But still, again, from a web scale perspective, this is very small amount of content.