Peter Norvig warns that we rely too much on simple explanations for AI systems. A single explanation isn't enough; you need to test adversarially and ask follow-up questions to catch when models rely on spurious signals. Real models exist in high-dimensional spaces where tiny changes can break them in unexpected ways.
Foreign.
Anytime you use neural networks, anytime you learn from data, form representation from data in an automated way, it's not very explainable as to, or it's not introspective to us humans in terms of how this neural network sees the world. Where why does it succeed so brilliantly on so many, in so many cases and fail so miserably in surprisingly ways and small? So what do you think is this is the future? There can simply more data, better data, more organized data solve that problem? Or is there elements of symbolic systems that need to be brought in which are a little bit more explainable?
Yeah, so I prefer to talk about trust and validation and verification rather than just about explainability. Then I think explanations are one tool that you use towards those goals. And I think it is an important issue that we don't want to use these systems unless we trust them and we want to understand where they work and where they don't work. And an explanation can be part of that. So I apply for a loan and I get denied. I want some explanation of why. And in Europe we have the GDPR that says you're required to be able to get that. But on the other hand, an explanation alone is not enough. Right. So we are used to dealing with people and with organizations and corporations and so on, and they can give you an explanation and you have no guarantee that that explanation relates to reality. Right. So the bank can tell me, well, you didn't get the loan because you didn't have enough collateral. And that may be true, or it may be true that they just didn't like my religion or, or something else. I can't tell from the explanation, and that's true whether the decision was made by a computer or by a person. So I want more, I do want to have the explanations and I want to be able to have a conversation to go back and forth and said, well, you gave this explanation, but what about this? And what would have happened if this had happened and what would I need to change that? So I think a conversation is a better way to think about it than just an explanation as a single output. And I think we need testing of various kinds. Right? So in order to know was the decision really based on my collateral or was it based on my religion or skin color or whatever? I can't tell if I'm only looking at my case, but if I look across all the cases, then I can detect a pattern. Right? So you want to have that kind of capability. You want to have these adversarial testing. Right. So we Thought we were doing pretty good at object recognition in images. We said, look, we're at sort of pretty close to human level performance on imagenet and so on. And then you start seeing these adversarial images and you say, wait a minute, that part is nothing like human performance.
You can mess with it really easily.
You can mess with it really easily. Right. And yeah, you could do that to humans too. Right.
In a different way perhaps.
Right. Humans don't know what color the dress was.
Right.
And so they're vulnerable to certain attacks that are different than the attacks on the machines. But the attacks on the machines are so striking. They really change the way you think about what we've done.
Right.
And the way I think about it is, I think part of the problem is we're seduced by our low dimensional metaphors. Right? Yeah, I like that phrase. You look in a textbook and you say, okay, now we've mapped out the space and cat is here and dog is here and maybe there's a tiny little spot in the middle where you can't tell the difference. But mostly we've got it all covered. And if you believe that metaphor, then you say, well, we're nearly there and there's only going to be a couple adversarial images. But I think that's the wrong metaphor. And what you should really say is it's not a 2D flat space that we've got mostly covered, it's a million dimension space. And cat is this string that goes out in this crazy path. And if you step a little bit off the path in any direction, you're in nowhere's land and you don't know what's going to happen. And so I think that's where we are. And now we've got to deal with that. So it wasn't so much an explanation, but it was an understanding of what the models are and what they're doing. And now we can start exploring how do you fix that.
Yeah. Validating the robustness of the system, so on. But take it back to this word trust. Do you think we're a little too hard on our robots in terms of the standards we apply? So, you know, of. There's a dance, there's a, there's a, there's a dance in non verbal and verbal communication between humans. You know, if we apply the same kind of standard in terms of humans, you know, we trust each other pretty quickly. You know, you and I haven't met before and there's some degree of trust, Right. That nothing's gonna Go crazy. Wrong. And yet to AI when we look at AI systems where we seem to approach through skepticism, always, always. And it's like they have to prove through a lot of hard work that they're even worthy of even inkling of our trust. What do you think about that? How do we break that barrier? Close that gap?
I think that's right. I think that's a big issue. Just listening. My friend Mark Moffat is a naturalist, and he says the most amazing thing about humans is that you can walk into a coffee shop or a busy street in a and there's lots of people around you that you've never met before and you don't kill each other. Yeah. He says chimpanzees cannot do that.
Yeah, right, right.
If a chimpanzee's in a situation where here's some that aren't from my tribe, bad things happen, especially in a coffee shop.
There's delicious food around. You never know.
Yeah, yeah. But we humans have figured that out. Right. And, you know, for the most part, for the most part, we still go to war, we still do terrible things, but for the most part, we've learned to trust each other and live together. So that's going to be important for our AI systems as well. And also, I think a lot of the emphasis is on AI, but in many cases, AI is part of the technology, but isn't really the main thing. So a lot of what we've seen is more due to communications technology than AI technology. Yeah, you want to make these good decisions, but the reason we're able to have any kind of system at all is we've got the communication so that we're collecting the data and so that we can reach lots of people around the world. I think that's a bigger change that we're dealing with.
