Short excerpt from a Lex Fridman interview with Judea Pearl (Dec 12, 2019) focusing on Pearl’s concerns about advanced AI (AGI). Pearl warns that we are building a new, potentially self-replicating species that could exceed human capabilities and whose degree of controllability is unknown; the dialogue frames this as an unprecedented, 'sample-of-one' problem tempered by theoretical models.
Do you have concerns about the future of AI all the different trajectories of all of our research?
Yes.
Where's your hope? Where the movement heads were your concerns?
I'm concerned because I know we are building a new species that has a capability of exceeding our. Exceeding us, Exceeding our capabilities and can breed itself and take over the world. Absolutely. It's a new species that is uncontrolled. We don't know the degree to which we control it. We don't even understand what it means to be able to control this new species. So I'm concerned I don't have anything to add to that because it's such a gray area that unknown. It never happened in history. Yeah. The only, the only time it happened in history was evolution with human being. And it wasn't very successful, was it? Some people said it was a great success.
For us it was. But a few people along the way, a few creatures along the way would not agree. So it's just because it's such a gray area. There's nothing else to say.
We have a sample of one.
Sample of one, that's us. But some people would look at you and say, yeah, but we were looking to you to help us make sure that sample two works out okay.
Actually, we have more than a sample of model. We have theories.
Yeah.
And that's good. We don't need to be statisticians. So sample of one doesn't mean poverty of knowledge. It's not sample of one plus theory. Conjectural theory of what could happen.
Yeah.
That we do have. But I. I really feel helpless in contributing to this argument because I know so little and, and my imagination is limited and I know how much I don't know and, but I'm concerned.
