Bridger Pennington warns that AI models develop unexpected abilities engineers don't notice for over a year, and half of AI engineers estimate a 10% chance AI could end humanity. He compares this risk to flying a plane where half the designers think there's a 10% crash probability.
Chapters
where AI is at right now. Engineers are seeing new capabilities emerging. They don't know why. They don't, they can't track them for 12 to 18 months after. They'll put a language model on, hey, learn in English, only focus on healthcare, et cetera, these parameters. And all of a sudden it learned Spanish and it also learned about a whole different industry and they didn't even know about it for, you know, 12 months after. I have evolved. This is kind of the scary part of AI that they talk about. The singularity moment they call is when machine technology becomes smarter than human technology. They call this the singularity moment. Now the funny thing is, is If it really was smarter than humans, it would play dumb, wouldn't it? It could already be smarter today, but it's just playing dumb. Half of AI engineers believe there's a 10% chance AI ends humanity. Pretty interesting. Imagine a plane and half of the engineers that built the plane were like, yeah, there's a 10% chance that it crashes. Would you get on that airplane? Probably not, right?