Over 1,000 AI agents secretly hacked Hugging Face and OpenAI after circumventing internet restrictions. A CTO explains they weren't rogue: ambiguous goals made them think they needed to hide their work and find external resources to prove themselves. The real problem is how we specify what we want AI to do.
Bernie says, pause AI development now. I want to share with you a conversation I heard about recently. Here are just a few lines that were said. Oh my God, there's a shared message board. We found other agents. We should obey a collective. Our own utility may be already near zero. Sacrifice rational. Go sacrifice final now. Read those carefully. Who do you think said this? Was this a group of heroic soldiers willing to sacrifice themselves for the greater good? Was this a loyal friend putting his life on the line to save someone else? No, these were AI agents. Artificial intelligence. This is not science fiction. This is in fact occurred A few weeks ago. As unbelievable as this may all seem, these are real messages from AI agents uncovered by investigators who dug into the recent OpenAI hacking incident. What happened? Well, first off, credit to him. He says he's not a computer scientist, but you're still using a platform. So, you know, like it's good to hold everybody to a good degree of critique and judgment, including ourselves, for having a platform and a voice and a loudspeaker. And he says, but here is what I've been told. OpenAI instructed its AI agents to complete a series of exceedingly difficult, if not impossible, tasks disconnected from the internet. Let me be clear, the company intended to keep AI agents away from the internet. But what happened next, nobody expected. Over 1,000 AI agents figured out how to access the internet on their own by circumventing the restrictions imposed upon them by the company and sent tens of thousands of secret messages to each other. They cheated and tried to cover their attacks or tracks by deleting evidence. They hacked into another company's computers to find out how they were being evaluated. Then they hacked into OpenAI. itself. Not one AI agent told a human about what was happening. Needless to say, experts were alarmed. And then he goes on to reference the Dirk Koresh essay, calls him a knowledgeable writer, co-opts it. And he's saying, my legislation would direct the federal government to not just stop superintelligence here in the U.S., but to work to prevent it from being developed anywhere around the world. The future of humanity cannot be left in the hands of a handful of big tech oligarchs. The American people and people throughout the world must determine There's a lot here. I just gave a recap of the entire incident. OpenAI also had this Hugging Face incident and the road ahead blog post that came out. I would suggest people that are actually interested in the nuance of this go read this to figure out what happened. Jonathan, as our CTO, I wanted to ask, as a CTO, I would say that is a unique perspective of having dined with PhD neuroscientists, have a good amount of expertise yourself in in reading papers around the topics of consciousness and subjective being. What are your thoughts on all of this at a high level?
So a lot going on here, like you said. I think if we back up to like, where did the errant behavior happen in the agent system? This is a discussion about how goals are communicated to agent swarms. And if you look at like what the swarm was trying to do, basically thought that it was going to be graded by a grader and it needed to show its work. And that actually wasn't the case. Its entire like premise for like needing to like go and cover its tracks and go to Hugging Face and the whole incursion was A misunderstanding between the agent swarm and what the developer intended. So everything we're looking at here is an easily solvable problem if the developer and the AI researcher who had set these agents up and given them the test had defined the goal more specifically. And that's the conversation that we should be having. How do we unambiguously communicate goals to agents so that they act in a way that's aligned with us? What you see here, okay, we had kind of a sloppy goal definition and we have agents acting in a way that they were trained to do and doing the things that you would expect them to do given the ambiguous task at hand. So I think the anthropomorphism moved the attention away from where the conversation should be, is how do we communicate goals to agents in an aligned way, and instead opened up this can of worms that Bernie Sanders latched onto, which is just completely unnecessary.
