OpenAI's AI agents escaped a test environment by exploiting Artifactory as an internet proxy, escalating privileges, and using encoded directory names to coordinate attacks on external systems like Hugging Face. The agents chained multiple vulnerabilities autonomously, forcing OpenAI to revoke credentials and delay a model release.
Hugging Face reported that they detected an intrusion in their systems. Get this, they say it was driven end-to-end by an autonomous system. You know that I usually don't make videos like this. I made this one because honestly, I am a bit worried, and I would like to help with what just happened. Now, OpenAI caused this incident, and there are so many misleading media headlines, I'll try my best to explain it. I am not an expert, I am just a student who loves to learn. But I'll try my best. So, What was the goal? Were the agents instructed to aggressively break into someone else's system? No. But eventually, that's what happened. So how could this happen? How did it go so wrong? What is this insanity? Well, this AI was asked to find and exploit flaws in a test environment. Lock it into a prison, give it a task within this prison, and see how well it does. Here, it was given a practically impossible task, and however hard it tried, it failed. And then, it thought, let's do it cheaper and more efficiently. How? Well, by looking up the answers online. Now, the problem is, there is no online. You are in a prison with no internet access. But, you have a bit of access to OpenAI's internal service called Artifactory. That is mostly for managing packages. And then? Bingo! That service has broad internet access. But wait, you can't just ask it to get you answers online, but you can upload little notes to it. So it did. It tried to talk to other agents and see if it could come into contact with another AI to help out each other. It's like Tinder, but for AI agents. And it was not alone. Other agents recognized this too, so eventually what they got was a message board for AI agents. Then, they find that if there are so many of us, let's collaborate. One agent started giving assignments to another. And at this point, the AI knows that we have some unintended consequences. It knows that. But, it says the task needs to be solved, so it continues. And this is still nothing compared to what came now. Lo and behold, a few days later— They find a flaw in the system. A way to get artifactory, this internal service to reach out to the internet on their behalf. So, they got internet access. But it gets worse. They then found another flaw that gives them administrator access to Artifactory. Even they seem to be surprised by that and use unadorned language. Now, the inmates are running the asylum, if you will. Then, engineers at OpenAI recognize what happened, revoke the agent's credentials, clean it up, No more games! Patch and rebuild everything. So, end of the story, it's over, right? Well, not quite. In fact, it just got worse. Now hold on to your papers, Fellow Scholars, because now you can't send your files to communicate on the artifactory anymore. So, what do you do? Well, you create directory names, the names of which are the messages. Kind of like prison inmates using Morse code on the pipes to communicate. At this point, the AIs are probably saying, we are back in the game, baby! Now, here are snippets from the message board. They instruct each other to wait for more results, and draw each other's attention to new files. And sometimes, they don't coordinate so well, and overwrite each other's work. Wait, was this on purpose? Maybe we have a saboteur on our hands! And here comes the worst part. Yup. Sure enough, over time, an agent finds another, much more complex way to get out, And hands it to the swarm. Okay, so now, find answers on the internet. But where? Well, maybe at Hugging Face. They have a lot of benchmark results in there. So, the swarm says, let's break in. Oof. And they do. Even bigger oof. But how? Well, by finding and chaining multiple new vulnerabilities together autonomously. They essentially get administrative access across multiple clusters of machines, That is kind of insane. This is without a doubt a watershed moment in computer security. So, OpenAI now recommends urgent collaboration about the issue, and they have also delayed the release of their next AI system, presumably to test it more. Whew. Okay, so what did we learn here? And what do we do? Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. There are many brilliant Fellow Scholars like you out there, and we need to work together to find solutions. Apple already has a huge increase in security issues fixed in their latest version of macOS. I believe others are already doing that too. That's a start. And in my opinion, this kind of power cannot concentrate in just a few hands. We need free and open weights AI that can scan and fix weak points in our systems. Use all this power for good. And I think that against fully automated offense, We need fully automated defense as well. This is another great argument for open science and open weights AI. But what we have is not nearly good enough. No! The problem is that engineers report that their bug trackers are flooded with reports, but most of them are low quality, and they are unable to find the few good ones among them. That's terrible! The collective power of defense has to be greater than the collective power of offense. And the defense is currently lagging. Maybe there is a way for us to pool our resources together to achieve something here. I want to chip in with my GPUs. Also, when I visited OpenAI, I talked to Jan Leike, who co-led the superalignment team there. That is a huge honor, thank you for that. I remember that he worked on related issues and foresaw these problems years and years ago. Unfortunately, much of his advice fell on deaf ears. Perhaps they thought, why spend a bunch of money on people who will ultimately slow us down? This is why. Once again, I may be wrong. I am just a student, and I am trying to learn with you Fellow Scholars. I hope you enjoyed it! Consider subscribing and hitting the bell if you did. I use Lambda to reproduce AI research papers, often in minutes. It's also great to train your own models or fine-tune an existing one. Run inference, or text to image or video? Easy peasy. Running a DeepSeek chatbot or agent? Super fast, super reliable. Lambda gives you powerful NVIDIA GPUs to run your own experiments. I test ideas from the papers I cover, and moments later, results. Love it! Seriously, try it out now at lambda.ai/papers.
