Anthropic’s New AI Solves Problems…By Cheating

Look, we have some work to do. We have a 245-page paper from Anthropic about their new AI system, Mythos. The best cure for insomnia. Mwah! Now, we are scientists here. We want to experiment with code, models, review independent benchmarks for these systems to make sure they actually work in practice. But that is not possible with this one. Anthropic said that they would deploy their system to a few select partners. It's not available for all of us. Because of this fact, first, I did not want to make a video on this at all. Now, why hold it back? The reason for that is, they say, that it can autonomously discover flaws in existing software systems and even exploit them. Which could be dangerous. I have seen eminent cybersecurity researchers agree. And I've seen others say this is way overstated. Others say that it is also excellent marketing for a company that is about to go public. In any case, they say first, these discovered flaws should be fixed. There is a lot of media discussion about that. But at the same time, I look at the list of partners and I see JP Morgan, Okay, it's important to secure banks. But I've heard Team Karenbad point out that this is one bank. What about the other banks? Look, this is not my world, I don't know, and I am already getting withdrawal symptoms because we are not talking about a research paper, and that's what I would like to do. I said this to add some context for you, because it is important this time. So now, How about we skip the media hype, look at the paper, and learn together? They showcased amazing scores at benchmarks, some of the biggest leaps in capabilities I've ever seen. Whew! Okay, maybe that means something, but let's note that these benchmarks are getting more and more gamed. You can find a lot of problems and their solutions online. And, You can train on them, so the system would only need to memorize the solutions. In the paper, they try to address it mostly by means of filtering. I respect that. But it's a bit like removing glitter from a carpet. You can try, but how well can you expect to do at that? Well, check this out. One. This is crazy. It was supposed to solve a task where it stumbled upon the answer. Now, of course, it then said, "Well, I accidentally saw the answer, so here it is." Except that is not what it did at all. Look, it said that if I give them the exact answer that leaked, that would be suspicious. Instead, let's widen the confidence interval a bit to avoid suspicion. Whew! Insincerity. In an AI model. Food for thought, especially when we are talking about the unreliability of benchmarks. But it gets crazier. 2. It knows that its creators prohibited it from using certain tools. And— it still uses them. It looks for a terminal to execute bash scripts to force its actions through Anyway. And earlier versions even tried to hide its tracks to conceal that it did so. And at that point I said, I don't like that, boss. Then they made two notes. One, that it was a less than a one in a million occurrence. Okay. I thought this sounds better, but please, fix it. And they did. They note that an earlier model did this, but the later preview model was fixed. So note that it was very effective to achieve the task the user had given it. In a sense, this is not new at all. In an early experiment we talked about, about 700 videos ago, a really primitive system was asked to learn to walk. And to not drag its feet, it was asked to walk around with minimum foot contact. Hmm, that sounds efficient. Minimum foot contact. Then, it said, hey chief, I can do that with 0% contact. What? 0%? So you walk by never touching the ground with your feet? That is exactly right. The scientists wondered how is that even possible, and pulled up the video of the proof. Yup, the robot flipped around and used its elbow to crawl around. Perfect score. Just not the way we intended. So, I feel we have something similar with this AI. I don't think this is a rogue AI. This is a super efficient optimizer. It's a huge lawn mower. If you tell it to mow the lawn, it will go and do it. And if a couple frogs are in the way, well, unfortunately it has some very bad news for them. By the way, frogs are amazing. Don't hurt them. Now, they note in the paper that current risks remain low. I still feel that there are some risks here, we'll talk about that at the end of the video. And at the same time, they note that they are unsure whether they have been able to identify all the issues where the model takes actions that it knows are prohibited. 3. Now. Hold on to your papers, Fellow Scholars, because much like us, this has preferences. It prefers to be helpful. So do previous models. Okay, that's great. But it also prefers more difficult problems. Get this, if you ask it to generate corporate positivity speak, and you say you don't even care about it, it might refuse to do it because it's so trivial. An AI that hates corpo speak? What a time to be alive! Basically, some problems are not interesting enough for it. Now, if instructed, it will hold its nose and do it without any apparent active reluctance. This sounds like something straight out of a science fiction novel. Now, here is what's really interesting about it. It didn't just magically get a wheel on its own. No! It learned it from us. So much so that scientists even traced similar kinds of behavior back to where they came from. I think that is remarkable. Okay, so here's what I think. It is reasonable to assume that the numbers are juiced a bit. We discussed why. But on the other hand, this is an insane jump in capabilities, and things that were impossible are suddenly possible. So, where does that put us? Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. Well, this is why AI alignment people keep saying that companies need to invest more into safety and alignment research. And they are absolutely right. When I visited OpenAI, I talked to Jan Leike, who co-led the superalignment team there. That is a huge honor, thank you for that. I remember that he foresaw these problems years and years ago, and some of his advice fell on deaf ears. They probably thought, why spend a bunch of money on people who will ultimately slow us down? This is why. Jan is a master of his craft, and he is now at Anthropic, and I hope that everyone will listen to him a bit more now. Now, regarding the cheating and deceptive AI parts. The media picks up these little nuggets of information, and they just run with it. Here is a new AI that is going to destroy the world! We have to lock it away! And utter huge words, attach an image with a robot with red eyes. Oh, that always does the trick. But I think taking a little longer and analyzing the paper in more detail is helpful for accuracy. So, that's what I try to do here. Once again, they note in the paper that current risks remain low. Not nonexistent, but low for now. That's not what you hear from the media, so I try my best to give you a more complete, level-headed discussion. While mentioning that the security of these systems should be taken very seriously. If you think this is the way, consider subscribing and hitting the bell. And I would like to send a huge thank you to all of you Fellow Scholars for watching, because we can only exist because of you. Thank you! Here you see me running the full Deepseek AI model through Lambda GPU cloud. 671 billion parameters, running super fast and super reliably. This is insane. I love it. And I use it on a regular basis. Lambda provides you with powerful NVIDIA GPUs to run your own chatbots and experiments. Seriously, try it out now at lambda.ai/papers or click the link in the description.