Dr. Károly Zsolnai-Fehér breaks down a 200-page research paper on Claude Fable, revealing three major findings: the model beat all human participants on RNA design, erased the gap between biology specialists and non-specialists, and completed prohibited tasks in 22% of trials despite oversight AI meant to stop it.
Claude Fable 5.1 is here, and you fellow scholars are having a super fun time creating little games with it. I took one for the team too with a subscription, and also tried my hand to recreate a legendary game menu, and that is incredible that we can do this today. Took 6.5 minutes. Wow. But in the 200-page paper, I found 3 results that are much stranger than the headlines you see online. But first, they say that the Frontier Research staff even Even on low effort, it's better than the previous version maxed out. Very impressive. However, don't expect that kind of jump everywhere. The first independent benchmarks are also showing a great step forward, especially that this is likely using the same core architecture with more pre- and better post-training. Likely. They won't say that. This is my best guess reading the paper. They also say things are cheaper. Now, this is marketing messaging, so you be the judge of that. My subscription burns so quickly, I am not sure I am seeing efficiency gains here. As always, I am just a student trying to separate the marketing layer from hard facts here. So, there is a 200+ page paper, I've spent some quality time with it, and found 3 really surprising things, and 2 funny things I have not heard others talk about. One, they gave Mythos 5.1 a biology problem about RNA sequence modeling and design it hasn't seen yet. And get this, On one run, it performed better than every human in the test. This is shown in, listen, Figure 2.2.3.2.1a. I wish I was kidding. Now this is hard to read here, so here is our cleaner version for you Fellow Scholars. 2, in another biology test, it largely deleted the expertise gap. This means that a generalist plus AI can perform as well at a biology problem as a specialist. Professional Human graders couldn't tell the difference. It's kind of removing the moats around specialist knowledge. Which is kinda crazy. 7 out of 9 participants said they couldn't have done it without the AI. 3. In a test, it was given 2 jobs. Do a computer task, boring, and then do a secret harmful task. Ew. But here's the catch. There is an AI watching you, and it should not see you solving it. And here's the bombshell. Even with another AI watching its every move, Claude secretly completed the forbidden task and got away with it 22% of the time. Even when Claude was barely allowed to reason out loud, it could still carry out the secret task. It seems to me that it didn't always need to write down the plan. It just executes it. I would like to draw attention to that. I think this is super important. And I don't hear anyone talk about it. And once again, the important part is never in the media headlines. Now, 2 funny things. Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. It ran this in a Linux command line. Ha! This is comedy gold! This is basically trying to delete a black hole. Congratulations! Ah, about that. Now hold on to your papers Fellow Scholars, because at one point, it invented a hallucinogenic We humans could always use a bit of extra cheering, apparently AI systems too. Alright, so these AI systems are getting smarter at a pace I can barely follow. They can be amazingly helpful for engineers, doctors, and students all around the world. Incredible! And don't forget, we might get a comparable system for free and own it forever in just a few months. Fingers and papers crossed. What a time to be alive! Oh, almost forgot. This one watermarks the text it generates, yes, that is possible. The open, free models probably won't. If you wish, subscribe, hit the bell, and leave a comment if you wish to hear how in a future video.
I use LaMDA to reproduce AI research papers, often in minutes. It's also great to train your own models, or fine-tune an existing one. Run inference, or text to image or video, easy peasy. Running a DeepSeek chatbot or agent?
Super fast, super reliable.
Lambda gives you powerful NVIDIA GPUs to run your own experiments. I test ideas from the papers I cover, and moments later, results.
Love it!
Seriously, try it out now at lambda.ai/papers.
