Why Opus 5 Won't Follow Your Instructions

So Dario says, second, on the messaging around AI, I do not agree that my messaging has been disproportionately negative. In fact, it has been about equally balanced between risks and benefits. I've written one major essay about each, and even in interviews where I discuss the risks, I make sure to frequently mention the incredible benefits as well as proposing possible solutions to the risks. Short clips from my interviews that end up on social media tend to be disproportionately negative as that gets clicks. To his credit, that is very true. That's been the news for all of time in capitalism. In fact, I wrote Machines of Love and Grace because I didn't feel the AI industry was painting an inspiring enough picture of how the technology could radically transform the world for the better. Bulk of the essay is devoted to refuting skepticism of AI's potential in health and biology and showing why I think it will be actually possible to cure most human disease in 5 to 10 years, as crazy as it may sound to ordinary people and frankly to biologists as well. I used to be one. And if you read my most recent essay, Policy on the AI Exponential, I discuss concrete proposals for how to streamline the FDA process to make sure the deluge of AI-accelerated drugs isn't slowed down by the regulatory process. I feel the urgency here. I lost my father to hepatitis C only a few years before the development of direct-acting antivirals, which cured 95% of patients and probably would have cured him. I do agree that the public has a negative view of AI and that this is a big problem, but I don't think it is primarily caused by me or any other AI leader warning about AI's risks. Bro, like, really? You don't think so? You don't think going out in public and saying everybody's going to lose their jobs and undergrads won't have jobs coming out of college, like knowledge workers won't have it, like you don't think that was at all your responsibility? That's just kind of like— I just got to comment on that. Like, come on, you got to take some responsibility there. He then says, I think it is fundamentally a crisis of trust that he caused. I think that ordinary people don't trust companies, governments, or the tech industry and always suspect that we are cooking up some new way to screw them over. The causes of this go back decades and AI is just the latest iteration of it. I don't think that a glitzy marketing campaign with a positive spin, which some have advocated that Anthropic do, is the way to win back that trust. At this point, saying that AI will cure cancer is more a cliché than it is inspiring, and most people think it is deceptive. The thing that will work is actually curing cancer. I think by far the most accurate criticism of AI companies, including Anthropic, is that we haven't yet delivered on our big promises to benefit the world. That is totally on us, and I think it's the criticism you should be making instead of all of this stuff about messaging and marketing. We are, however, doing our best to fix this. Anthropic is ramping up its efforts very quickly in biology and medicine, and we hope to have incredible results in the coming years and some early glimmers in the coming months. When we've actually accomplished something real, the whole world will hear about it as loudly as possible. You have my word on that. But until then, I don't want to make empty promises, and in the meantime, I feel compelled to speak honestly about the very real risks of AI and how to address them. Honesty is the right thing on the merits and in terms of public credibility and trust, it is no worse and in fact, maybe better than an approach that ignores or distracts from risks, which people instinctively understand are real. I really don't like his lack of ownership here about the public having a negative view of AI. It's like, dude, you like went and caused this. And it's clear that it's for your regulatory capture campaign. Taking a step back, if you're for a FINRA-type organization where you look at FINRA from the outside, if you're a fintech startup founder You're like, you don't like that. It's just a self-regulatory body that keeps the incumbents richer for the banks. And if Dario was saying, like, let's do that, let's also slow chips to China, let's stop distillation. Distillation is something Dario could stop. He could just KYC his whole app and then cut China off of paying him for distilling his models. You could do a lot of this. It's just really counter to a lot of the facts, the observable facts that I see. But I do like this part of it that Dario says, which is the thing that will actually work is actually curing cancer. So, you know, the proof's in the pudding, like go cure cancer, go solve that. I think everybody on the planet would really love if Anthropic solved curing cancer. I do like that take, which is let's just actually solve the core problems that we set out to solve, the promise of the technology. Jonathan, what's your take on this from a CTO perspective? Dario's second part of his tweet, he's kind of talking about how he wasn't that negative and he didn't really have any responsibility in the public's view of AI being such a negative thing.

I've never heard such a unified sort of like, what is this guy talking about response to this second part of the thread. Everybody probably in the responses, everybody on the podcast media is like, dude, you haven't been listening to yourself. When you have one person who thinks that they're like, they're saying one thing and that all of everybody else in reality thinks they're saying a different thing, you know, what leads to that cognitive dissonance, right? Where is he sitting? At some point I sympathize that he is probably well-intending, but just Like, not skilled at communication to masses. Like, most of us aren't skilled at communication to masses, and if we just like put our opinion out there, we would be misunderstood. But his job is to be the CEO of his company and to communicate effectively to the masses, and showing that he's unskilled at doing that in a really kind of obvious way. So I think that's funny for something he seems like thinks they walk on water. It's like, you're so inept in this direction. I think that that's kind of funny. The cancer was an interesting case. I don't know if you saw this announcement this week, but there was a pretty big breakthrough from Moderna and Merck using personalised mRNA vaccines, and it did use a form of AI to sort of make this breakthrough. And this is a big deal. There's a roadmap to cure all cancer through our pharmaceutical companies, which is exciting. And what role did Anthropic play in this? What role did OpenAI play in this? So there's a set of tools that stem more from DeepMind's protein folding simulations that created the foundation for this. But it is the type of thing that DeepMind's been working towards, that Anthropic's working towards, that OpenAI is working towards. And I think when you step back and you say, like, all right, I've got this company Anthropic, they're making more money than anybody in history, and they're growing faster than anybody in history, and they keep putting out models. What they know how to do is put out models that pass benchmarks and are generally good for some set of tasks. But then they show us that, like, through releasing Opus 5, they don't know, like, what a good benchmark max is going to mean in the real world for, like, real customers and what a good benchmark is how it's going to relate to delivering cancer vaccines. They don't know that. And they sort of don't have a privileged position to say that our technology is going to solve these problems because they don't know how to make benchmarks that are going to be predictive of solving these problems. I think that's where all this trust comes to face, is people see, okay, there's this guy that can't communicate well making all this money. I don't really want him to make more money. It seems like he's going to. He's not really helping me. They're not really solving the big problems they said they're going to solve yet. And so yeah, you got kind of like this big PR tension in terms of like public, and the way it manifests is of course like no more data centers. Like let's just cut these guys off at the throat, right? Yeah.