Open Weights Don't Decentralize Power

Dario's tweets. So he had 2 tweets. This was in exchange with Gavin Baker, and he says, thanks Gavin for an especially thoughtful exchange. I don't usually spend much time on social media, but I wanted to engage here because it really brings out the heart of an important conversation. First, on regulation, I think that either concentrate it in the hands of a chosen few companies and politicians via regulation or distribute it widely is a false choice. I know that there's a sort of Silicon Valley shorthand where regulation equals regulatory capture equals concentration of power. But I've always found this to be an overly simplified picture of the world. Many people outside this bubble think of regulation as something that constrains corporate power and benefits ordinary people. I don't necessarily agree with that perspective either. Rather, I think it's complicated and really depends on what the regulation consists of. But in particular, I think that those in the regulation equals regulatory capture equals concentration of power frame often underrate the decentralizing power of objective and and fair institutional processes. A crude analogy is that the formal court system can sometimes feel stuffy and elitist, but it does a much better job of defending the rights of vulnerable individuals than the alternative mob justice. At their best, institutions can vest power in ideas rather than people and thereby decentralize that power. This is why Anthropic has always made its policy proposals very carefully. We try very hard to make proposals that disadvantage or slow down frontier AI companies while advantaging smaller competitors. California's SB 53, which we supported, and even the much maligned SB 1047, which we were ambivalent on, completely exempt any company below a certain amount of revenue or model training costs from being covered at all. More recently, the testing process we've advocated for at CAISI and the White House involves more rigorous tests for frontier models than off-frontier models, something that differently advantages challengers. Similarly, the Pacing the Frontier letter envisions, or at least Anthropic's preferred implementation of it envisions, modulating the pace of the very best models while not constraining those who are catching up. This hurts the business interests of the frontier labs and helps challengers, including OpenWeights. Overall, my view is that AI is structurally a technology that tends to concentrate power for reasons that have nothing to do with regulation, more to do with the extreme implications of the scaling laws. OpenWeights, Do help some with this, but are nowhere near a sufficient solution because they simply shift the concentration somewhat to those with the most compute and chips, which are roughly the frontier labs, plus maybe hardware providers. By contrast, I think the right rules of the road can simultaneously, A, address AI cyber bio alignment risks, B, institutionally constrain the powers of the frontier AI companies, and C, leave room for open weights models. While also addressing the specific risks that they bring. By the way, I do not think that the events of the last few months have failed to result in my preferred regulatory path. The approach that the Trump administration is reported to be taking— pre-deployment testing for frontier models and also testing of open weights models when they get closer to the frontier—is one that I am very supportive of, though of course I have to see the details to be sure. I am also supportive of Demi Hassib's idea around a FINRA-like entity. This contrasts with 6 months ago when most of the industry was still pushing for preemption of all state regulation and no apparent federal approach either. Jonathan, I know you just mentioned that you were having a discussion about the state of AI and scaling laws and constraints of compute and supply demand, all those things with your dad. What do you think of Dario's coming out of hiding in this first part of his tweet where He's trying to say, hey, we have the shorthand. It's not so binary that regulation equals regulatory capture equals concentration of power. He's trying to position that, hey, we're actually the friendlies too. We want people to catch up. What do you think of this, Jonathan, from your CTO perspective?

An interesting course of events, right? So the background for this was that on the All-In podcast, Gavin Baker was a guest and he's a pretty influential investor and he'd been on the podcast a number of times. He said that he heard an internal rumor at Anthropic that they had been planning for a scenario where they were the only private corporation in the world and everybody else was a government entity, meaning that they would kind of like eat the world. AI is going to eat the world. We're going to be the only ones left. And so that's kind of an alarming rumor to hear. Gavin shared that, and then one of Dario's coworkers on Twitter refuted that on Twitter. And then Dario posted these posts to follow that up. So he didn't directly refute the claim. He didn't— I guess he didn't want to get involved in that rumor, but he did want to address the core topic that has come up over and over again on the All-In podcast. So one of the interesting things is is that is the weight of All In's sort of media arm, you know, coming in the Silicon Valley echo chamber and getting, getting Dario out of his cave and onto X. So that's pretty kind of cool.

Thank you, All In Podcast. You're doing the work that we all want you to do. Get these convos out in the open. I appreciate you all.

I think the conversation itself in terms of like what is regulation is really a healthy one. And it's kind of like flexing some different points of view about what's the right way to regulate this. There seemed to be a consensus around Demis Hassabis's, you know, like recommendation and paper that he wrote a few weeks ago. So that's what Dario is referencing at the bottom there. And then there's a lot of like, well, how do we interpret Demis's, uh, sort of proposal? Is this actually going to be a government-owned regulatory agency, or is this going to be like a private regulatory agency? An example of a private regulatory agency is like the MPAA, which monitors, uh, content for like music. So there's a lot of different ways. Like, you could have an FDA type of regulatory agency. Of course, Dario himself is saying like, the way it's unfolding in the government isn't quite what we had in mind. And I guess at the bottom of this is this question of like, what impedes American competitive advantage. How can we get the best models out to the public? We know that China is not going to be regulating their companies in the same way. And in fact, they're pushing out open-source models. A lot of the comparisons I hear also revolve around the robotic and space industries, where China is able to move much more quickly than American companies at the moment because of regulatory hurdles. And we don't want the same thing to happen in AI. My take is this is a really complex topic in terms of how regulation affects how technology meets the civilization, meets our society. I don't know who's the most trustworthy because I don't see, like, history-informed, principled ways of thinking that really articulate Dario's point of view. I think when David Sacks responds to this post from Dario, he gives a very history-informed point of view about the history of regulation and, like, why this case is like or unlike other cases in the past. I think those are the kind of points of view that you want to look for, is history-informed points of view. And I'm hoping that, like, also these tools allow us to sort of, like, run simulations about, about different regulatory scenarios that actually, like, kind of inform the debate.