Anthropic embedded a long-term benefit trust into its charter, giving external AI-safety trustees board seats with no equity stake. This lets the company refuse dangerous releases and turn down a $200 million Pentagon contract without shareholder pressure overriding safety decisions.
Chapters
Dario was a first-time founder. None of the top venture funds wanted to participate in this round. Wasn't a hot company at all. The boom hadn't happened yet. ChatGPT hadn't been invented yet. Nonetheless, they were true believers in this safety mission. And so one of their investors suggested they come talk to me. I told them, look, if you don't get this right, here's what's going to happen. And they were very determined to do something about it. They wrote into their charter that they were going to do this. They were a PPC from the very beginning. They had the right to enact these additional reforms, what's called the, now the long-term benefit trust. And so even today, Anthropic has directors on its for-profit board who are appointed by and are accountable to an outside group of trustees who are AI safety experts who do not have equity in Anthropic. So they do not have a financial incentive in its growth. They have an incentive to see it done properly. So whenever you see Anthropic do the right thing, like when they refuse to release a model because they think it's too dangerous, think about how much that's costing them. They got into this fight with the Pentagon and they were in an impossible situation. They turned down a $200 million contract and bore the wrath of the world's largest army and government. That took a lot of courage. Part of that courage is enabled by the fact that they have this structure and investors can't just Ostario on a moment's notice.
