Former OpenAI And Googlers Researchers BREAK SILENCE on AI

So one of the pieces of information that came out last week that actually got buried under a mountain of different AI news was a letter released not just by OpenAI or by Google, but by a lot of Frontier Labs that actually work on AI at the Frontier level. And they actually made a letter that I would say isn't necessarily shocking, but it is concerning to the extent that we need to kind of realize what is actually going on on these top companies and how things are currently being developed. So the piece I'm referring to is a letter called A Right to Warn about Advanced Artificial Intelligence. And this is something that I think you should understand. Now, whilst this doesn't, I guess you could say, kind of impact how current AI systems might be developed today in terms of like how systems are released, I still think that this conversation is, is really important because one of the things that this letter is, you know, shocking in terms of like, what, you know, being like's going on about it. Like, one of the things that actually really did, you know, catch me by surprise was that this letter was, you know, like signed by, you know, a lot of people who worked at OpenAI. We can see that around, literally, you know, of the 11, around nine of them worked at OpenAI and the other two work at Google, DeepMind, and one of them being formerly from Anthropic. So it's clear that whilst, yes, the large majority of the people from this is from OpenAI, there is also a decent majority, well, not majority, but still those who are still at Frontier Labs or were working at Frontier Labs that share the same opinion. And it was also endorsed by, of course, Yoshua Bengio, Geoffrey Hinton and Stuart Russell, very, very influential names in the artificial intelligence space. And Geoffrey Hinton is regarded as the godfather of AI. Now you can see I didn't get to this piece of information straight away, but there's actually a lot of information surrounding this piece because it's not just a letter that they wrote, there's actually a lot of other pieces of information that are regarding this. So what essentially we have here is a letter where they're basically stating that they want the right to be able to warn the public, which is you and me and about any sort of artificial intelligence dangers, if there are any. So it starts by stating that we are current, okay, current and former employees at Frontier AI companies and we believe that the potential technology of AI to deliver unprecedented benefits to humanity. It then goes on to state that we also understand the serious risks posed by these technologies. And these risks range from further Entrenchment of existing inequalities, which, which is something that I talked about in my private community, which is something that a lot of people aren't, you know, paying attention to. And of course, the manipulation and misinformation to loss of control of autonomous AI systems potentially resulting in a human extension. And for those of you who think that that is, I guess you could say, you know, Hogs Wash or something that is very, very far away. A lot of the industry leaders actually place AGI at maybe five to 10 years away. And you have to understand, five to 10 years away is not that far considering, you know, we're in 2024 right now. So by 2030, we could definitely have some really, really incredible powerful systems. And in the decade after that, human extinction isn't going to be something that is, you know, I guess you could say off the cards, because one of the things that people don't even think about when it comes to human extinction is, is that human extinction might not even be a big risk. I think with these powerful, you know, extremely, extremely powerful technologies. Okay. And the unprecedented benefits is that you could also have bad actors gaining, you know, unfiltered access to these models and being able to carry out, you know, a significant amount of damage. And like they've said in a recent interview that I was watching on Lex Friedman, Roman Yaplovsky actually said that one of the dangers of super intelligent AI was the fact that the problem is, is that the damage that an AI can do is always proportional to its capabilities. Like whatever kind of, you know, weapon that you have, it's always proportional to the capabilities of that weapon. So if we do have super intelligent systems or systems that are just really, really good, I mean, the problem might not even be the autonomous system. It might actually be humans as well. So that is always something that people need think about. Stating that, oh, stop thinking about Terminator is something that, you know, you kind of do need to think about yourself. So. Or hundreds of millions of people, billions of people, or the entirety of human civilization. That's a big leap. Exactly. But the systems we have today have capability of causing X amount of damage. So when they fail, that's all we get. If we develop systems capable of impacting all of humanity, all of universe, the damage is proportionate. What do you are the possible ways that it says AI companies have themselves acknowledged these risks, as have governments across the world and other AI experts, and they state that we are hopeful that these risks can be adequately mitigated with sufficient guidance from the scientific community. Policymakers and the public. However, AI companies have strong financial incentives to avoid effective oversight. And we do not believe bespoke structures of corporate governance are sufficient to change this. And I think this part right here is quite important because the corporate governance of OpenAI, the way how things have been, like if you looked at how OpenAI's governance structure was, you could kind of see that. And the thing is, with the governance structure being mentioned, saying that we don't believe in the structures of corporate governance are sufficient to change this. I think it's an important point because if we actually take a look at how, how Sam Altman was fired, the governance structure that OpenAI did have in place actually played a crucial role in the board's decision to fire the CEO, Sam Altman. And there were basically like six key factors. One of the first key Factors was that OpenAI's unique structure involved a non profit entity that actually controlled a for profit subsidiary. And this setup was designed to basically ensure the mission of, you know, developing safe and beneficial AGI remains paramount, even over profit motives. And the nonprofit board ultimately has the authority over the for profit subsidiary, which means that the board's decisions are driven by the mission rather than the shareholder interests. And now here's point number two, which is where the board composition and independence actually get into play. So the board of OpenAI actually was composed of independent directors who don't hold any equity in the company. And this independence is intended to prevent the conflicts of interest and ensure that decisions are made in the best interest of the mission, which is what they're trying to do. But the problem with that is that it also means that the board can actually make decisions without the influence of investors and shareholders, which is, you know, one of the things that contributed to the decision of firing Sam Altman with consulting the stakeholders like Microsoft. You remember Satya Nadella? He was completely surprised and he was like, whoa, whoa, whoa, what on earth just happened? What is going on here? You know, I need to get in contact with you guys now. So, because it doesn't have a fiduciary duty to shareholders, instead the primary responsibility being the mission to the nonprofit, this actually does allow the board to prioritize concerns about Altman's leadership and the company's direction over the potential financial implications of for his removal. Basically the point here is that the board structure that OpenAI had allowed the company to essentially remove Sam Altman with prior notice or consultation from other stakeholders. And this was something that was pretty crazy because we got to see Sam Altman be removed in a shocking manner and nobody really knew it was coming. But the problem here is that this led to a revolt from investors and employees, ultimately forcing the board to reinstate Altman. And this incident actually underscores the consequences of a governance structure that inadequately balances different organizational goals and stakeholder interests. Now, in contrast, there's an article that actually talks about how, you know, governance is really important for pretty much the future of humanity because these AI labs are doing some really important things. And one of the things that they are doing, or I should say one of the frontier labs are doing, is that Anthropic, which was founded by former OpenAI employees, actually developed a governance model designed to support both its mission and its financial goals more effectively. This article here actually does talk about how Anthropic managed to do this. And it says the counterexample Anthropic, it featured a corporate board structure where some directors are selected by shareholders and others buy a long term benefit trust and ensuring a balance between mission alignment and fiduciary responsibilities. And this structure actually aims to prevent some of the conflicts that we saw at OpenAI by incorporating checks and balances and accommodating the perspectives of various stakeholders. Overall, the chaos at OpenAI has led to a lot of people believing that, you know, the corporate governance structures that are currently in place are pretty much insufficient to run the things. And that's why I think even one of the recent interviews that we got with Leopold Aschenbrenner actually spoke about how some of these labs are probably going to be nationalized because some of the things that are going on just on all of the levels mean that the government is going to be getting involved.

Today I think people talk about AGI and they're always just talking about the private AI labs. And I think I just really want to challenge that assumption. It just seems pretty likely to me, as we've talked about, for reasons we've talked about, that look like the national security state is going to get involved. I think there's a lot of ways this could look like. Is it like nationalization? Is it a public private partnership? Is it a kind of defense contractor like relationship? Is it a sort of government project that soaks up all the people? And so there's a spectrum there. But I think people are just vastly underrating the chances of this more or less looking like a government project when we have literal superintelligence on our cluster, right? And it's like you have a billion superintelligent scientists, they can hack everything they can stuxnet, the Chinese data centers. You know, they're starting to build the robo armies. You know, you like, you really think that'll be like a private company and the government would be like, oh my God, what is going on?

Even, and even recently, well, not recently, but OpenAI actually did announce some improvements to its governance structure nearly five months after the Sam Altman ouster. And basically they just decided to add three people to its board of directors. And interestingly enough, Sam Altman actually rejoined the board nearly five months after he was abruptly forced out. So they basically did this entire post to basically state that, look, we know our board was, you know, pretty crazy, but we're trying to improve it as much as we can. And I'm going to actually come back to some of this because it does say additional independent directors will help monitor Outman and keep him in check. But this is, you know, quite interesting because there's a little bit more that I do want to talk about, which is why this letter is so important. So they go on to state that AI companies possess substantial non public information about the capabilities and limitation of their systems, the adequacy of their protective measures and the risk levels of different kinds of harm. However, they currently have only weak obligations to share some of this information with governments and none with civil society. Now this is very, very true because the problem is that I do remember that some of the governments aren't actually getting the information about these AI systems before they are released. And that is a pretty big concern. Think about it like this. You have a private company developing some very, very powerful levels of intelligence products. You know, you agree with, you know, some nations or certain governments that hey, look, we're going to give you this in advance, but it's a weak obligation. There's nothing contractual about it. And currently I'm going to show you guys an article which shows you that, you know, Big Tech isn't actually willing to give these governments the transparency they need to ensure that these systems are safe for the public. So here we have rishi sunak, the UK's prime minister. And you can see it says Rishi Sunak promised to make AI safe. However, Big Tech is not playing ball. A landmark AI testing deal hailed by the UK leader is being shunned by by emerging tech's biggest players, including OpenAI and Meta. So it says. At a historic gathering in Bletchley park late last year, American tech luminaries including Sam Alman and Elon Musk agreed to closely share their closely guarded AI models. With the British government, which was billed a major coup for the country's digital savvy prime minister. However, and this is why this letter was written six months later, Rishi Sunak's AI Safety Institute is failing to test the safety of the most leading AI models like GPT5 before they are released, despite heralding a landmark deal to check them for big security threats. So the problem here is that of course whilst they could say that okay, we're going to do this and we're going to do that and they come over in these events and things, it looks like it doesn't seem that these big tech companies are looking to do that. So you can see right here, we cannot rely on goodwill. And an anthropic spokesperson said the company is in active discussions with both the UK and US institutes about testing and an OpenAI. They just didn't respond for a request for comment. It said here that it is concerning that the promise of the pre release of testing seems to be failing to materialize in any meaningful way, said Andrew Strait. And he said that this means we will see new models being released with no independent assurance or understanding of their safety from risking harm to people and society. And it's clear that it should be now abundantly clear that we cannot rely on the goodwill of AI companies. Voluntary agreements are no substitute for legal mandates to provide this access. So he's basically stating that look, we can't trust these companies just to say, hey, we'll give you guys that. Of course there needs to be legal mandate if we actually want things to get done. The letter continues to say so long as there is no effective government oversight of these corporations, current and former employees are among the few people who can hold them accountable to the public. Yet broad confidentiality agreements block us from voicing our concerns except to the very companies that may be failing to address these issues. Ordinary whistleblower protections are insufficient because they focus on illegal activity, whereas many of the risks we are concerned about are not yet regulated. Some of us reasonably fear various forms of retaliation given history of such cases across the industry and we are not the first encounter to speak about these issues. And basically what they're trying to do is if you don't know, there is a situation on hand where a former AI research not researcher, someone that worked on governance at OpenAI and someone that knew a lot about AI at OpenAI. Basically Daniel, who was working at OpenAI and I'm going to show you guys the post, you basically were, you know, the long story short was this that if you wanted to leave OpenAI, and if you wanted to criticize them, you basically had to give up a large amount of money in the form of stock compensation. And we know that OpenAI is now, you know, like a hundred billion dollar company, so you're basically giving up around a million dollars. Okay, So a lot of the people who are speaking out on OpenAI, they were basically in a situation where their hands were tied with, well, they weren't able to criticize OpenAI in any meaningful way. And they're basically stating that, look, these confidentiality agreements block us from voicing our concerns, except to the very companies that may be failing to address these issues. And this was something that is pretty bad for the industry because if there is a reason why someone leaves the company and it is detrimental to society, maybe they're doing something wrong. There needs to be some kind of safeguards so that those employees can voice their opinions and in a way where, you know, they can still actually maintain whatever equity they have or at least whatever job that they have. So it's something that could help the industry in terms of transparency. You can see right here that it says that many employees could lose out on millions of dollars if they refuse to sign. So essentially you can see right here, on his way out, Mr. Coco Tajlo refused to sign OpenAI's standard paperwork for departing employees, which included a strict non disparagement clause barring them from saying negative things about the company or else risk having their vested equity taken away. Many employees could lose out on millions of dollars if they refused to sign. Mr. Coco Taglo's vested equity was worth roughly $1.7 million, which amounted to the vast majority of his net worth, and he was prepared to forfeit all of it. And this was pretty, pretty crazy because a lot of people were wondering, why don't certain OpenAI employees, you know, talk more freely about the stuff when they leave the company? And this was something that Sam Altman even actually discussed on Twitter. He said in recent regards to the stuff about how OpenAI handles equity, we never clawed back anyone's vested equity, nor will we do that if people do not sign a separation agreement. Vested equity is vested equity, full stop. There was a provision about potential equity cancellation in our previous exit docs, which, although we never clawed anything back, it should never have been something we had in any documents or communication. This is on me, and one of the few times I've been genuinely embarrassed running OpenAI. I did not know this was happening, and I should have, which is pretty crazy. That The CEO didn't know that this was happening. That is just, you know, I don't know if there's a lack of foresight or because he's doing so many things, but that's pretty crazy. And then he says the team was already in the process of fixing the standard exit paperwork over the last month or so. And if any former employee who had signed one of those old agreements is worried, worried about it, they can contact me and we'll fix that too. Very sorry about this. However, you can see here that according to leaked messages and documents obtained by Vox, senior leadership at OpenAI, including Altman, were very much aware of these equity clawback provisions and signed off on them. And some of the crazier things that are on this article by Vox, they actually discuss the high pressure tactics at OpenAI. It says throughout the hundreds of pages of documents leaked to Vox. So basically Vox actually got like a lot of leaked documents. So this is something that was really, really interesting because Vox actually had the scoop on what was going down at OpenAI. They said in the two cases the Vox reviewed, the lengthy, complex termination document OpenAI sent out expired after seven days. That meant the former employees had a week to decide whether to accept OpenAI's muzzle or risk forfeiting what could be millions of dollars. A tight timeline for a decision of that magnitude and one that left little time to find outside counsel. So it's pretty crazy because you're only giving someone a week to think about whether or not they're going to, you know, forfeit millions of dollars. I mean, it's pretty, pretty high in risk. And it says when ex employees ask for more time to seek legal aid and review the documents, they face significant pushback from OpenAI. They said the general release and separation agreements require your signature within seven days. And here's what they basically said. The later documents the company sent him, which Vox has review viewed, say if you have any vested units and you do not sign the exit documents, including the general release as are required by company policy, it is important to understand that among other things, you will not be eligible to participate in future tender events or other liquidity opportunities that we may sponsor or facilitate as a private company. In other words, sign or give up the chance to sell your equity. And here's where we get to a further part of the letter. Therefore, they state, we call upon advanced AI companies to commit to these principles that the company will not enter into or enforce any agreement that prohibits disparagement or criticism of the company for risk related concerns, nor retaliate for risk related criticism by hindering any vested economic benefit that the company will facilitate a verifiably anonymous process for current and former employees to raise risk related concerns to the company's board, to regulators and an appropriate independent organization with relevant expertise. That the company will support a culture of open criticism and allow its current and former employees to raise risk related concerns about its technologies to the public, to the company's board, to regulators, or to an appropriate independent organization with relevant expertise so long as trade secrets and other intellectual property interests are appropriately protected and basically this entire letter here is one where they talk about how there needs to be some kind of facilitation of ensuring that there is a healthy discussion of if these models are going off the rails, that we can actually warn the public about what's going on. Because right now these companies can pretty much do what they want and they're under no legal obligations. Of course, if something does go wrong, that is of course going to spark the next area of regulation. But people just want the ability to talk about these things without them having their entire career ruined and potentially some of their equity being lost. So let me know what you think about the right to warn about advanced AI systems. I think this is something that is remarkably important, but I genuinely highly doubt that other than anthropic, most companies that are now caught in this terminal race condition aren't truly going to sign this.