OpenAI Former Employees Reveal NEW Details In Surprising Letter...

The California Senate Bill 1047, known as the Safe and Secure Innovation for Frontier Artificial Intelligence Systems act, is a legislative proposal aimed at regulating advanced AI models to ensure their safe development and deployment. Now, this has been one of the most controversial pieces of discussion going around the AI industry, and that's because there are many different things that that could really impact the future of AI, including statements from Anthropic, a few OpenAI whistleblowers and key industry figures. So in this video, I'm going to dive into all the key aspects and then we're going to dive into why this is such a contentious issue. So the key aspects of SB 1047 are as photos. The bill targets AI models that require substantial investment, specifically those costing over $100 million to train. It mandates developers to conduct safety assessments, certify that their models do not enable hazardous capabilities, and comply with annual audits and safety standards. There's also regulatory oversight. A New Frontier Model Division within the Department of Technology would oversee the implementation of these regulations. This division would be responsible for ensuring compliance and could impose penalties for violation, potentially up to 30% of the model's development costs. Now, some individuals have argued that bills like this are necessary to prevent potential harms from advanced AI, while critics are claiming that this could stifle innovation and concentrate power among a few large tech companies. The bill's language is considered vague, leading to concerns about compliance and liability for developers. And many critics, including many tech companies and AI researchers, argue that the bill's focus on the AI models themselves, rather than their applications, could hinder innovation and place undue burdens on startups and open source projects. They fear it could lead to a consolidation of AI development power and slow down progress in California. Now, today there was a letter from OpenAI whistleblowers in which they explain their reasoning for their position on this letter. And their positioning is driven by OpenAI's recent statements surrounding the letter. You can see that the Twitter Control AI, a Twitter account that is focused on controlling AI, and an organization that's focused on the safety aspects, tweets this OpenAI's Chief Strategy Officer Jason Kwon, last week said that we've always believed that AI should be regulated and that commitment remains unchanged. However, this week his statements were quite different. He says that the AI revolution is just the beginning and California's unique status as the global leader in AI is fueling the state's economic dynamism. SB147 would threaten that growth, slow the pace of innovation, and lead California's world class engineers and entrepreneurs to leave the state in search of greater opportunity elsewhere. And interestingly enough, Sam Altman has clearly stated that we do need air regulation. And this is him talking in October of 2020 about how these systems should be regulated.

You know, there's kind of a cohort in Silicon Valley that's very worried about what AI could do to manding it. Does that concern you at all?

For sure? I think it's going to be fine. I also think it's like very bad thinking to not take, take the apocalyptic downside very seriously. I am more optimistic than I used to be that we can get through this. I think just saying, like, oh, don't worry about it, it's going to be fine, is a very bad strategy. I'm like super proud of the safety team and the policy team that we have at OpenAI. And there's like very good technical work to do. We're doing some of it, others are doing some. We should probably all do more about how, how we build these systems in a way where they're very human aligned.

You know, how can we have some sort of, you know, way for people to feel confident that technical experts are taking those necessary safety kind of given the consequences of potential mistakes, or do you think, you know, people should be able to just trust.

No, I don't. I think there has to be government and we're trying to push for this as as much as we can.

Yeah. And how so far have you found the, the interplay between governments and I. Do you work with government regularly? You know, is there any sort of regulatory things you face or how does that. We do relationship work?

There's not much regulatory stuff yet on AI. I'm pretty sure there will be regulation in the not distant future. I really think there should be.

Now, I want to show you guys some key parts of this letter, but because there are some parts that need to be brought to your attention. As you may know, the people that wrote this letter, William Saunders and Daniel Coco Tajo, were people that actually worked at OpenAI and left due to safety concerns. This letter was released today. You can see it's August 22, 2024. And the letter starts by stating that OpenAI and other companies are racing to build artificial general intelligence, or AI systems that are generally smarter than humans and Its right, OpenAI's mission statement. And the company is raising billions of dollars to achieve this. And along the way, they create systems that pose a risk of critical harms to society, such as unprecedented cyber attacks or assisting in the creation of biological weapons. And if they succeed entirely artificial general intelligence will be the most powerful technology ever invented. And I'm going to highlight that because that is a clear statement that most people truly haven't grasped yet. Now you can see here that they said, we joined OpenAI because we wanted to ensure the safety of incredibly powerful AI systems that the company is developing. But we resigned from OpenAI because we lost trust that it would safely, honestly and responsibly deploy its AI systems. In light of that, we are not surprised by OpenAI's decision to lobby against SB 1047. It clearly states here that developing frontier models without adequate safety precautions poses foreseeable risks of catastrophic harm to the public. We are not the only ones concerned about the rapid advances of AI technology. And earlier this year, Science published Managing Extreme AI Risks amid Rapid Progress, a consensus paper from 25 leading scientists describing extreme risks from upcoming advanced AI systems. And Sam Altman agreed. He stated that the worst case scenario for AI could be lights out for all of us. So this statement right here is actually quite true. Sam Altman has stated on multiple different occasions, dangerous these AI systems could be and the kind of things that could happen. And every time I hear these individuals talk about the safety precautions at OpenAI, I do truly wonder how powerful the systems that they do have are and if the current preparedness framework that they're currently using in order to deploy models safely is actually going to be something that they stick by. Considering the fact that we are now in these terminal race conditions in which companies are forced to outdo one another in order to gain customer satisfaction. You can see right here there are some key issues to where they describe how OpenAI has previously not safely deployed their system. It says in the absence of whistleblower protections, OpenAI demanded we sign away our rights to ever criticize the company under threat of losing millions of dollars invested equity when we resign. For the company touting cautious and gradual deployment practices. GPT4 was deployed prematurely in India in direct violation of OpenAI's internal safety procedures. And more famously, OpenAI provided technology to Bing's ChatBotch, which then threatened and attempted to manipulate users. And OpenAI claimed to have strict internal security controls despite a major security breach and other internal security concerns. The company also fired a colleague in part about raising concerns about their security practices. That of course is referring to Leopold Aschenbrenner. Now you can see right here. They also spoke about how prominent safety researchers have left the company, including co founders, the head of team responsible for controlling smarter than human AI Systems said on resignation that the company was long overdue getting incredibly serious about the implications of AGI and that safety culture has taken a backseat to shiny products. While these incidents did not cause catastrophic harms, that's only because truly dangerous systems have not yet been built, not because companies have safety processes that could truly handle dangerous systems. We believe that there should be public involvement in decisions around high risk AI systems. And SB creates a space for this to happen. It requires publishing a safety and security protocol to inform the public about safety standards. And it protects whistleblowers who raise concerns to the California Attorney General. If a model possesses an unreasonable risk or capable of causing or enabling critical harm, it says it provides a possibility for consequences for companies if they mislead the public and doing so lead to harm or imminent threat to public safety. And it strikes a careful balance that protects legitimate IP interests. Now what's interesting about this is that they say here that OpenAI's complaints about SB1047 are not constructive and don't seem to be in good faith. They state that OpenAI, they don't protect whistleblowers and they do nothing to prevent a company from releasing a product that would foreseeably cause catastrophic harm to the public. And it's perfectly clear that they are not a substitute for SB 1047 as OpenAI knows as much. So basically what they're stating right here is, is that currently in the AI space we are waiting for a disaster to happen. I know many people think that the AI debate is one that is just pointless, but I mean, these guys do actually genuinely have a point about this. Companies have completely disregarded safety precautions in order to get products into users hands as quick as possible. And now with the future cycles ahead of us, we know that systems are going to be a lot more smarter, a lot more capable and thus a lot more dangerous. If this is true, looking historically at how companies have acted in the past, can we not see how releasing a product that would foreseeably cause catastrophic harm could be possible in the near to short term future? And I think this is, you know, plausible. It does say that we cannot wait for Condras to act. They've explicitly said that they aren't willing to pass meaningful AI regulation. And if they ever do, it can preempt California, it can preempt Sierra regulation. And Anthropic joined sensible observers when it worries congressional action will not occur in the necessary window of time. They basically state here that SB1047's requirements are things that AI developers, including OpenAI have already largely agreed to in voluntary commitments to the White House and Seoul. The main difference is that SB 1047 would force developers to show the public that they are keeping those commitments and hold them accountable if they don't. Now, of course, this is where they talk about the fear of mass of exodus of AI developers. And it says the fears of a mass of exodus of AI developers from the state are contrived. OpenAI said the same thing about the EU AI act, but it didn't happen. California is the best place in the world to do AI research. And what's more, the bill's requirements would apply to anyone doing business in ca, regardless of their location. And it's extremely disappointing to see our former employer pursue scare tactics to derail AI safety legislation. And here's the main point from all of this. They state that Sam Altman, our former boss, has repeatedly called for AI regulation. Now, when actual regulation is on the table, you he opposes it. And he said that previously, obviously they would support all regulation, but yet OpenAI opposes the even extremely light touch requirements in SB 1047, most of which they claim they voluntarily commit to raising the questions about the strengths of those commitments. Like I said before, this letter was written by William Saunders and of course Daniel Cocotaldjo, former OpenAI member of PolicyStar. So this is something that is rather surprising considering the fact that OpenAI have consistently shown their position when it comes to regulations surrounding AI, because they've seemingly been rather supportive. However, maybe when it's actually coming to it right now, for whatever reason, they're on the fence now. Interestingly enough, OpenAI for members are not the only people that have written about this letter and the issues that this kind of poses to the area. Here we can see Anthropic's letter that was written just yesterday. It does say a few things, and some of these that I want to bring to your attention are pretty incredible. So you can see right here, it says pros and cons of SB 1047. It says, we want to be clear as we were in our original SIA letter, that SB 1047 addresses real and serious concerns with catastrophic risk in AI systems. AI systems advancing today are gaining capabilities extremely quickly which offer both great promise for California's economy and substantial risk. And our work, and this is where it gets interesting, is that our work with biodefense experts, cyber experts and others shows a trend for the potential for serious misuse in the coming years, perhaps as little as one to three years. That's a crazy statement. But when you think about the pace of AI development, don't think that this isn't a possibility. And here's some of the key things about this paper, just the bits that you might want to pay attention to, where it says, here are some thoughts about regulating frontier AI systems. Regardless of whether or not SB1047 is adopted, CA will be grappling with how to regulate AI technology for years to come. And it says below we share our general perspective on AI regulation, which we hope may be useful considering both SB107 and future regulatory efforts might occur instead, or in addition to it. So basically, they're stating some of the problems here that most regulatory pieces fail to address. And one of the key issues that I've seen before is, of course, that AI regulation is driven by the speed of progress. Regulating things usually does take time. You've got different bills that you have to pass. You've got like, all these committees and, you know, honestly, just government nonsense, which is really slow. But I completely understand why it needs to go through so many different areas before a bill is passed. But the point here is that this doesn't work well with AI because AI is just advancing extremely rapidly. So it says here that on one hand, this means that regulation is urgently needed on some issues. We believe that these technologies will present serious risk to the public in the near future. And on the other hand, because the field is advancing so quickly, strategies for mitigating risk are in a state of rapid evolution, often resembling scientific research problems more than they resemble established best practices. We believe that this is genuinely one of the most difficult dilemmas, and it's an important driver of the divergence in views among different experts on SB1047. And in general, and it's rightly said trying to regulate something that changes literally every 12 months is, is, you know, insane. Like, it's just so hard to do that. And one resolution to this dilemma, which they've spoken about, is very adaptable regulation. In grappling with the dilemma above, we've come to the view that the best solution is to have a regulatory framework that is very adaptable to rapid change in the field. Which does make sense, it says, in terms of specific properties of an AI frontier model regulatory framework, we see three key elements as essential transparent safety and security practices. At present, many AI companies consider it necessary to have detailed safety and security plans for managing AI catastrophic risk. But the public and lawmakers have no way to verify adherence to these plans or the outcome of any test run. As a part of Them. Basically what they're stating here is that, look, these guys always say that, okay, we're going to test if these models pass a certain threshold, and if it passes a certain threshold, we're never going to release the model. But how do we know what is going on internally if they don't release these findings to anyone? They could simply release models that are completely dangerous if they haven't tested them in certain ways. Transparency in this area would create public accountability, accelerate industry learning, and promote a race to the top with very few downsides. Anthropic also talks about incentives to make safety and security plans effective in preventing catastrophe. Basically, what they're stating here is that, look, you can prescribe rules all day, but the main thing that you need to do is incentivize the right outcome. This is how humans are driven. If you incentivize someone with the right thing, they're always going to do what you want them to do.

You can see here it says, we believe it is critical to have some framework for managing frontier AI systems that roughly meets these requirements. And as AI systems become more powerful, it's crucial for us to ensure we have appropriate regulations in place to ensure they're safely. Sincerely, Dario Amodei, CEO of Anthropic. So overall, what we have here is a comprehensive view of where companies stand. It's clear that Anthropic does want regulation, but understands that even the current regulation, if it's proposed, isn't going to do what it needs to. And OpenAI seem to be edging towards not regulating their systems, surprisingly, considering their recent position regarding regulating AI systems. Either way, I do want to know if this legislation is going to be accepted or not. It seems to be rather interesting where everyone stands. Regulating AI is most certainly hard. Let me know what you guys think about air regulation. Do you think it makes sense? Do you think things like this are going to work? And if you guys do want to know about OpenAI's method of their safety, this is their preparedness framework Beta. And basically they do have an updated one, but I can't find it. But the long story short is that.

You know, if models reach a certain.

Level, they're basically saying they won't release them, which is why I've said that, you know, and the model, basically, if it gets high or critical on certain evaluations, they're not going to release them or.

Which is why I've said that before.

I don't think we're going to get frontier models in certain areas because it's going to be pretty hard to do that whilst increasing the knowledge of the model. So you've got cybersecurity, this one is biological and other threats, this one is persuasion and this is models autonomy. So this is basically identic behavior to go off and do stuff that's pretty insane. So I personally do believe that what we're walking into is a gray area because regulation is pretty difficult. But here's what I think is going to happen. I think that regulation will lag behind AI development and somewhere, somehow, something's going to happen and whenever it does happen, it's probably going to then force a regulation. Like usually what happens is in spaces that are pretty innovative since regulation can't keep up with what's going on and frameworks like this might not always be effective. Unfortunately, we're probably going to have to.

Wait for something bad to happen and.

Then once bad happens, we then put in regulation to prevent that regulation from happening again. For example, if we look at the tsa, the tragedies that happened in America, how it completely changed air travel, things like that, I do think, unfortunately, we're probably going to have to see another scenario like that. I do hope that that isn't the case. I would much rather regulation just allows these companies to also innovate and also not share their secrets, because I think that's the main thing that they're scared of. But I guess we'll have to see.