OpenAI Researcher BREAKS SILENCE "Agi Is NOT SAFE"

Take a look at what he said, because if the lead of probably the most advanced AI company in the world at the moment has left due to safety concerns, I think it's worth paying attention to. So he said, it's been such a wild journey over the last three years. My team launched the first ever reinforcement learning with human feedback large language model with InstructPT, published the first scalable oversight on LLMs, pioneered automated interpretability and weak to strong generalization. More exciting stuff is coming soon. So he does state that, you know the there's going to be some more research coming out of this, but trust me when I show you, like I've looked around and some of the things that I've seen, the developments are pretty shocking even to me. So he said one of the things that he did was he stepped away from this job because it's been one of the hardest things I have ever done because we urgently need to figure out how to steer and control AI systems much smarter than us. And I think the wording here is very precise because it's taken like 24 hours to write this thread and you can see we urgently need to figure out how to steer and control AI systems much smarter than us. And I think the key word here, urgently is something that shouldn't be overlooked because urgently means that we need to do this now. And I'm guessing that by, you know, when we look at the overall picture, OpenAI just isn't focusing on that. And I mean, this is probably the most reputable source of information when it comes to this kind of technology. Because I don't think there's anyone you can ask about aligning super intelligent systems that's going to have as much information as the people working in OpenAI are. And that's simply because OpenAI are truly ahead of the competition. And these people who are working on super alignment, they are, I'm guessing, frequently there in those spaces looking at how these advanced systems are evolving and how they are moving. And if he said we need to urgently figure this out, this is definitely a cause for concern. And a lot of people have mocked the AI safety crowd and said E ACC EAC or E Accelerate, whatever it is, you know, the acceleration thing. But I think there was a tweet that I retweeted and I think this tweet, it should be retweeted more because it's very, very important. And there was something about the fact that safety culture has kind of become this thing synonymous with world culture, and they're just simply not the same Safety is something that we should all be focusing on because the impacts, like when you truly do look at the way, how you classify how dangerous AI are, the things that it could do, it could literally impact every one of us. And not just like super alignment, super intelligent AI, the biorisks, the social risks, the wealth disparity, just a million different things that I've looked at. It really is something that's important, but I guess we might actually have to find out the hard way. So he says, I joined because I thought OpenAI would be the best place in the world to do this research. However, I have been disagreeing with OpenAI leadership about the company's core priorities for quite some time, until we finally reached a breaking point. So it's clear that over the time, at his time working at OpenAI, there have been some clear things that he may have brought up and during that time it seems that they weren't taken seriously. And I think that is rather interesting because he says, you know, I've been disagreeing about the company's core priorities for quite some time. So this clearly shows us that this wasn't just a one time thing. This isn't just he woke up and he was like, you know what, I quit. This has been a long standing thing. He's been disagreeing with the leadership for quite some time. And it seems that now there is a breaking point to which it's like, you know what, Enough is enough. I've said my concerns, I've tried to fix things and it seems that what we have here is a situation where it doesn't seem possible. So he thought, you know what, the only thing that I can do is I completely have to step away. And that's completely understandable. But like I said before, this is still concerning because for the top safety researchers to reach a breaking point and then have to leave, it kind of brings into the question, what on earth are they actually prioritizing OpenAI for some of the most esteemed and some of the most knowledgeable people on AI safety to be leaving and just completely disbanding. He says, I believe more of our bandwidth should be getting spent ready for the next generations of models on, on security, monitoring, preparedness, safety, adversarial robustness, super alignment, confidentiality and societal impact and related topics. And I completely agree with this because the ramifications of AI are quite hard to quantify because there are knock on effects of knock on effects. And that is something that is very, very hard for humans to predict because there are just unintended consequences of any technology, for example social media, it causes all of these body issues, it causes, know, social separation, it causes depression, it causes anxiety. There are a huge range of things that you literally just can't predict. And I think it's important to at least, you know, be focusing on this as more of a priority than maybe they are. And I get, I get it. You know, the thing is, is that, you know, when trying to run a company like OpenAI and you've got this terminal race condition where OpenAI are competing against another billion dollar company like Google. The problem is, is that these companies are pushing themselves to get out systems in a faster and faster race. And OpenAI has committed to iterative deployment. But I think the problem is that it's a winner takes all market. And because OpenAI understands that, I think they're just trying to speedrun it to AGI because maybe they believe, and I don't think that this is probably right, but maybe they believe that once they get to AGI, they can use it to probably solve everything. And he says that these problems are quite hard to get right and I'm concerned we aren't on a trajectory to get there. So basically saying that, look, the current strategy that is going on at OpenAI or where they work, working on it's currently not working. Okay. And he says, you know, the trajectory that's going to get them there, like the path that they're going to take currently it seems like they don't have that path. So I mean, it's pretty concerning for him to resign and say that. Like that's genuinely like, you know, before when Daniel, the person who is also at OpenAI said some concerning things, I think it also was pretty concerning because he gave up a pretty sizable stake like he gave up his stock compensation and order to speak about this. And something that was only revealed today was that if you leave OpenAI and you want to negatively talk about the company, you can only do so if you give up your stock compensation. So the fact that a previous researcher actually did that just goes to show that some of the stuff really does need to be talked about. So he's basically saying that, look, I'm concerned we aren't on a trajectory to safely get there, which is definitely concerning. To me as someone who's on the outside of all of this, just looking in like researchers are leaving, this is pretty, pretty concerning. So he says, over the past few months my team has been sailing against the wind, which is basically just a metaphor saying that, you know, he's trying to move forward, but there are forces pushing back against them. And sometimes we were struggling for compute and it was getting harder and harder to get this crucial research done. So one of the things that most people didn't know was that with OpenAI there is a compute shortage. And they actually spoke about this quite a lot, which is why they partnered with Microsoft for the $10 billion deal. And the problem is, is that Super Alignment, they, in the initial blog post, what they said was, they said that Super Alignment, they were supposed to get like 20% of all of OpenAI's compute, leaving the 80% for remaining stuff. But I'm guessing that they said, since they said they were struggling for compute, I'm guessing that the 20% that had been previously allocated may have not been able to be distributed. And they said it was getting harder and harder to get this crucial research done. And I mean, when we look back on it, the Super Alignment team, I think they only posted maybe one or two blog posts since their inception and it's been a couple of months. And I'm not stating that, you know, you can do, you know, that that kind of research really easily because of course it's pretty, pretty hard to do that. But I'm guessing that maybe that's just because they just didn't have the compute in order to, you know, find out how these systems worked and of course do the, do the needed research, which is surprising to say the least, because they had an agreement on how they were going to do things. And it seems that, you know, it just didn't happen. You know, the proof is in the pudding. They were struggling for compute and it was getting harder and harder to get crucial research done. And he goes to continue to say here that building smarter than human machines is an inherently dangerous endeavor. OpenAI is shouldering an enormous responsibility on behalf of all of humanity. So this is something that we do know, and I've explained it in videos before, basically if you build something that is smarter than you, and we can use the example of, you know, let's just say, for example, chimps to where we are. So, for example, if we look at chimps, compared to us, we're only marginal, we're only marginally smarter than chimps. But the thing about that is that we are now, you know, like sending rockets into Mars and stuff like that. So it's like, you know, if we build something that is truly smarter than us, are we even going to understand what it's doing and are we going to be able, you know, be able to stop this creature, this tool, this whatever it is, this conscious entity, whatever you want to call it, are we going to be able to stop it from doing what it wants to do? And are we going to be, you know, basically see even coming? Because we could find ourselves in a very unintended consequence. And you know, if we take the previous example, like comparing us to chimps, I know that's not probably the best example, but so I can think of right now. But the point I'm trying to make is that making something that's smarter than you has never historically been favored out. Like usually when a smarter species comes along, whatever they want is the status quo. And the previous, you know, people who were smarter are, or, you know, the people who are, you know, less intelligent, they genuinely or genuinely just have to succumb to whatever the status quo is that's set by the more intelligent beings. And I think that that is a pretty concerning thing when we are talking about rapidly trying to develop these smarter tools or smarter beings that you know, really could present enormous risks and enormous dangers. And he says but over the past years, safety culture and processes have taken a backseat to shiny products. So he's basically saying that, you know, the safety culture has, has pretty much taken a backseat to the newness of all of the things that are being shipped within OpenAI. I'm guessing the newer models, some of the new features, plugins, whatever it is that people do want, I'm guessing that's taking a complete backseat to safety. And I'm guessing that is of course, because this is no longer just a research organization, it's now a business and a private company which completely changes the kind of ecosystem on how it grows and the decisions they make for long term growth. So I think that is going to be key there. Now in addition, he says we are long overdue and getting incredibly serious about the implications of AGI. We must prioritize preparing for them as the best we can. Only then can we ensure AGI benefits all of humanity. And he's basically saying that, look, we're long overdue. So this is something that I didn't want to see. Because if he's saying that we're long overdue, I'm not saying that they do have AGI, but I mean what this means is that the timeline that we're on now is that like we are playing catch up right now is basically what he's saying. Because if we're long overdue, it means we're past the point of where we should be, which means that we need to kind of like, I guess, as some people would say, you know, pause the AGI development so that we can catch up with where the relative safety research needs to be. Now what's crazy about this is that I made a video yesterday and I genuinely thought that they had probably solved this because if the researchers are leaving, I'd presume that they thought that, okay, our mission is complete, we solved the issue. And then I'm guessing that they'd gone on to work on other projects. But with this information, it seems that that is not the case. It seems that, you know, that these researchers weren't able to actually get their work done. And because of that they just decided that, you know what, it's time for us to go somewhere else. And he's stating that we must prioritize preparing for them the best we can because of course, the AGI implications aren't quite stark. Like, I don't think people are really going to take this in. Like, I think this video might get a few views. And the thing is that like when you have the top leaders of the super alignment, you know, and in the open eyes division, like leaving, okay, stating that, look, these guys aren't focusing on AGI, they aren't focusing on safety. You know, we tried, but they just aren't prioritizing it and you know, we don't want to be, be a part of this. I think that's like an alarm belt, you know, that ringing and it's like, okay, something clearly needs to be done. And I'm wondering now if, you know, there's going to be some kind of government intervention. Because whilst yes, right now OpenAI is just a private company, they can do what they can do. I'm wondering at the point they start to get these super powerful systems that the government, you know, starts to intervene as like, okay, this is like some kind of, you know, national security risk. Because if you guys aren't focused on safety and these systems go rogue, then what on earth does happen? And I'm not saying that they can, you know, hack the grid and all those kind of systems like there, you know, some kind of like sci fi world. But I'm stating that, you know, the societal implications of releasing such systems. I'm wondering what is truly going to happen. And he's stating here that OpenAI must become a safety first AGI company if they are to succeed. And this makes sense, okay, this completely 100% makes sense. Because if they are to succeed, they do need to become a safety first AGI company. Because if there are any unintended consequences. We're gonna, we're gonna unfortunately see it come out of OpenAI first and it's definitely gonna impact some people. I honestly hope I'm not one of them, but definitely, I do think unfortunately that is gonna happen, which is why all of these people are, you know, urging us to, you know, know, focus on safety. And I mean, and if it did happen, would, would I even be surprised? Like if something bad did happen with AGR related development, would I even be surprised completely not. Because another thing that I didn't even realize, okay, and I did kind of speculate this yesterday, but was that OpenAI, the team actually did completely dissolve. So it says OpenAI dissolves team focused on long term AI risks. Less than one year after announcing it, OpenAI has disbanded its team focused on the long term risks of artificial intelligence. A person familiar with the situation has confirmed to CNBC. The news comes days after both leaders, OpenAI co founder Ilya Sutskever and Jan like, announced their departures from the Microsoft backed startup. So the team disbanded. Guys, like I said in a previous video where I showed you guys the members that were on previous research papers and how they were contributing to the development of the actual research, them actually not being here is a pretty, pretty surprising thing because if the team has disbanded, that means that there is no one currently working on Super Alignment. And I'm guessing that there will be some statements, you know, coming soon because this is a really big headline. Like it might seem understated right now, like it might not seem that it's a big headline, but trust me guys, this is real, real important news because I think government agencies and other agencies are going to start to look at OpenAI and they're going to start to think, okay, what on earth is going on here again? Because you know, we've got a safety team that just completely disbanded. We've got Ilya Sutskever leaving, we've got this guy leaving, you know, what on earth is going on here? So I think that this is pretty, pretty interesting with us to how the development of this will continue. And Sam Altman did actually respond to this by stating that I'm super appreciative of Janlike's contributions to OpenAI's alignment research and safety culture. And I'm very sad to see him leave. And he's right that we have a lot more to do. We are committed to doing it. I'll have a longer post in the next couple of days, so I'm guessing that there's Going to be some kind of new team formed. I'm not sure what this new team will be. Maybe it's going to be focused on completely different, you know, versions of safety. But I'm guessing that Sam Altman has realized that. Look, people like Ilya Satskova and Jan, like, can't just leave without him stating something because it really does look bad. Like on the industry as a whole, it really just looks insane if you have someone that's just leaving, especially for such an integral part of your company. Safety is something that they always spoke about, that they would prioritize. So someone leaving is of course not good. Like the team being disbanded to do what they do might prompt further criticism from wider people from the industry. Now, there were a few tweets and I can't verify this tweet or not, but the tweet was deleted. This is from Rune, an OpenAI employee. And he says, my feeling, I speak for nobody but myself is super alignment. Got plenty of attention, but then Ilya blew the whole thing up and this post has been deleted. I just want to say that this is complete, of course, speculation. Tweet was deleted and this is just his personal opinion. But it's pretty crazy because right now OpenAI seems to be imploding again and this is not something you want to have from arguably one of the most advanced companies in the world. So it's pretty incredible that all of this is going on. You know, people are saying that, you know, they didn't get any of compute. I mean, it's just, it's just pretty crazy, like all of this is going on as people would say, what is OpenAI without the drama. And Elon Musk, of course, he does have, you know, a certain opinion. He says by implication, safety is therefore not a top priority at OpenAI. And he says OpenAI must become a safety first AGI company. And Elon Musk is basically saying that if Jan, like is stating that they must become a Safety First AGI company, and if he's stating that they must do this, then that means that currently it's not a top priority, which is, I don't know, it's. It's surprising. I mean, I think Sam Altman is going to think of something because I think he's a very, very intelligent individual. I think whatever's happened over the last couple of days, there's going to be some significant developments, including probably hires, including probably some AGI board. I have no idea. But yeah, I know that OpenAI does work on safety, but the fact that the Super Alignment team is gone. That's not good at all. So let me know what you guys think about this. I think that this is a significant development, a really, really significant development, because so many people have left OpenAI. And the thing is, I wonder if OpenAI will be the last AI company around, because the only thing. And the reason I'm saying this because I understand that business and AI are two hard things to kind of merge, because if you're focusing on safety, the problem is, is that in the AI world, it moves so quickly. And if it moves so quickly, you can get left behind. And if you get left behind, your company goes out of business. If your company goes out of business, you literally don't have time to work on safety because, you know, you're insolvent, just like stability AI is. And of course, they're not working on, you know, dangerous systems. But I think that it's a difficult thing to do. But I do think that they can probably get it done. And I do think that, you know, with entrepreneurs, what you try to do is you try to move fast, you try to break things, you try to, you know, break into new industries, you try to release new products, and that inherently doesn't align with how safety testing works. Safety testing, you know, it's still why we don't have GPT5. Safety testing is something that does take an extensive amount of time because if things aren't safe upon release, the entire industry is going to be like, whoa, how could this be released? Yada, yada, yada, and then the entire industry kind of slows down. So I think it is not easy to do this. But I think what there is, is, you know, like I said before, that is that there is a giant compute struggle at the moment for OpenAI. They are probably working on so many different things and so many different projects. You know, like we've seen with GPT4O, that was something that most people didn't even predict. It just came out of nowhere. They've got Sora, they've got GPT5, they've got arguably some agentic systems, some other stuff as well. So I'm guessing that, you know, with everything that they're trying to push and all the innovation they're trying to do, safety just kind of took a backseat. So, I mean, it will be interesting to see how this development comes from Sam Altman, because I know that there's going to be some key updates. But let me know what you guys think about this. Are you guys concerned? I'm sure that people who are focused on AI safety are going to clearly be like, okay, we told you so. We told you that this is the thing that needs safety. But let me know what you guys think about this, because I think this is pretty important news, and I will see you guys in the next video.