Anthropic's Chilling 18-Month Warning: AI Apocalypse in 18 Months

So a crazy article has come up from Anthropic that I completely missed. Basically Anthropic is stating that we have 18 months, just 18, before something catastrophic happens within AI. Now currently the date for this article, not the date of recording this video, was 31-10-2024, which means 18 months from today, April 30, 2026. Now that is actually a lifetime away when you think about it in terms of AI terms, because that is a few months away and essentially a few months in AI. We all know how crazy a few months can be. So you can see here, it basically says that look, increasingly powerful AI systems have the potential to accelerate scientific progress, unlock new medical treatments and of course grow the economy. But along with the remarkable new capabilities of this AI come significant risks. And governments should urgently take action on AI policy in the next a billion months. The window for proactive risk prevention is closing fast. Basically what they're trying to say here is that the way how AI models have been rapidly advancing, we might be losing time to get that window down where we can actively implement guardrails to actually prevent AI getting out of control. And if you want to know how crazy AI development just has gone in the past couple of days, take a look at this, where it actually looks at how we've gone from GPT1 all the way from GPT4 now to this innovation area where we're going to be having scaling breakthroughs and even potentially superintelligence.

GPT1 which was nothing special to O1 which is sort of PhD level in math and computer science. And then O1 I think kicks off this new phase of we sort of think about as an innovation for era, which is basically now until super intelligence. We'll see if that's six years or maybe, maybe even a bit less than that. The hallmark I think here is that we're spending $200 billion on the models. We probably can't spend a lot more than that, probably can't spend 200 trillion on the models. So there's only limited amounts of scaling left in terms of orders of magnitude. And so we need corresponding innovations to come alongside. Definitely advanced reasoning and test time compute is one of those. And we think there's probably a few more handful of others that will get us, you know.

Now the article continues to state that narrowly targeted regulation can allow us to get the best of both worlds. Realizing the benefits of AI while mitigating the risks and of course dragging our feet might lead to the worst of both worlds. Poorly designed knee jerk Regulation that hampers progress while also failing to be effective at preventing risks. Basically what they're stating here is that, and I do believe this, I'm not just reading this off the script, I actually genuinely believe this is that if we do this really slowly, what's going to happen is that there's going to be knee jerk regulation, which basically just means that something will occur first, like the tragedy will occur first and then the regulation will be built around such tragedy so that it cannot occur again. And I don't think we want to do regulation like that. We would rather prevent the issue from ever happening than for us to wait for a catastrophe and then of course, for us to then construct the regulation around that. Now they also talk about how what's going on inside AI companies is pretty crazy. So it says in the last year, and this is where they talk about urgency, AI systems have grown dramatically better at math graduate level reasoning and computer coding, along with many other capabilities. And inside AI companies we see continued progress on as yet undisclosed systems and results. Now, essentially what they mean by this is that this means that they are looking at systems that have not been publicly announced or revealed. And, and this could include proprietary AI models or technologies like advanced AI systems that are in development but kept confidential for competitive, ethical or security reasons. Or they could be talking about breakthrough results, which is where they have significant progress or discoveries in AI capabilities that these companies are not ready to share with the public. And of course, possibly because they are being tested or because revealing them could pose certain risks. So this is where they're stating that these advances offer many positive applications, but progress in these same broad capabilities also brings with it the potential for destructive applications, either from the misuse of AI in domains such as cybersecurity or biology, or from the accidental or autonomous behavior of the system itself. Now they're basically stating here that these advances that we're going to see with these future models are offering all of these insane applications. But the thing is, is that because we're currently building these general models that are able to do quite a lot of different things than they generalize to other categories. The problem is, is that this progress also goes to those areas for misuse. So basically what they're stating here is that, you know, the broad capabilities also brings with it the potential for destructive applications, either from the misuse of AI in domains such as cybersecurity or biology, or from the accidental or autonomous behavior of, of the AI system itself. So that last line there is a little bit more chilling considering the fact that 2025, 2026 is going to be the year apparently for advanced AI agents. So the craziest thing about this all is that they state that in the realm of cyber capabilities, models have rapidly advanced on a broad range of coding tasks and cyber offense evaluations. And take a look at this guys, because this, when I read this I was like, whoa, this is actually pretty crazy when we actually think about it. So it says on a software engineering bench, software engineering task models have improved from being able to solve 1.96% of a test set of real world coding problems, which was Claude 2 in October 2023, all the way up to now, it can do 49% just in basically a year I think because it says Claw 2 was October 2023 and then Claude 3.5 Sonnet in October 2024 is 49% guys. So can you imagine what October 2025 looks like in swe? There's probably going to be close to 90% guys, and that is going to be pretty incredible. I can't even speak. And it also says internally, our Frontier team has found that current models can already assist on a broad range of cyber offense related tasks. And we expect that the next generation of models which will be able to plan over long multi step tasks will be even more effective. So that's of course something that we know. When you give these models the ability to think for such a long time, they're able to come up with a lot more coherent responses and are able to find solutions to problems that they otherwise weren't able to. So I think among you guys, they, this is going to be something that's really, really crazy because if we're seeing that rate of progress on the software engineering progress benchmark in just one year, we can really start to think about, okay, what happens year after that, the year after that. I mean it's really going to get to that point where things get incredible. I mean this is why they're literally stating that look, within 18 months we're going to have really advanced AI systems and if we're not able to actually fix that right now and implement these guardrails, then things are going to be able to go off the rails. So overall what we're looking at is a situation that basically tells us that these systems are going to be even better than these current ones and it's probably going to happen a lot faster. Now they basically talk about here that they were also looking at the potential for cbrn, which is chemical, biological, radiological and nuclear misuses. And the UK AI Safety Institute actually tested a range of models from industry actors, including anthropic and and they concluded that these models can be used to obtain expert level knowledge about biology and chemistry. And for several models, replies to science questions were on par with those given by PhD level experts. So for those of you who aren't aware of what I'm saying, guys, they've got a situation on their hands where these models. The problem is is that pretty much every model has some kind of jailbreak. I'm not sure how you're going to get these models to not say certain things. Literally every time a model is released, I see someone on Twitter called Pliny and and he goes model jailbreaken, model poned. And basically there doesn't seem to be a real proper solution to the issue that is jailbreaking. And they're basically saying that look, models can be used to obtain expert level knowledge about biology and chemistry. And of course the knowledge that they're giving them isn't like GPT 3.5 level knowledge. It isn't just complete jargon, it's actually the kind of knowledge that is really good and is given by PhD level experts. So when we actually look at where we are now, the models that they've been testing, you have to think about this in the next three to five years because these models are going to get so smart that if you have that model available to the average person, we know that this is going to be some kind of issue. You can see here, it says about a year ago we warned that frontier models might pose real risks in the Cyber and CBRN domains within two to three years. Based on the progress described above, we believe we are now substantially closer to such risks. Surgical careful regulation will soon be needed. So basically what they're stating here is that look, when you've got all of this progress, the SWE bench going so crazy, we need to think about the fact that this is going to be some kind of exponential improvement and we need to be prepared for this kind of risk. Of course, you can see how AI systems have progressed dramatically in their understanding of the sciences. In the last year. The widely used benchmark, the GPQA, saw scores on its hardest section grow from 38.8% when it was released in November 2023, to 59.4% in June 2024 and to 77.3% in September, which is OpenAI01. Now the crazy thing about all of this is that human experts score 81.2%. So when we actually take A look at how the GPQA benchmark, which is a benchmark for understanding really hard questions. We can see that human experts scoring at 80% is pretty much going to be on par with what these systems have in the next iteration, are probably even surpassed them. And it says for now the uplift of having access to a frontier model relative to existing software and Internet tools is still relatively small. However, it is growing rapidly. As AI models advance in capabilities. The potential for misuse is to continue on a similar scaling trend. Right here the basic statement, look, compared to what current models are available, we actually have a situation on our hands where we can look and see that if this continues with this current trend, we're going to see a situation where these models are going to do a lot more than the available Internet. So here's where they state a year of Anthropic's responsible scaling policy. Grappling with the catastrophic risks of AI systems is rife with uncertainty. We see the initial glimmers of risks that could become real serious in the near future, but we don't know exactly when the real dangers will arrive. We want to make the critical preparations well in advance. And the thing is kind of scary, these are skin like, look, we see the initial glimmers of these risks that could become serious in the near future, but the problem is we don't know when these dangers are going to arrive. Now basically this is where they discuss their responsible scaling policy, which is how they manage to increase the capabilities of these models whilst keeping them safe. So they state that the first principle of the responsible scaling policy is that it is proportionate and the strengths of the safety and security measures increase in proportion with the defined capability thresholds that the AI systems meet, basically saying that look, they're not going to use any insane laws to lock up this AI, but they're just going to use something that is proportional to the kind of capability that is measured. So for example, let's say you have an AI agent that is able to do a bunch of crazy tasks. Maybe you can find that agent to be able to only do tasks in a sandbox or a work environment. So it's just something that restricts it in the areas where it needs to be restrictive. They also talk about the fact that the RSP should be iterative, basically saying that with every update they need to make sure that they update this regularly because of course AI is dynamic. And of course they're basically saying that, look, we're going to regularly measure the capabilities of their models and rethink the security and safety approaches in light of how things have developed. Here's where Anthropic actually talk about how they're going to internally document their findings and recommendations regarding the safeguards they've implemented. So it will be interesting to see, see what kind of things they talk about because sometimes what Anthropic will do is they will basically conduct these safety experiments and they'll be like, okay, we conducted a safety experiment and we found that Claude ran off and did this, he ran off and did that. So we're basically recommending that if you have a model that is this capability, make sure it doesn't do this. And I think it was like the other day that literally Claude went off to Google something that was pretty random. And I saw someone else have the same issue where the model just went off to do something on its own, which is of course, I guess an emergent capability, considering that we haven't really given these models the ability to control computers before. But now that this is a thing here, I think this is showing us where we kind of need to ensure that we have certain safeguards. Now I do wonder if this is going to become a real thing. They're basically stating here that currently we don't have any way to verify any AI companies adherence to responsible scaling policies or whatever plans that they do have because we don't know the outcomes of the training run. Basically they're saying that, look, if any AIs are internally going rogue, we don't have any kind of knowledge of this because these companies are extremely private and, and if they are doing it, we want them to be able to publish it. So that way we can actually see what is going on in these companies, at least on the risk side. And the ways that they're reigning in these models in terms of these risk evaluations. And they're basically stating, look, transparency alone doesn't guarantee robust policies. Because the problem is that yes, these companies might be open about what's happening, but they might just lie. They might just be like, okay, we've got a responsible policy where we check one output from the model and it could be an extremely weak policy that isn't thoroughly vetted. And of course they could release these models instead of a while that could have a potential catastrop catastrophe. These models do exist in the wild. Crazy things can happen if they're uncensored, unfiltered, and people can literally do what they want with them. Of course, if these companies have these relaxed security measures, transparency isn't going to solve that. But they also do talk about accelerating the AI industry. They talk about, of course they must not impose burdens that are unnecessary or unrelated to the issues at hands. And it says one of the worst things that could happen to cause the catastrophic risk prevention is a link between forming regulation that's needed to prevent risks and burdensome or illogical rules. Any bill or law should also be so simple to understand and implement. Complexity creates confusion and makes it harder to predict what will happen in practice. They're basically saying, look, they don't want any crazy, insane laws that's going to cause people to do insane loopholes and stuff like that. It should just be pure, simple and cut so that everyone can understand it and of course not be unnecessary. Now this is where they get into the FAQs and I think this is really good because I know a lot of people are going to have questions, maybe you've even commented something down below. But it says here that this post talks a lot about CBRN misuses and cyber risk and why not other nearer term risks like deepfakes and child safety? And this is a great question, but they say this post is not an attempt to address every possible safety problem posed by generative AI systems. Instead, it aims to lay out principles for grappling with some types of risks which aren't well addressed by regulation today and which show up in computationally intensive frontier models. We continue to address near term risks through things like our election integrity and work with other organizations. Basically it's like, guys, of course those things are still issues, but right now they're focusing on Genai because this is the one that is steamrolling ahead without any kind of regulations at all. Now of course they're basically stating, and I know most of you guys are going to ask this question, won't regulation slow down innovation and reduce our ability to compete with geopolitical adversaries? Basically saying that, look, do we want to slow down ourselves and let these other countries race past us in terms of technological advancement we're going to, or do we want to innovate and stay ahead? But it says here that even within the RSP framework, we have advocated for high flexibility in designing these tests because the science of AI risk evaluation is nascent and we don't want to create unnecessary burdens through inflexible rules. But it is unrealistic that any regulation would impose literally zero burden. And our goal should not be to achieve a large reduction in catastrophic risk. And they state that our goal should be able to achieve a large reduction in catastrophic risk for a small imaginable cost in compliance burden. Basically saying that, you know, overall they don't want to slow down the ecosystem. Ideally they want to speed it up and they just want to develop something that is lightweight, that is flexible, that's able to simply prevent catastrophic risk. That is the only thing that they are focusing on here, not unnecessary regulation, just focusing on preventing catastrophic risk. And remember, this is coming through at Anthropic, a company that does need the regulation not to hamper innovation. And of course most of you guys might be commenting about, you know, open source ecosystem and it says our view is that regulation of frontier models should focus on empirically measured risks, not on whether a system is open source or closed. Basically stating that, look, open source versus closed source, either way it doesn't matter. The only thing that should matter is if you test the model and if it comes up as something risky, you should of course implement some safeguards for said model. So with that being said, let me know what you guys think about Antopic. Basically stating that we have 18 months left to a potential disaster if we don't manage to fix some things. It will be interesting to see how the landscape plays out. But if you enjoyed the video, I'll see you in the next one.