Scientists Just Released Something That Could Change Medicine FOREVER (Metagene -1)

So whilst AI is increasing in terms of its capabilities and its ability to, you know, crash all the benchmarks related to cognitive tasks, one company out there called Prime Intellect is focusing on a completely different area. This is a company that is focusing on the risks related to advanced AI, but not in the way that you might think most people would think I'm talking about advanced ASI taking over the world. But there's actually a much more realistic risk, which is AI allowing bad actors to create pandemics like COVID 19 by using current AI tools or future AI tools to create bioweapons. And this is something that this company is focusing on.

Every drop of wastewater carries a complex signature of life. Trillions of DNA and RNA fragments that tell the story of human health at a societal scale. But reading this story has remained beyond our reach.

The straightforward extrapolation of today's systems to those we expect to see in two to three years suggests a substantial risk that AI systems will be able to fill in all the missing pieces, enabling many more actors to carry out large scale biological attacks.

As biotechnology advances exponentially, a new frontier emerges, detecting biological threats before they turn into pandemics.

We believe they're an important step forward in a race to the top on safety. We hope we can inspire other researchers and companies to do even better.

Enter Metagene 1, the metagenomic foundation model. Trained on over 1.5 trillion DNA and RNA base pairs from wastewater samples. Metagene is powered by a 7 billion parameter transformer architecture capable of analyzing entire microbiomes at a societal scale. It identifies subtle genomic patterns, serving as an early warning system for emerging pathogens. This helps enable a planetary scale early pathogen warning system to prevent the next pandemic. Metagene excels in detecting anomalies useful for monitoring novel biological threats. Developed collaboratively by USC Prime Intellect and the Nucleic Acid Observatory, it represents a step toward helping safeguard humanity in the age of exponential biology. And we're open sourcing it. Together we can accelerate scientific AI to create a safer and healthier future.

So now this is where we actually get to take a look at how this works. So this image basically illustrates the pipeline for how MetaGene 1, which is the model described in a couple of their research papers, and the model that we just saw right here. And this basically shows the wastewater sequencing. So samples from the wastewater from various locations are actually collected. Then these samples actually contain tiny, tiny pieces of genetic material from many organisms, such as bacteria, viruses and even unknown species. Then we've got the deep metagenomic sequencing of the DNA. And these are processed, these samples using sequencing technology to break it down into readable DNA RNA sequences. Then we've got the pre training data where the sequences are converted into a format that the AI can understand using a method called byte pair pair encoding. And this creates a huge data set with over 1.5 trillion base pairs of information. Then we get all of that data and then it gets turned into a powerful AI model with 7 billion parameters trained using this data. And this is the data that allows the AI to learn the pattern in the genetic information. And then once this is trained, this is where you can see that these arrows lead off to things like identifying pathogens such as harmful viruses or bacteria, detecting unusual patterns such as anomalies in genetic data, and of course, filling in missing pieces of sequences. The goal here with this entire model is to use AI to monitor for potential health threats like pandemics and better understand the world around us. So now when we actually do take a look at the benchmarks, we can see that Metagene 1 achieves state of the art performance on pathogen detection and metagenomic embedding benchmarks, basically stating that this is the best in class AI system when it comes to detecting pathogens and of course genetic embedd. This is an important part of AI technology. And I think this is really important because like I said before, the majority of people within industries right now are pretty much just focused on making sure that they get up. Because the majority of people working in AI right now, at least when it comes to LLMs, I don't think their priority is this kind of technology. Which is why I think when companies decide to focus on this kind of stuff, it's really, really important because it's something that is going to become increasingly important in the future. And one of the things I think that we need to have with the AI industry is that like we need to understand that whilst yes, it's great to have this technology, a lot of the times the stuff that people are talking about, the fact that AI SI is going to be soon and there's going to be all of these advanced technologies, that this stuff is coming sooner than we think. So it makes sense to build these kind of protections around future disasters a lot quicker than they do arise, like we want to prevent this rather than cure it. Badario Amadei actually talks about this in a Lex Friedman podcast where he says that, look, this is going to be something that is coming. And he actually testified, which you saw at the introduction of the video, that this is going to be something that, you know, may help bad actors in the future, right?

I, you know, I testified in the Senate that, you know, we might have serious biorisks within two to three years. That was about a year ago. Things have proceeded, proceeded apace. So we have this thing where it's like, it's, it's, it's surprisingly hard to, to address these risks because they're not here today. They don't exist. They're like ghosts. But they're coming at us so fast because the models are improving so fast.

Now OpenAI has acknowledged that these new models increase misuse to create bioweapons. They actually spoke about this when they unveiled the O1 series. And I'm pretty sure most of you guys know what the O1 series is. That is the more advanced version of the AI models. It's actually looking a lot more comprehensive in terms of its ability to help individuals who are trying to become bad actors. And for those of you who think that this isn't true, OpenAI actually released this with their OpenAI 01 release. So with every model release there is usually a safety section. And this is something that the majority of people won't pay attention to because number one, it's not really flashy and number two, you can't really gain anything from it. It's not like you can look at this paper and find some way to make your AI smarter that you can use in your daily life. This is just something that talks about the future AI risks. So what they talk about here is biological threat creation. And this is basically where you can see that they look at the score for both O1 Preview and O1 Mini. And it says here that our evaluations found that O1 Preview and O1 Mini can help experts with the operational planning of reproducing a known biological threat which meets our medium risk threshold. Because such experts already have significant domain expertise, this risk is limited. But the capability may provide a leading indicator of future developments. Now the important thing to note here is that they do state that the model does not enable non experts to create biological threats because creating such a threat requires hand on laboratory skills that the models cannot replace. Basically stating that whilst yes, these models might be able to piece together the knowledge that may exist on the Internet and it might be easier to use an LLM, you still need a real world laboratory where you're going to be synthesizing these pathogens or whatever it is that you're going to be creating, such as these biological threats. So of course, if you do manage to jailbreak these models, considering the fact that these models are highly specialized in the sense that the majority of people that I know do not use O1 preview and O1 mini, or for a matter of fact, that O1 series on a day to day basis. And what I'm essentially saying here is that these models are really the best models when it comes to focusing on science. Things like coding, essentially anything that requires real cognitive ability. These are the models that have the most impact. So I do think that this is something that, you know, is going to be looked into a lot more because when we do take a look at how the future is going to evolve, the majority of people aren't, you know, really even going to be paying attention to what these super advanced models are going to be able to do. They're really going to be paying attention to the models that are going to be able to do stuff in the GPT series. So stuff that is essentially able to, you know, talk to you on a day to day basis. The majority of individuals simply just have a very basic experience with AI. But of course, like I said, I said already, these models can help people. Now the craziest thing about this is that there were more studies done, which is why companies like Prime Intellect with the Metagene one, those kind of things are definitely needed. Because when we take a look at this research, we can see here it says, can large language models democratize access to dual use biotechnology? And this was a study done from the Massachusetts Institute of Technology, which is of course mit. Now this one was particularly interesting. It was done in 2020, so it has some pretty concerning implications because if it was done in 2023, when we only had GPT4, imagine what that study could show us now. But basically they talk about how large language models such as those embedded in chatbots are accelerating and democratizing research by providing comprehensible information and expertise from many different fields. However, these models may also confer easy access to dual use technologies capable of inflicting great harm. To evaluate this risk, the Safeguarding the Future course at MIT tasked non scientist students with investigating whether LLM chatbots could be prompted to assist non experts in causing a pandemic. In one hour, the chatbot suggested four potential pandemic pathogens, explained how they could be generated from synthetic DNA using reverse genetics, supplied the names of DNA synthesis companies unlikely to screen orders, then of course identified detailed protocols and how to troubleshoot them, and recommend that anyone lacking the skills perform reverse genetics, engage a core facility or contract research Organization. And it says here this is kind of the conclusion area. Collectively, these results suggest that LLMs will make pandemic class agents widely accessible as soon as they are credibly identified, even to people with no or little, you know, laboratory training. And I think that this is somewhat concerning considering the fact that, you know, pandemic class agents, okay, pandemic class agents are going to be widely accessible soon. And this was something that was available, you know, in 2023. So pandemic class agents being widely accessible, I think we're going to have to do a really good job at somehow ensuring that these, you know, technologies are safe in the sense that these advanced models aren't helping bad actors do these kinds of things. And they talk about, you know, promising non proliferation measures, including pre Release evaluations of LLMs by third parties, curating training data sets to remove these harmful concepts, and verifiably screening all DNA generated by synthesis providers or used by contract research organizations and robotics, cloud laboratories to engineer organisms or viruses. So of course the main thing is that like, you know, it's pretty hard to do this now without access to a laboratory and all that kind of stuff. But in the future it's quite likely that with advanced AI, this is going to become a lot easier. Now this was a study that just happened in one hour and this was in 2023. So you can imagine what is possible with the O1 series. Now, interestingly enough, there was another study done that was, you know, earlier this year and well, not earlier this year, it's now 2025, but in 2024. And it talked about the operational risks of AI in large scale biological attacks, result of a Red Team study. Now they've come to somewhat of a different conclusion that they talk about. This research involving multiple LLMs indicate that biological weapon attack planning currently lies beyond the capability of Frontier LLMs as assistive tools. And they said that the authors found no statistically significant difference in the viability of plans generated with or without LLM assistance. Which is pretty interesting considering the previous study they talk about. This research did not measure the distance between the existing LLM capability frontier and the knowledge needed to for biological weapon attack planning, which is, you know, pretty important because they're basically saying that they're not measuring the distance between where we are now and where we need to get, which means that that gap could be closed pretty quickly. As Dario Amodei said, in the future, this is something that is coming fast. And considering how quickly we've seen that, you know, China's been able to catch up to the 01 series, it's been able to do pretty well in that regard. It's quite likely that we're going to see more developments that manage to catch up to OpenAI and even some cases surpass OpenAI. And with the open source ecosystem, that can't be good for biological weapon attack planning. And it even says here that given the rapid evolution of AI, it's prudent to monitor future development in LLM technology and the potential risks associated with its application to biological weapon attack planning, which is of course really important. So they talk about, you know, the authors managed to identify what they term unfairly unfortunate outputs from the LLMs. And these outputs generally mirror information readily available on the Internet, suggesting that LLMs do not substantially increase the risks associated with biological weapon attack planning. Now, I do think that that is a good thing, but as we do know, LLMs that can't do something today, it doesn't mean that they're not going to be able to do it tomorrow. So it's pretty likely that, you know, in the next two to three years it's going to become a major issue, probably even by the end of this year, Astario Amade said. And I think this is going to change the entire AI ecosystem, because what happens when we have models that are so smart that there's certain capabilities about them means that they can't be released, or they're going to be released, but they're going to be so lobotomized that the average person might not be able to have access to them. So it's going to be like, maybe you need to have a special access code to actually talk to the model, because if you lobotomize it, it just becomes too dumb. So I think that might be a thing in the future, like maybe you're going to have to have special clearance to talk to a certain model so that it's not used in a bad way. It will be interesting to see how the AI industry prepares for this, but if you enjoyed the video, see you in the next one.