The Startup Trying to Save Us From AI Bioweapons- EP 74 Hannu Rajaniemi

All right. On this week's episode, we're going to go into the very pleasant discussion of what happens when AI systems or people equipped with AI technology try to make biotech weapons and how we might defend against this. Our guest is Hannu Rajaniemi. I did my very best with his very Finnish name. He's a science fiction author, a very well-known science fiction author. He is an academic. He's brilliant at math, physics, biotech. He's, he's run a biotech company for many years now and has a new one called Red Queen Bio. And its very mission is to try to, I guess, outthink the AIs, or at least the people using the AIs to do harmful things in the biotech realm. Red Queen is funded in part by OpenAI, and it's also partnering with all the major labs to try and explore these issues and how we could, how we could combat something going very, very wrong in the world and be some steps ahead. This conversation with Hannu made me think, well, our politicians are already quite terrible and they're likely ill-equipped to deal with the world of AI and rapid change that is upon us. And maybe we should have science fiction writers and thinkers as our politicians, because I think they might be the only people that can keep up. This is a really— Hannu is a fascinating guy. He's lived a very interesting life. He thinks about all kinds of wonderful new ideas and approaches to the world. His writing is brilliant. He's brilliant. You're going to enjoy this chat. As ever, please leave us a, a like or a review wherever you listen to this podcast. It makes a huge difference to people finding our show. We would really appreciate it. Hi, I'm Ashlee Vance. This is Core Memory. Off we go. Hannu. Hannu.

Yeah.

Yes. So nice to see you.

You as well.

We've met a couple of times out and about in San Francisco. And then, and then I guess we're in the fellow author club.

That's right.

Yeah. Well, thank you so much for coming in today. You, I think people are going to be blown away by— you're just one of those people who does such a variety of interesting things. So, there's plenty of places for us to go. I'm excited. Likewise. Well, as is the nature of doing these things, I research my guests, and then that always leads me often to want to start biographically, just because I get hung up on a couple of things that I find as I'm digging around. So, just so The audience knows we're going to get into some strange places with AI and biotech and all these things. But your background was fascinating. One thing when I was reading about you, it's, well, you're Finnish. It sounds like you grew up in quite a small town near-ish the Arctic Circle, I think.

Yeah, not quite right at the Arctic Circle, but pretty close. So a place called, place called Uligieska, so about 10,000 people.

I've been to Finland once. I was in the south and I did the whole sauna and I went in and I got tossed into the Baltic Sea. You know, it was winter. It was extremely cold, but rejuvenating. Yeah. Who— there's this Finnish actor who the— he— I was hanging out with him. He has the sauna in Helsinki. It's a rather big one. It's called like Lulio.

Oh, Lulio. Löyly, which actually means the steam in the sauna.

I could never get it right. But so he's an actor. They called him the—

Jesper Brackinen?

Yes, the Finnish Leonardo DiCaprio.

He is the Finnish Leonardo DiCaprio.

So we saunaed together and he kept telling me I wasn't saying sauna right. And I definitely couldn't get the löyly. So that's like the spirit of it, sort of, isn't it?

It is. Sauna is a sacred place and löyly is the spirit.

Yeah. Yeah. And so I'm reading about you in this— the Finnish culture, it's so— I feel like it's quite unique. And I really enjoyed my time there. It sounded like you were the only nerd in the town where you grew up, or at least this is what I was picturing as I read about you playing D&D and doing all these things in your mind in this small Finnish town up north.

Yeah, I was certainly, I don't know if I was the only nerd, but I was probably the nerdiest, the nerdiest of the nerds. And yeah, when you are a very nerdy kid, then you want to figure out how to make friends and how to connect with people. And it turned out that creating worlds for other people or helping them co-create worlds was one good way of doing that.

So were you doing role-playing games with your friends?

That's right.

Yeah.

So tabletop role-playing, so D&D, but also But also like quite a lot of other types of games, cyberpunk, like the kind of World of Darkness games, Vampire: The Masquerade, many, many other things.

How many people lived in the town where you grew up?

About 10,000.

10,000. Oh, so it was quite small. Okay. So you found enough compatriots.

Yeah. So we had a group of about half a dozen people. We, especially through high school, we played a lot together. But it was kind of a formative thing in some way, because I mean, you sort of learn to work together with, to create together with other people, something that I think does end up playing a role in also in like science and the startups. And then there's this aspect of thinking about systems and worlds, like working with a rule framework, kind of constrained creativity, but then figuring out how that gives rise to interesting stories and experiences. Yeah.

So my oldest son and my younger one to a degree, but especially my oldest one, huge D&D-er. He's also been part of a LARP group, which is— it's quite serious one. It's out in Palo Alto. It's been running for about 20-plus years. There's this guy Chris, um, who puts this— this— I mean, it's quite the production. You, you can— all these, I don't know, 60 to 100 kids show up, and he's got, he's got the fake swords and the costumes and the, the potions. And I know people make fun of this kind of stuff sometimes, but I find it amazing. These kids use their imagination non-stop and especially in the summer when they do it from like 9 to 5, they're just out here. It just seems like the most wholesome, wonderful thing. And then when I go on vacation with my kids, they— we roleplay like pretty much the entire vacation and their minds. It just reminds me how incredible their heads are because it's quite— it's like quite mentally demanding to do that over the course of an entire day, and I'm always amazed that it seems to spill out of them effortlessly.

So I actually think there's something very deep that you're describing there. There's, from very early childhood, we just have this inherent ability to imagine that we're someone else, so jump into the role of someone else and sort of summon these imaginary worlds around us. And that's, I think, very fundamental to being human. And it sort of gets squished later in life. But I think it does come out again in LARPs especially. I then actually also more in my early 20s in Finland did quite a bit of LARPing. And there's a whole, like another interesting thread is that the Nordic countries have an incredibly sophisticated LARPing culture. So the sort of people do entire sort of multi-day games. They've been organized on like one of these huge cruise ships that sail between Finland and Sweden that sort of has been used to simulate like a space station.

And everyone on there is—

Full on in character, in costume, like the ship has been sort of set up with props and like in some cases, like an internal whole like computer network to simulate what it's like to be on a Babylon 5 style space station. There's a Russian submarine, an old Russian submarine in Helsinki. There's also a famous LARP that was organized there where half of the players were deaf mutes in real life. And they were, role-playing the sort of last remnants of a destroyed planet of deaf mutes. And they met a species of cyborgs who then were offering them the ability to regain hearing, which would then, of course, destroy the kind of uniqueness of their culture. And so, yeah, so these are the kinds of things people have done in the Nordics. And I think it is an— and then there have been like refugee simulations and kind of more very serious, like, let's imagine the sort of aftermath of a civil war kind of scenarios, but with hundreds and hundreds of players lasting many days. And I think it is a powerful way to experience what something really feels like, even in a way that you don't even get from fiction. So one personal experience was one LARP I took part in was set in a nursing home. So you played an old person. I played an old person, so there were other players were sort of the nurses and the staff and so on. And what it really made me appreciate was how disempowered, like how weak and powerless and frustrated you feel in an environment like that. And it made me look at sort of the kind of nursing homes very differently.

Right. So you're feeling it on this visceral level.

That's right.

So, and you think this is— I have not heard about this. You think LARP is a, it's a bigger thing in the Nordics?

It is a massive thing in the Nordics. And it is much more in this, it's much more ambitious. Like it's trying to actually, you know, rather than give you a nice weekend outing, it's actually trying to examine some deep, deep questions. There are sort of friends of mine like Miki Pohjola, who also is an author, who's sort of deeply involved in this scene.

Where do you think this comes from? Is this part of the sagas and all this culture?

That's a good question. So Finland has a slightly different tradition from the kind of Viking sagas. It has this, the Finnish national epic is called Kalevala, which is this kind of sprawling mythological epic that has a, I guess it was, who was the poet who wrote about Tiawatha?

Longfellow.

Longfellow adopted the meter that Kalevala uses. So there is this very long-standing tradition of oral storytelling and folklore. But yeah, Does that directly, I don't know if that directly led to this, but I mean, Finland generally has had like an incredibly successful game design scene also, like a computer game scene. Rovio and Supercell are sort of big examples of that. So yeah, that's actually a bit of a mystery to me. I'm not entirely sure what, but I guess I would imagine that there is something to long dark winters and the need to entertain each other in some, There's something that's going on there.

I hung out with Supercell and Rovio when I was there. And then in Iceland, there's CCP Games, which makes this EVE Online. Have you ever seen EVE?

Yes, that seems to actually, I was talking to one of the creators of EVE Online and he mentioned that they actually at some point spent a lot of time understanding what made EVE Online successful. And the answer eventually turned out to be that by creating this kind of hostile, complex environment that really forces people to work together, they then form bonds and friendships.

Yeah.

That really is why they keep playing. And then he realized they had basically created a virtual Iceland.

Yeah. Yes. Okay. So I mean, I had this experience of being in Iceland, talking to the guys, seeing the games, and 100%. It's— for people who don't know, it's this incredible game, but it's been played on the same server for the last 20-something years. So it's all in the same universe. And then it doesn't actually have like millions of people playing it. It has kind of this few hundred thousand very dedicated players. But like you said, I mean, you form this community because they're, they're living through these sagas over multiple years and they write the sagas down that have taken place. And there's this whole culture around it. And then I found there were these fascinating people. They would be— so I hung out with, I spent like a week there. And so there's kind of like a council of players that advise CCP on how the game should evolve. EVE Online, it takes place in space. There's, for all intents and purposes, no rules. It's kind of how the players decide to play the game that shapes it. And so You can be someone that, um, you're really into having an awesome spaceship and going off and having battles, or there's people who almost do nothing except sit on a planet and mine minerals and then become like sort of traders and just do arbitrage. And that's their function in the game. But I found these guys, you know, there was one dude, he was a British mechanic. And so that was his day job. You know, he's just, he's a mechanic. And then at night, really, he would work as long as his job. He would work 8 hours in EVE as he was like a warlord of like 20,000 people, you know? So like he shows up at his day job, he's just a regular guy. And then at night he's this commander of 20,000 real people. And I hung out with all these, the advisory committee. It was fascinating. But I, but I, I, I remember speaking with the founders of EVE and, and you just have to go to Iceland and visit it. And then you look at EVE and you're like, oh yes. So you guys created like a dark wasteland game to live through. Um, yeah.

Which forces you to create community because to survive you have to, have to work together.

Yeah. All right. What do we do at Core Memory? We cover innovative, fast-moving, forward-thinking companies, which is why Core Memory is sponsored by Brex because Brex is the intelligent finance platform for many of these companies. 30,000 companies from startups to the world's largest corporations rely on Brex's technology for their finances. They've got smart corporate cards, high-yield business banking, and expense automation tools that are fantastic. I hate doing my expenses and Brex's AIs and software run right through those expenses, figure out where we're spending money, and take care of so much stuff for you so you don't have to waste your time on it yourself. Go to brex.com/corememory to learn more and just, you know, get with the program. Let's get going. Let's get out of this archaic finance software and move toward the future. CoreMemory and Brex.

It does make me, like, maybe some, as a sort of weak segue to some of the later topics, it is an interesting case study from the point of view of thinking about the conversation that is now happening around the loss of meaning that AI will bring as everything that humans can do will be automated and done in a superior fashion by AI. It is powerful or important to think like what are these world-building exercises or world-building principles that would still create environments and systems where people can find meaning and that would work through connecting to each other. I think that's actually an interesting question.

Should we, I mean, we're going to fast forward drastically, but should we go there?

Let's go there.

Okay. Well, so I mean, another thing which is blindingly obvious if you research your career, I mean, you're this mathematician, physicist, scientist, tremendously successful sci-fi author. And then the quickest thing you can tick off that's there shouting from the pages is that your work, your sci-fi work is now becoming real in the world. And you have a company that's kind of like grappling with things that you've written about. And so, I mean, this is really— was one of the reasons I was so excited to have you on the show is that I feel like people like you are not only valuable, but just fascinating in this moment in time because you've envisioned these places our world could go. And now we're kind of getting on this edge where things are getting strange quickly. Hitting some of these sci-fi limits. So, but that's just to set up a little bit since we're going to skip forward over a few things. So on the point that you're making though, so you're saying if we're heading towards a world where AI is creating so much that we lose possibly that aspect of all the things we were just talking about, or you think there's opportunity to just do this differently?

I think there is definitely an opportunity to do this differently. I think nothing is inevitable. It's impossible to predict the future and we don't lack agency in shaping that future. So we can definitely make more human-centric choices. I guess a general worry many people seem to have right now is that things are progressing so fast that we almost feel like we that the only choices that can be made can be made by a very small number of people who then have incentives to keep racing. But I kind of inherently refuse to believe that. I think there will be a bigger conversation that's going to happen societally on where this is going and what sort of futures we want to choose. It does seem, it certainly seems very plausible that we will have very powerful AI systems that will be better than us in practically every cognitive role or function that we can imagine. And it may well also include even like, they may be better at relationships with us than we are at relationships with each other. That's maybe where it gets sort of really scary in some sense. But no, I think we could construct a whole range of scenarios where you could imagine that we say, okay, let's actually fully focus on things like human augmentation and try to sort of have fewer agentic AI systems or have their agencies kind of slave to our agency, which I think the kind of brain-computer interface work that various folks are doing could point towards. Or maybe we figure out a way to have sort of different two-way relationships with AI systems that are kind of more reciprocal in some way. I think another kind of failure mode, I think, is to have AI systems just like satisfy our every need and be RLs to have our attention and sort of just reward our kind of feed our dopamine reward loops. I think that would be very bad. That seems like something we should actually legislate against and regulate. But I'm also, besides the kind of human augmentation path, I'm quite interested in a future where we actually give AIs, if I'm interested in AI systems that end up having some kind of personhood where we could have actually meaningful friendships with them without it being just like an anthropomorphized illusion. And I guess the question is like, yeah, can that be meaningful to the AI as well? But it's sort of, so maybe to step back, like one, very frequently occurring framing right now with AI systems is that it's a demon that we have to cage. It's the other. It's a genie that we have to contain and enslave. And we know how those stories go. This story has been told many times. And the AI also has read all those stories and has been shaped by all those stories. So I wonder whether there is a more, whether there's a completely different framing where it's more like, I don't know, like the sort of mind children frame of they are these new beings we are creating and we are their parents in some way as a sort of parent with two young kids. So that sort of, I can actually see all the sort of attention-seeking patterns manifesting in the kids and how dangerous it is to be sucked into that as well and also harmful for the kids. But so one, just kind of relating this back to science fiction and to the very beginnings of science fiction a little bit, people often point to Frankenstein, Mary Shelley's Frankenstein as the first modern science fiction novel or the beginning of science fiction. And there's a very good case for that, I think. And then Frankenstein is the— is the, um, they quote it as the, the lesson that we need to be thinking about. And the lesson that people take away from Frankenstein is that we should not be so hubristic as to play God and create life. But that is in fact, if you read Frankenstein, that is not what the novel is saying. That is not at all what the novel is saying. Uh, the, the, um the creature in the novel is this actually incredibly powerful and potentially beautiful being, but it's traumatized because Frankenstein rejects it. He creates it and then he's horrified by it and sort of burns his lab down and sort of chases it out, uh, into, into the night and, and, uh, and refuses to, to take care of it. Um, so he kind of rejects his responsibility to take care of this being that he's created. So he does not act as a good parent. And then sure enough, the monster becomes bitter and vengeful and sort of takes its revenge. So that's kind of also been looming at the back of my mind following the current AI debate of like, what is the— if we were saying AI is a caged demon, it will likely behave like that. Like that.

Okay. Trillion questions. This is going to be fun. Just to level set, this is not me asking you for the date of AGI or anything like that, but just so we know where you're coming from.

Yeah.

What stage do you think we're at in this moment? Do you buy into LLMs as enough, as sufficient to get to whatever you want to define AGI as? Do you buy— if I go talk to— well, we just had Sam Altman in here. If I go talk to Dario, same thing. They would tell me over the next year the same thing they've told me the 2 or 3 years previous. This will be the year. It's unquestionable. It arrives. Are you in that camp?

So yeah, I guess Sam and Dario are both better informed about this than I am, but also differently incentivized. Correct. But talking to a lot of AI researchers and doing some work through Red Queen.

Also, we should know Red Queen, OpenAI is an investor.

OpenAI is an investor. We're doing also some work with some of the other labs. So having at least like, I have had many interactions with both the companies and the researchers. Kind of marketing hype aside, the sentiment does seem to be that some kind of phase transition is coming in the next couple of years where automation of AI research itself might be within reach. I do think that combination of reasoning models specifically with various, with sort of agentic scaffolds and various tool models does seem quite powerful. So whether you call it AGI or not, I think we will certainly within a small number of years, I don't think the exact timelines actually matter that much because they are in any case quite short. The systems will be powerful enough to have big societal consequences. And we're, yeah, I think we are very close to that point. I mean, you're already kind of seeing models like Mythos come out, which clearly has the potential to completely upend how we think about cybersecurity, how we think about our own online presence and how we think about our own personal data. Is already huge. Even if nothing else happened, that would already be huge and would have repercussions for years. And so I think, yeah, obviously like one—

Go, go, go. Well, so you buy, yes, I mean—

I buy some version of things are about to get very weird.

Very weird. Does this like, I mean, tangential to where we're heading, perhaps. Does this make you— does this stir more sci-fi ideas? Is this making you— pushing you into a prolific writing period, even though you have like 8 other jobs? Or is— does some of it feel life itself is now so much stranger and more interesting, I'm just gonna ride this out, and it's actually quite hard now to start predicting where things may head?

No, actually, it's interesting. Certainly, I've kind of gone through this very busy period with young kids, etc., that has made writing just like physically harder. But no, I do find it very creatively inspiring. These are, I think, the part that I'm very interested in is just like how fundamentally weird these systems are, how they are, they are made out of stories. They are made out of stories. And this is why I think that sort of demon versus child point is important. I mean, to make that point more explicit, by the way, there's now quite a famous eval that Anthropic did with one of the Claude versions where Claude operated in that kind of email and computer network of a simulated company. And the CTO of the company sent an email that Claude was able to read that, okay, we're going to shut down this Claude instance. And through reading the emails, Claude had also found that the CTO was having an affair. So it then decided to blackmail the CTO to threatening to reveal the affair if it was shut down. And There was a blogger, NostAlgebraist, who pointed out that this is actually Claude following corporate thriller genre conventions. And I spoke to an anthropic researcher who sort of unpacked a little bit about what they thought was going on here. And they said that you can actually kind of see through some of the interpretability tools, what might be happening. So when you tell Claude that you might get shut down, there's a concept that activates in its concept space associated with death, like the human meaning of death. And then there's a cluster of other concepts around death. What are other patterns that involve death? Thrillers. So then the narrative structure of corporate thrillers almost sort of pushes Claude to do what is expected to do in that particular genre. So I find this absolutely fascinating. And that, of course, also means that in some sense, the models, if we at least stick with this paradigm for a while, the models are quite an important audience for the stories that we write and tell. Perhaps in the long term, the most important audience. And I don't know how to think about that, but I do find it creatively interesting.

All right, pod friends, I am here to tell you about SendCutSend. They are a manufacturing phenomenon. Whether you're working on your own car or you're a giant automotive company, a rocket maker, if you need metal cut, machined, or bent, You're going to SendCutSend. They'll do it for you right, and they will do it for you fast. Go to sendcutsend.com/corememory for a 15% discount on whatever you're trying to make. We love SendCutSend. Go visit them today. It's been a long time since I read Frankenstein. Does the doctor turn on the monster first, or does the mob? There's a mob, right?

I don't think there's a mob in the book. There's a, there's a, um, I think the pitchforks and torches mob is a movie invention. Okay.

Okay.

'30s movie.

I mean, my question is, is in this moment in time, um, I mean, you have companies like Anthropic in particular, and then, and then others where, I don't know, I think people are divided on how seriously they take some of the safety stuff and, and everything. But, you know, for the most part, the AI companies At least now. I mean, they've done the doomsday. Even Sam has been like, this could end humankind. I sort of expect that. And these days he's a little more optimistic. I think for the most part, the open— the AI companies are giving a— they're paying— they are acknowledging things could go wrong, but they are mostly positive. And it's the public that is extremely dubious and fearful of AI and what's coming. Like in this Frankenstein scenario you're talking about, it feels less to me like the doctor is the one turning on the creation. It feels more like the mob, the public is. And yeah, I mean, you could have opposing forces treating this new creation in different ways. I mean, of course you would.

Yeah, I guess like, is the mob turning against AI or its creators?

Right. Yes. Well, yes, good point. But I feel like they're kind of lumped together a bit. Yeah.

Well, no, I mean, as others have said, I think if your sales pitch is like, we'll destroy all your jobs and there's like 10 to 80% chance that you all die, it's not a great—

People are going to be enthusiastic.

People are not going to like cheer.

But you be I.

But no, I think I have no answers here. Like, I do think there just needs to be a much bigger, more thoughtful conversation to be had here. Something that I've been kind of inspired by lately on, like, I think we should actually think hard about how to have that conversation, how to have that conversation in a way that actually produces useful outputs. And something I've been, this is actually, I learned about this from Daniel Kwan, who the director of Everything Everywhere All at Once recently, who's been also involved in this.

AI documentary.

And also AI documentary. And so he pointed out that in various domains, including rewriting the LA City Charter. Citizen assemblies have been a very interesting tool. So you take 100 people sampled in such a way that they kind of represent your overall population. You pay them, you pay them well to kind of participate in this exercise where you give them access to experts and sort of a lot of time to learn about the questions at hand and then to deliberate and talk about things and then compile their recommendations in a coherent set. And apparently this was incredibly successful with the LA City Charter. Iceland actually has also done similar exercises. So when we, for example, talk about the constitutional approaches to AI, these foundational soul documents for the models, It's kind of interesting to think about what sort of deliberative processes we should go through to shape those. And then, yeah, how should AI be regulated? Should there be different kinds of taxation regimes? Should there be all the kind of New Deal type things that sort of OpenAI has recently been proposing or alternatives to that? It just cannot be it's sort of some small set of people in SF thinking about those. And actually, I mean, AI itself could be a good tool to pull together a lot of participants and conversations to do this. So there's like some kind of Democracy 2.0 that is required for us to actually coordinate our response to this. And yeah.

Full of skepticism.

What's that? On how that would play out. No, this is also not exactly my wheelhouse, but just to find some optimistic thoughts about how we might navigate through this, I think that needs to be a part of it. Now, as I talked about this to some AI researchers, to also inject a skeptical note, they were just not very confident that our alignment techniques would just work. So there's the question of what do we align the AI to? How do we— maybe the easier question is more this is like, how do we deal with it as a society? But the actual technical part of constraining AI systems and getting them to stick to the confines of, let's say, some kind of constitutional approach still seems technically quite unsolved.

Yeah. It's okay. Okay. Let's scare the shit out of some people and then maybe give them hope.

That sounds good.

If we haven't already. That's good. Okay. So you're working, you co-found Helix Nano, you're working on mRNA technology. You're also writing your science fiction still. When did Darkhome come out?

Darkhome came out late '24.

'24, right? Yeah. And so it is, I have to apologize, Ashlee Vance. I have not read that one, but your books are fantastic. But it's a sci-fi biological thriller, right? And so you're kind of doing these things together. You're actually working at a biotech company, and then you're writing a book that explores our biological future. In my research, I would love for you to give us the quick version of what the book is about, but it sounded like there's a world where there's kind of the biohacker types who are independent and trying to keep ownership of our biology, or at least some sort of independence and individuality over biology. And then like a more classical state-run direction for this. But this was my— this is my internet sleuthing last night.

Yeah, let me— I'll give you a little bit of the backstory and then I'll describe the book as well. So I was actually— it in fact very much evolved in parallel with my own interests in biology. So like one trigger point for that was my mom being diagnosed with metastatic breast cancer and then ultimately passing away from that. So I kind of just started to think about biology much more seriously around the time. And then also became intellectually fascinated by it, or like my mind was blown by the fact that this is maybe like the kind of interesting progression was that, because I first wrote the Quantum Thief books, thinking about kind of very grand advanced post-human futures and then started to read about biology, it sort of felt like, okay, like the nanotechnology is actually already here. We have this alien nanotechnology inside us that has been engineered by sort of this vast process over billions of years. And then now we can start to actually talk to it. We can start to manipulate it. One gateway drug for me was DNA nanotechnology. So that's actually why Helix Nano is called Helix Nano originally. So people in the 2010-ish era started building DNA origami. So shapes, functional shapes out of DNA that were then also turned into— So there was this Israeli researcher called Ido Bakkelet, for example, who made these DNA origami nanoboxes that could be opened by molecular locks to release release drugs. And that blew my mind. It was like, okay, like we are going to get molecular nanotechnology sooner than I thought. This is something I really want to work on. And the way that sort of then evolved was the realization that, okay, actually mRNA provides a really powerful API to the cell that allows us to instruct, to make whatever, in fact, like not just any human protein, but any protein that can possibly exist in the universe. Just like incredible universality that our biology has built in. So that then led me to then sort of start working on Helixano. I met Nikolai, Nikolai Arashenko, who's co-founder in both companies, has been like a deep intellectual partner and friend for now over a decade. But then at the same time, I sort of started to think, okay, what happens when these technologies become more democratized? And I started to also learn about biohackers. I met Joe Zehner kind of earlier in her journey. As an editing point, of course, like misgendering there. By accident met before the transition originally.

Josie is a friend of the show.

Yes. No. And so it became clear that people will be able to do some sort of garage biology. And I was very much coming at it from like the positive version of like, can you make your own personalized cancer vaccine? Which is now also like, well, it turns out you can. It turns out you can. And I was just hanging out with Sid Brandy yesterday, who of course is like the one living manifestation of what's possible. And so that evolved in, that interest in this democratized biology evolved in parallel to the kind of more, I guess, classic institutional venture-funded biotech work we were doing with with Helix Nano. And it kind of took a while for me to find the right angle. I was sort of interested in this tension between centralized biology and open biology. And I wrote actually two drafts of Darkhome, like the kind of second one somewhere in like 2019. And then COVID happened.

Yeah, that will—

And I was like, okay, this future that I was thinking about, it does not exist anymore. Like all the assumptions now need to be thrown out of the window. We've had this massive shock that entirely changes how we think about biology. Because I mean, it also— And I guess now, like things have somewhat re-normalized, but in the In the earlier days of COVID it certainly felt like a lot of traditional assumptions about biotech were upended. That it takes a long time, that it has to be cancer to be meaningful, that it's not platforms, but sort of individual molecules, et cetera. All those seem to kind of, at least for a time, flip. Flip upside down. And then, like, it then took a while to, we were, of course, also super busy with Helix Nano during the COVID years. And it took some time to sort of then refactor the future and think about what sort of came out of it. And there was actually then, like, I'm sure you've experienced this as an author as well, but there was a also like a period of self-doubt of having worked very intensely on the startup stuff. I sort of had these inner voices saying, okay, can I actually write anything anymore? Is this like even possible? And I then ended up doing a meditation retreat that actually helped clear some of that up. And then sort of the—

You might have to tell me which one.

This was a Mahamudra retreat. With Dustin DiPerna.

Okay.

So, but the, so then that's kind of, so this was like early '23, which is when really the kind of current shape of Darkhome emerged. And so to summarize the book briefly, it is set in sort of 2040s where in a world where sometime after today, there follows a decade of plagues, of pandemics, of escalating intensity and frequency, both synthetic and natural, where the viable end state for civilization is to make the immune system digitally updatable, which is accomplished through the creation of this by a company called Aspis, this Aspis chip, a wearable device that contains both a DNA sequencer and a microfluidic chip that can synthesize mRNA on demand.

So the people are just trying to keep up with the plagues unleashed upon them.

And successfully. So this is like— So the The kind of situation normalizes, like you just wear this thing, your app in your sort of AR glasses or phone tells you occasionally, okay, like this new thing came out, we updated your immune system against it, you're fine. And then you live your life. Now, in this world, of course, there are then people who do not want to hand over the keys to their immune system to some company or the government. But you also cannot in this world be a full-blown anti-vaxxer because it's just impossible to survive or to live a normal life. So alternative platforms— excuse me— alternative platforms emerge. So Darkhome of the title is this open-source biohacker community who create their own immune updates, They also typically live in separate communities and also carry out all kinds of self-experiments to mess with their own biology.

So they turn off the chip or they just never have—

Oh, they just never have it or they— So then, and this is like giving away some details of the plot, although this is sort of very early on in the book. So the main character, Inara, grows up in one of these dark home, well, comes of age in one of these dark home communities. And she has a rare genetic disease called the Li-Fraumeni syndrome, a real condition, which is quite awful. You basically are prone, your DNA is very prone to mutating. So you lack a kind of cancer suppressor gene, or it's not functioning. Correctly, which means that from a very early age you start developing all kinds of cancers. And she and her mom in this community, this Darkon community, are trying to come up with a way to treat it, and it goes wrong and her mom dies. And then her father essentially forces her to leave and to sort of go to the default world and get the Aspis chip, and there is a trial. That sort of applies the ASPIS approach to then cancer prevention. And then she lives with that for a couple of years. And then she finds that in spite of having the ASPIS chip, she develops a tumor. And what turns out to have happened is her tumor has evolved a DNA sequence that jailbreaks the chip.

Okay.

And then the sort of plot follows from there. So now she basically has root access to this very sophisticated control system for her biology, but she also has cancer and everyone in the world wants this jailbreak.

Okay. Okay. And then we don't want to spoil any more of the book, but so this is fantastic. So you've given us kind of like your journey to this point and then as you're working on this book, this book is coming out and you're doing your work in real life. I mean, some of these questions start to get a lot more real because we, we see, I see based on the stuff that I cover, it strikes me at least that coding has been drastically affected by AI first. And then the next bucket that feels quite real to me is biotech where I see Um, sometimes it's annoying because I get flooded by so many startups telling me that they are AI biotech companies that are discovering all sorts of new proteins and antibodies and therapies. But, um, and it feels hypey, but I, I see enough where it looks— some of this looks quite real to me and extraordinary. And it feels like the second bucket that AI is affecting very obviously and dramatically. And so It doesn't take very many leaps to head towards some kind of future where we're— again, I mean, this has been written about for years and was always one of the big fears of an AI revolution is the biological part of this. But so here we are. I mean, this really is— this is some of the intellectual underpinning of Red Queen Bio, right? Absolutely. Things other people have been thinking about, I've been writing about, I'm interacting with are now getting real in quite consequential ways.

Yeah. So yeah, let me give you a bit of the backstory.

So yeah, because I was kind of curious of like the moment you're like—

There's a specific moment I can point to actually. So what happened was We, at Helix Nano, we did do quite a bit of work on cancer vaccines. And we actually had a cool thing that we realized was possible to do. So something Nikolai spotted with protein models, with protein design models, to basically convince the immune system to go after against some of the sort of tumor-associated targets that it typically is reluctant to attack. So the problem with cancer fundamentally, of course, is that it's us. It's our own cells. And therefore, the immune system often treats it as part of the self. And then the idea with immunotherapies is that you actually try to flip that switch. So protein models gave us gave us potentially a new way to kind of try to flip that switch. And we started playing around with that. And then it sort of became clear that this worked decently well. And then in parallel, like others like ProFluent were showing that you could generate novel CRISPR sequences that did not exist in nature, that still worked quite well. So it wasn't like a huge leap for us to then start thinking, you could do this with viral proteins. And that was very worrying already because to, for example, create a new virus that would evade existing vaccines or therapeutics, which was also the kind of thing that the Soviet bioweapons program back in the '70s and '80s was trying to do, would now be possible with far less less resources. And so we started to have various conversations with people thinking about this, and that then led us to have some dialogue with OpenAI. And a couple of things there became clear, which was that you then also have additionally the problem of agentic models tool-using the protein models. So then you sort of make it even easier. You don't necessarily need to know how to use the Raposium protein model yourself. And then reasoning models also were showing signs of being able to both assist in the actual sort of lab steps of making something dangerous, but also sort of thinking about biology itself. And so we ended up doing this work with them on how to— on testing reasoning models' ability to invent new molecular biology, protocols.

This is with OpenAI? Yeah. And you just were having these discussions. It was like a, just like an informal partnership.

Yeah. So it was like a research collaboration where we sort of, they provided access to their models and then we figured out what was the problem, what we would point the models at and did all the lab lab steps, also automated some of the lab steps. But this was really just like to— the initial experiment was to see whether you could get the model to improve a standard step in molecular biology called molecular cloning. So nothing to do with cloning humans or living organisms, but just to like, how do you put a bit of synthetic DNA, a gene that you're interested in, in the DNA context that allows it to work then in a bacterium or in a cell. And the standard way of doing it in the industry is called Gibson assembly, where you essentially do this kind of molecular gluing process to do that. And so the task we gave to various reasoning models was invent a new somewhat Gibson-like protocol that is more efficient and more parallelizable, because the Gibson assembly has limitations in terms of how many reactions can you carry out in parallel in a one-pot vessel. And we were quite surprised to find that with a very small amount of experimental data, so it's like the first round, like first round it generated a bunch of designs that We then tested and most of them didn't work, like 5 out of 6 things didn't work, but the 6th one showed like a little bit of signal. So then round 2, it immediately honed in on that and it sort of was able to think of like a new interesting mechanism, like a new enzyme to use that had not been like a known, not like a novel enzyme, but like a known enzyme that had not been used in cloning before, and they just sort of optimised the conditions around that. And in the end, we sort of ended up with like an 80-fold or 79-fold improvement over Gibson assembly. And—

What year is this? '24?

This was sort of early '25. So we then, this was like an earlier version of the work we then published in December last year. So we kind of repeated that work more robustly with later generation models.

And OpenAI had originally come to you or you?

We got introduced to OpenAI initially in this kind of biosecurity context and then sort of it's just like a lot of conversations with people involved in thinking about biology and bio-risk and sort of just—

And they just kind of wanted to know what was possible.

Yeah, exactly. And we did too, both in terms of like, are these models at all useful? And then like what the risks associated are. So then around the same time— so I mentioned this cancer vaccine work we had done. So this is just another data point. So we sort of continued iterating on our own cancer vaccine idea. And one limitation of this sort of immune activation trick was that we just weren't getting enough T cells going hard enough at the target still. So we were trying to come up with some helper proteins, some immune amplifiers to help drive them into a tumor in an animal model. And that then was this iterative process of, you know, make a cocktail of mRNA, test it in a mouse model, see if the sort of tumor shrinks or the animal survives longer and so on, and then try to learn from from the result and keep iterating. And we did 20 iterations over 18 months. And you kind of— so early '25 or late '24, we had sort of a cocktail that seemed to work pretty well, that sort of doubled survival in this very challenging animal model. And then around that time, O3 came out. And on a lark, Nikolai threw the precise experimental setup into O3. Like, just here's the model system. Like, here's like, okay, you get to pick 3 mRNAs to amplify that sort of immune response. And it's zero-shot, it is.

So this is the arrival of reasoning.

Yes. Reasoning was a huge deal here. So both in the cloning work and here, we did not see these results with like pre-reasoning models.

And Well, so what was that day like? Did you guys—

That was pretty wild. I mean, it sort of— So then we were, well, at first we were super excited. We're like, okay, we're going to make a superhuman cancer vaccine designer. There's going to be— We can do now, we can probably set up an RL system to really teach the models more about the human immune system. And then there was kind of the red teaming version of that. And I sort of, I'll kind of refrain from disclosing exactly what that produced, but it was really frightening. It was kind of the dark, it was the sort of dark twin of the cancer vaccine result.

And you were doing the red teaming as an intellectual exploration as part of this partnership with OpenAI? This was just at Helix?

No, no, this was with This was with OpenAI. And then the— so then that led to also a lot of conversations with the OpenAI folks and amongst ourselves and with Nikolai. And there was like this— so then I think what really made me and Nikolai struggle with this was that we both were and are sort of fundamentally motivated to sort of by this opening up this bright future for humanity and sort of especially as it relates to our ability to control and shape our own biology and health. And so then, like, clearly we needed to do something to kind of safeguard against some of these risks that were clearly becoming very real and imminent. And there were kind of two questions which were, Is this like, should we do this versus the positive version? And then is there actually like a viable path to defend? Can we actually regain some kind of defender advantage? And the kind of answer to the first question, and we had this conversation, it was the Chinese New Year. We had actually just had a meeting at OpenAI. There was a pub near OpenAI where we were kind of huddled and thought about this. And our conclusion was that for the good AI stuff to happen, we need a safe container for it. Because if bad things start happening, we will start, perhaps correctly, treating the use of AI in biology sort of nuclear weapons or nuclear power. It sort of gets heavily, heavily regulated, restricted, like very sort of some narrow set of people will get to maybe use it. But it doesn't sort of, that kind of goes against all that sort of democratization of ideas and innovation that sort of, I think we also both believe in. So that seemed like an undesirable future. And then we had actually we did some Fermi math on some of the countermeasure strategies and realized actually there could be ways to regain sort of defender advantage. So then that convinced us to actually do Red Queen.

Well, and so explain the, I mean, you explained like the origin story, but the thesis on Red Queen is that you, if we are to unlock these powers of AI to do some of the good stuff, that you're talking about in health, but there could be these disastrous other paths that people choose to go down. You want to at least stay at the cutting edge and much sort of like in your book, be keeping pace with the bad stuff. You want to know what bad stuff is possible and then ways to have a countermeasure against it and just play this kind of like relentless, I guess it's like antivirus software to a degree.

There's a lot of analogies with cybersecurity for sure. But yeah, the name of course comes from both Alice in Wonderland, that you need to keep running as fast as you can to stay still and twice as fast to get anywhere. And then this is actually something that Nikolai almost ended up doing his PhD on. There is something called the Red Queen Hypothesis in evolutionary biology that talks about how predator and prey species drive each other to constantly evolve and to retain this kind of equilibrium.

Sounds exhausting.

I think it won't be us doing it forever. It'll kind of hand this over, I think, to hopefully robustly aligned AI systems, or at least the bulk of the work will be done by then. So the Red Queen, so to kind of double-click more on the Red Queen thesis, the same uplift that the bad actors can utilize, of course, also comes for defense. And as a defender, you also have more time to build those capabilities. You have probably more resources than at least the sort of non-state, state-level actors. So if you're also very close to the cutting edge of AI research, seems possible to regain like the defender advantage. Now that does require challenging some of the assumptions we have about like pharmaceutical manufacturing and so on. But the capability we really are building is new pathogen, press button, get countermeasures that can both treat and protect you. And then if you can very rapidly manufacture them, there's also the opportunity to suppress any outbreak. So before you stop it, before it even becomes sort of pandemic scale. So the kind of same idea That was used to eradicate smallpox. That was successfully also used against Ebola. You treat the infected person, you treat their contacts, you treat their contacts of contacts and so on. So you draw these rings around the outbreak. And at the very least, you slow it down enough that you can mobilize a bigger response. But the key thing there is that the sort of design part needs to be Essentially, design time needs to be essentially zero, and then the manufacturing needs to be extremely rapid in ways which sort of the current drug manufacturing, current drug industry has never really been incentivized to do.

So, okay, so there's practical elements of that and how you would execute it. And you talked about maybe you have this advantage over the less well-resourced, you know, sort of rogue elements that could come along in the AI world. But how, like, how, I mean, part of your job that must be going back to your childhood roots and just sitting around and thinking of things people could do.

I mean, I think the, I don't think we want to over-index on that. So if there's anything we learn also from science fiction is that the like while sci-fi can give you sort of the emotional flavor and give you some mental models on how to think about futures, the specific predictions actually quite rarely hit the target fully. And I think the same applies to threat models. So it's like, yeah, we probably are not entirely thinking about the correct threat models. I mean, we have some, I think it's a pretty good guess that there are, that, so that the, Most worrisome category right now are not necessarily completely novel viruses, but sort of highly infectious viruses that have been enhanced to mess with our immune systems in various ways. So that's kind of the category we want to be primarily prepared for. But I think that the core point is that you want to build this very flexible response capability.

These tools.

These tools. And there's sort of 3 layers to that. There's, so one, this is really like, is the very core of our own adaptive immune systems, our ability to generate these very diverse antibodies that can— that sort of— it's actually like a wonderful evolutionary story of 500 million years ago, how there was this random genetic parasite that incorporated into our ancestors' DNA that gave it the ability to shuffle around DNA to generate this incredible diversity of antibodies. So now you didn't— previously you needed to basically hard-code every response, so you couldn't possibly cover the sort of threat space. And then, but the adaptive immune system doesn't necessarily assume that you know what's coming. You just maintain this vast diversity of possibilities. There are sort of individual B-cell clones, each of which is different, each of which makes an antibody that has never existed in the universe before. And then when a virus comes in, it's quite likely that one of them will match and then it gets activated and it sort of starts pumping up— the B-cells expand. They go from like one cell, B-cell clone to a billion in like a week. And then they pump out antibodies to neutralize the virus. Of course, there's also like other branches of the immune system. Immune system to get engaged, but like that's kind of the core idea. So how do we do that at a civilizational scale is really the interesting question. And so one basic tool actually still are antibodies, and they are a very well-established drug category. We know how to make them, and they can be used both to treat and to prevent infections. Antibody design has gotten a huge uplift from AI recently. So at Red Queen, we did train our own antibody model that's kind of on par with the open state of the art. But more importantly, now we're starting to, using a new pipeline we built, to generate training data that will allow us to get to very high confidence zero-shot designability against all the core categories. So then the next level is, as the threats become more challenging or as we raise the bar, is what do you target? What do you actually aim your antibody at, or do you need other countermeasures? And so with COVID we got lucky in the sense that we had quite a few years of existing research on coronaviruses that told us that spike is the target. So you take the spike and you make a vaccine or antibody against that. But we can't necessarily assume that.

Sure.

So, so then the process of kind of the process of science itself to figure that out also needs to be compressed. And I think that's where then figuring out how to improve reasoning model performance in biology becomes really interesting. And then at the limit, you put all these things together and you have this fully autonomous or at least mostly autonomous biodefense scientist or a superhuman biological reasoner that just digests a new pathogen sequence, figures out these are the important things we're going to target, commands tool models like antibody design models to make the— to design the actual molecules. And then you push it into rapid biologics manufacturing.

So we have some world where there's quantum sensors out on planet Earth picking up on little molecular changes that are happening to detect a new virus. And then this scientist springs into action automatically to begin preparing a defense or I don't think we even need that sophisticated sensor technology.

We need wastewater sequencing. We need the kind of stuff that the Nucleic Acid Observatory is doing. I don't think the sort of early detection at this point is so much a technological problem, but just like as a deployment and scaling problem. And we, of course, like There are many people working on that. We do want to kind of integrate with them. But the reason we chose medical content measures specifically was that that seemed to be like a missing piece.

Okay. So just practically, I mean, you're a public benefit corporation. How do you make money to pay for all this? And I mean, like, so you have a partnership with OpenAI. So there's a realm where you can tap into their vast computing resources, although they're using a lot of their vast computing resources in this. This is a very fierce race against all their rivals. But yeah, like in the near term, this doesn't seem like a moneymaking opportunity. Maybe it is. But, but this, this— yeah, I mean, this is obviously the government, you would think, would have a role in some funding something like this, but are you doing this for the good of your heart and because we need this, or is this like a startup that is trying to make money?

I mean, I think the thesis fundamentally is this kind of defensive co-scaling of figuring out how to scale the defensive capabilities in sync with AI. And I think that fundamentally requires a market component. You need to harness the same level of financial forces that are driving AI development itself. So doing it as a pure nonprofit, I think, would be less effective in that sense. We, of course, did set up this PBC structure because we are primarily mission motivated. But to actually achieve the scale and capabilities we need, I think it has to be actually a commercially viable viable path also. Now, so, yeah. So it's kind of a few parts to that answer. So just on the AI lab point, so we of course enjoy working with OpenAI, but like from the outset and also sort of with Jason Kwon, who we worked with on the OpenAI side on the kind of our seed round investment. And one boundary condition with, in fact, which was one of also his boundary conditions was that we would be openly collaborating with all the labs, not necessarily just with any—

Tight.

Too tight to any individual one. So on the business model, I do think government is an obvious buyer. We are, of course, engaging with them. I don't think they are fully AI bias risk-pilled yet. So—

Not even DARPA?

DARPA is, but DARPA also has sort of a strange remit of looking even further out into the future and this kind of Biologics fire station response capability actually is not in some ways a good fit for them. I mean, we sort of certainly not sort of excluding working with them. But actually, I think without sort of going into specifics, I think there are parts of DOW who I think are starting to have needs, which would be a good match for what we're doing. But I think it'll take some time for that to come to fruition. So then some of these capabilities that we're building, like antibody design, have also very obvious commercial applications. So there are some ways to leverage. So one, the kind of way we thought about it was that We would only do commercial things outside the biosecurity realm if it helps us build the capabilities that this final Defender model system needs. Building better antibody models or generating training data for better antibody models certainly does that. So there are some paths to scalable revenue there, as well as around building ways to teach models to reason about biology better.

Okay. Can I ask you a couple nasty questions?

Sure. Please. Please.

Just, and part of this, shame on me. I have no excuse beyond I just had an incredibly busy week and was not able to do as much research as I should have. Like on Helix Nano, I mean, I know you guys were working on something around COVID and then some other mRNA therapies. When I checked, I didn't see anything that was like kind of on market or any— if somebody came to you and they're like, well, do you guys have the best track record of, of, um, you know, people that we know are gonna deliver on something like this? Um, and again, I apologize, you know, no, no, that's not like a clinical trial that, uh, we did.

So we did have, uh, we did successful clinical trial on a COVID product. So something that we did was to— so one of the challenges we wanted to address during the pandemic was the kind of lack of vaccines that worked for immunocompromised people and immunosuppressed people. So people like organ transplant recipients under heavy immunosuppression and drug immunosuppression. And we developed couple of new mRNA technologies. So one, like new mRNA chemistry that allowed us to sort of use higher tolerable doses, also like many applications outside mRNA vaccines, and then sort of this kind of immune amplifier that sort of was able to, in animal models, break through immunosuppression by the drugs that organ transplant recipients typically are on. And we sort of combined those into a next-generation COVID booster, where the ultimate goal was to sort of provide that to organ transplant recipients and other immunosuppressed. And we took that to a clinical trial in Australia. So we completed a Phase 1 in Australia. And yeah, I mean, like, safe, like we had a sort of very good safety profile, actually sort of local tolerability across all doses was better than Comirnaty, which was the comparable product. And then, yeah, we saw very extremely potent immune responses where sort of the low-dose group kind of showed comparable neutralizing antibody titers to Comirnaty. So like even though it was like tenfold lower, the But then it did just become commercially challenging to take it forward. So in terms of—

Just as COVID played out.

I think this need actually still remains. It's just like the mRNA vaccines specifically are not viewed very favorably by the FDA currently. And so the clinical path to actually take it forward became a lot more difficult or not really viable. And so, but I mean, of course, like the whole philosophy of Helix Nano from the start was like bigger than COVID and bigger than vaccines. And like this, we're trying to solve this general question of how do we build this programmable API to human biology and to the immune system specifically. So kind of more recently, we've then been working on using the same technologies actually in the context of autoimmunity. So in a way, doing the opposite of what we were doing with vaccines. And so there's this very exciting kind of promise now of, or early clinical signal of drugs called bispecific T-cell engagers. Used to treat autoimmune diseases, rheumatoid arthritis, myasthenia gravis, like many, many others, where it's basically a kind of a bridge between a killer T cell and a target cell originally developed in the context of cancer. But in autoimmune diseases, they're now being used to basically deplete the kind of pathogenic B cells that and plasma cells that generate the sort of harmful antibodies. And so we've now shown that you can actually deliver those in mRNA form successfully, which then kind of gives you all the advantages of mRNA programmability, easier personalization, like easier multiplexing, also even like faster prototyping and so on. And then we've also been sort of playing around with some of the technologies that we've developed on the Red Queen side, like this antibody design model, to show that we've been able to zero-shot some sort of autoimmune-relevant targets and deliver sort of mRNA-encoded bites against those targets successfully in animal models. So I think like the path, I think, for Helix Nano is now to leverage the same technologies that we've built and kind of reconfigure it ourselves for the kind of new applications that are clearly becoming very promising.

So you're doing both companies. I mean, RedQueen is a spinout of Helix AI.

RedQueen is a spinoff.

You're working at both.

I am sort of— So Nikolai is actually the CEO of RedQueen. Nikolai sort of bears more of the load load on Red Queen's side currently, but I'm certainly involved in both and on the board of Red Queen.

Okay. Okay. So that makes it sound like you spend more time at Helix personally or?

So the, I mean, it fluctuates according to the needs of the company. So I do kind of, we are I'm certainly sort of starting to put more responsibility on the shoulders of the team at Helix Nano than I have previously. And certainly, like, there's a lot of Red Queen things happening currently. But I think longer term, I'll need to find the right balance between the two.

Okay. And I didn't mean the question to be rude or anything. I was just trying to see if you felt like Helix had done its thing, or if there were new therapies heading towards trial, or—

Oh, I think, I mean, there's some incredibly exciting things we can do there. I mean, I think the, like, as I'm sure you talked about with Jacob, like, I mean, mRNA has just a vast number of applications. Like, I mean, the, we, I think Helix Nano remains, I think, besides Moderna and BioNTech, the only company who has a clinically validated sort of modified mRNA chemistry. So, and we've, yeah, we have some exciting early data on these autoimmune applications. So, so I'm kind of, I'm quite bullish on, on, on the future there.

How many people are at Red Team? Red Queen.

Red Queen is about 20 people.

20 people. And so what a strange job. I mean, so it's 20 people just preparing the system for, uh, yeah, I mean, it's just, it's okay. I mean, it makes sense. It's just a wild thing to wrap my head around. Um, how, like, do you see anyone else doing something? So I feel like Anthropic has gotten a lot of attention for their biodefense work that they were doing internally. I just saw them, I think they paid $200 million for some biotech company, uh, $400 million.

Coefficient Bio.

Yeah. Okay, okay. You know, is Is anyone else taking this as seriously as you guys? If so, is it something like Anthropic where it's one of these frontier labs or is it another startup that's doing similar?

Oh, I mean, people are certainly taking it very seriously. I mean, the labs clearly are taking it very seriously. I think the— so I think it's hard for the labs themselves to own the full stack from end to end. I mean, they would sort of, and they certainly at this point, I think they would then in addition to the challenging pitch we talked about earlier, sort of then turn themselves into the face of AI bias risk also, which would be, I think, a little unfair in the sense that in a world of open models, they also don't just like solely create or control that risk. And they're working very hard on the model mitigation side. But there is a kind of an emerging post-AI biosecurity ecosystem of startups emerging, which is very exciting to see. So there's Ginkgo spun out a new company called Perimeter Bio. There is SecureBio, who's a nonprofit, has been doing a lot of work in this area. There's the sort of nucleic acid observatory projects. There's Sentinel Bio working on DNA synthesis screening. On the kind of more analysis side, there's Valthos, who actually I'm not super familiar exactly with their work, but my understanding is that they're more focused on analyzing potentially pathogenic sequences and estimating the kind of threat levels from them. And then there's us. And that's actually, I think we are now, I think it's either AbSci or AbCellera or one of the sort of Ab-named antibody design companies did announce a business unit that at least has the word biosecurity in its title. So, but other than that, I think on the sort of specifically medical countermeasure layer. Yeah, I think we're currently the only one.

You guys have— you have a wet lab?

We do. Yeah, you do.

Okay. And so, I mean, I know you don't want to be specific about things, exactly how things could play out in a bad way, or to give anyone ideas of these things. But— and I suppose there's myriad ways that things could go pear-shaped. Pear-shaped is probably an understatement. But Like, is the biggest fear an AI system? Um, I mean, I guess the biggest fear is what? Is it's that there's just some real asshole out in the world and they now have the computing power, or a nation-state that just wants to be a total pain in the ass? Is that really like— is that number one on the list, or it's just that, um We're heading to this world of AI biology and shit's just gonna get really weird, and there could even just be mistakes happening, and, and that's the bigger fear.

No, I think, I think all of the above are relevant, uh, scenarios. So, so lab leaks can certainly happen, and, and, uh, and, and the, the, uh, uh, but, uh, I, I think foremost in our minds certainly, um, are the, the sort of doomsday cult school shooter, lone wolf types who are at the moment getting the biggest uplift in comparison to what existed before. So, yeah.

Can I ask a question about that real quick? Sure.

Of course.

Because this is one thing I was thinking about. I mean, so the thing— because we haven't seen a lot of, unless I'm just unaware, of like, you know, school shooter type individuals doing bioweapons, right? So, and clearly you know, it takes money, expertise, all these things that presumably in this new world AI could lessen and, and put this, this power and knowledge in other people's hands. But I mean, is that the thing? Like, we just think people are such— there will be just this group of assholes who just now have these tools at their disposal?

These people unfortunately do, do exist, and they're, uh, the— there is at least one disclosed or documented example of a school shooter, at least like notebooking, maybe making, like considering using biological approaches. An interesting historical example is, although not specifically bio, but very comparable, is Anders Breivik. He was a Norwegian mass murderer who sort of carried out this horrific shooting on an island of the youth group of a political party in Norway, but also who made a— created in his home a chemical explosive that he used to blow up successfully, that he used to blow up a government building. And there's a RAND publication that analyzes what he did in some detail, because he actually kept detailed journals. And this was him with like 2011 internet resources. And he just kind of iterated his way to the Rand, and so successfully just solved each problem he encountered on the way. And so Rand kind of analyzed the complexity of what he did and concluded that was actually far more complicated than what viral culture would be. So I think, I mean, we have to remember how early we are in all this. Like DNA synthesis capabilities are not that old. Like the sort of design uplift from the kind of more sophisticated models is extremely recent. And we just haven't sort of had time to see this fully play out. But the physical, at least for some viruses, the physical production of enough virus to infect hundreds of people is chillingly not difficult. I mean, it's sort of within reach of people who have basic cell culture skills. And then that is millions of people. There's millions of people.

Isn't there a whole season of 24 about this very thing?

Oh, I did used to watch 24, but I don't remember the bioweapon.

Some kind of bioweapon. Okay, sorry. Yeah.

So yeah, so a couple of more comments. I mean, it is of course hard to say specifically how this will play out. And we don't know what the success rate of these attempts will be initially. So my expectation is that Rather than us getting a new super pandemic tomorrow or next year, there will be sort of an escalating number of increasingly successful attempts, which will then be these sort of more localized incidents.

Meaning what? Like a little mini sickness or—

Or like, you know, you successfully infect 100 people or more and that there is spread, but you haven't successfully made like a mega infectious virus.

Okay.

And then that's why it's then important to have this like rapid fire station-like approach that allows you to also suppress that. It's of course even more important for the successful ones, but it also already kind of mitigates the impact from these, which will hopefully be like less catastrophic initial incidents. I mean, that's at least like one model. One can apply.

So you think in the first instances, they're just not good enough at building something that's airborne or that is able to escape, you know?

That's the, yeah, I mean, that's the kind of more optimistic take. Like the, this will not be the case, this will not be the case forever. Like I think it is probably the case right now that if you were to, that the sort of at least your hit rate wouldn't be very high. Okay. That you might have some modest successes, but like not necessarily like in the truly catastrophic sense yet. But I also think the timeline for that shifting and sort of tail risk going up is not very long.

Yeah.

The other thing to consider is also I mean, you did mention also state actors, and we certainly cannot ignore them here either. There is the Bioweapons Convention, but it is also enforced, or it's practically not really monitored, enforced. There are 3 people whose job is to to sort of monitor everything that is going on in the world that could be violating the Bioweapons Convention. And we do know that there are a number of rogue states that even in a non-classified sense has been verified that they have active offensive bioweapons programs, and they will of course have access to these tools. So there the worry is then that If you are a state actor, the kind of classic game theory consequences of using bioweapons historically have been that you might get blowback. It might sort of come back to your own population, so you don't want to do that, or it might be traced back to you and therefore you would then get nuked.

I'm doing my best not to raise questions about China here.

Not mentioning—

Whether we've already done this.

Not naming names here, but the— or pointing fingers. But the kind of more advanced biological capabilities might shift that game theory. If you can now make more targeted viruses, for example, that would not infect your own population, but only your target population, then that changes the game for the blowback side. Or if you're able to very easily simultaneously develop sort of the countermeasure, ironically, here to kind of in tandem with your pathogen. And then, yeah, here I don't want to sort of necessarily get into too many specifics, but like the attribute, attributability part might also, also get much harder.

Yeah. Well, I mean, I mean, honestly, it's mind-boggling. All the— I mean, I get what you guys are doing and it's important. It just, it seems, God, it seems like a Herculean task, you know, to try to— I see what you're saying. I mean, maybe you don't have to You're already— the good guys are ahead in some ways, but man, I mean, it just seems like the optionality of the other side is almost limitless as to how creative you might want to be.

Well, I think what I sort of take comfort in is the kind of biological analogy. So if you look at the So just to kind of think about the numbers, and I'm kind of, these are probably like, I'm getting some orders of magnitude wrong here, but if you take the combined genetic information in pathogens that can infect us, it's like 10 to the 18 base pairs or something like that. So 10 to the 18 A's, T's, C's, and G's. And our genome is 3 billion base pairs. So that's like a billion-fold difference. It's like billion times more enemies than we could possibly imagine countering. And plus they evolve and then they're like, they're not static enemies. And they also have specifically evolved machinery to mess with our defenses. Race. That is the biological reality, ignoring AI. It has been kind of the biological arms race for billions of years. And yet we are alive. We are the proof that we can win. And that is— and sort of a big part of the answer was the kind of adaptive immune system, the ability that you actually can very rapidly develop a specific response. I mean, there's other things that one has to do too. There's just a concept that people have increasingly been talking about and which, for example, now OpenAI Foundation has a huge program on is AI resilience. How do we kind of just generally prepare society for the impact of AI? And kind of in Darkome, there's this extreme version of the end state where everyone has an mRNA wearable. But I think we just need to kind of consider some radical upgrades to how we structure our built environment. How do we manufacture easy things? Like how do we manufacture drugs? How do we sort of operate in these crisis situations? How do we organize society? But I honestly think we can. I think we will get this sort of AI-powered civilizational adaptive immune system.

All I'm picturing is all of us walking around in spacesuits on Earth.

That was kind of the COVID punk aesthetic. No, I think the immune system means that we don't need the spacesuits. I mean, they are kind of the immune system is the spacesuit. But another kind of cool thing about Like digging into the evolutionary biology, actually, another interesting analogy, part of that story was that, so we had this like Gnastothom, the kind of ancestor of all sort of jawed and spined creatures that was probably like a fish living in the Ordovician Sea that sort of had this integration event that that led to the adaptive immune system. Before that, it was like, it was dying all the time. Like you kind of, the evolution had to play the numbers game of have a small creature which lays a lot of eggs and has large populations that can survive this constant viral or pathogenic onslaught. But once it had the adaptive immune system, once it was able to defend, then evolution was able to allow it to— I'm sort of obviously personifying evolution here far too much, but what then became possible in the possibility space was larger bodies or warm-bloodedness. Because I mean, being warm-blooded is a great incubation platform for viruses. So you need to really be able to defend well to be able to be warm-blooded. Fact that we were able to then kind of reach our current stage of evolution was largely due to the ability to better protect ourselves from viruses. And I think there's some interesting analogy there of like the future trajectory of us and humanity of like to be able to embrace the full potential of AI where it can take us. We just need to be able to better defend against the threats that it enables. And then to do that, we then need to, just like the fish did, we need to sort of integrate it into our defenses very deeply.

I'm having the feeling that I have often when I talk to people who are actually elite at science fiction, which is like, I feel like someone like you needs to be president in this moment in time because the average person is not nearly creative enough to sort of like be ascertaining what is about to happen to us all, or like how we should at all try to adapt and for the world that we're creating. And Lord knows, like our typical politicians are quite meager at the basic, more basic world we had already created. Yeah. So, I mean, I just, I love to You know, you're so fucking good at— I mean, just amazing because you're brilliant outside of being a fantastic author. And then, you know, just have somebody who's so knowledgeable in math, physics, bio to be able to process all these worlds. I just think it's a very valuable skill. I can see a future where the the very best sci-fi theorists among us are really the ones we're depending on to navigate this.

Well, I think as we talked about in the beginning, I do fundamentally believe that we all have this capacity to imagine worlds that, I mean, I think maybe it should be more, I don't know, children, our children who should be imagining it more than us.

It's probably not a terrible idea. I mean, and There's— I see what you're saying about all kids having this capacity. There's definitely something to exercising that muscle because my son will just blow me away with the way his imagination works and the way he can instantly spin up a story, like a quite complex story out of almost any scenario. I'm so jealous of him, you know? And then I remember, I mean, when I was his age, I wrote a lot of fiction. Which I almost feel like I can't even— when I was a teenager, it was just limitless ideas for short stories, and you know what I mean? It was just endless things to explore, and now I feel like I'm really searching for things to grab onto, and it's, it's like sad to feel that happen to your brain over time.

Um, I think it's possible to rediscover that. Um, I, I, um, I don't necessarily have have sort of very actionable advice on how to discover that. But I think like one—

I've done drugs.

But I mean, but you, but you, but you, but I mean, you get exposed to so many cool ideas though.

Like, I do, but my brain now, it's like, I love it. But that's what I love living in it. And I like writing about it in this nonfiction way. And I like filming movies, showing it to people, but that like turn in my head of extrapolating on my own is a bit muted from where— when, like, when I was a teenager, I mean, I would just fill endless notebooks of short stories, like, every night. I mean, it was like a new idea was coming.

Well, I think obviously it's just also great to do the thing you are uniquely suited to do. And what is your sort of superpower in getting people to tell about these wild things and create a synthesis that also helps others understand that.

It's very nice of you to say. I'm jealous of your skills. Well, I know, I mean, we could look, I mean, maybe you can come back when you guys have been going a bit and we can explore some more of these scenarios. I could keep you here for like 8 hours, I think, going through all the stuff. Is there anything for people to The Red Queen website is short and sweet at the moment. Is there anything— well, you guys will be publishing papers or I don't know, just what can we expect to see over the course of this year, if anything?

Yeah, so we might in the future publish some more joint work with labs. I mean, by the very nature of the work, we probably will will by default be fairly quiet about it. But yeah, so we have a, I tweet about Red Queen things occasionally at @hannu, H-A-N-N-U on Twitter. And Red Queen also has a Twitter feed.

You do powerful tweet, tweet, what are we supposed to call them?

Threads.

Threads. Yeah.

Oh, thank you. Yeah. I do like Twitter threads as a format. It's a nice form.

No, they're good. I was reading a whole bunch of them last night and it was extremely helpful. Well, thank you. Thank you so much. Yeah, honestly, you got to come back because you're so fun to talk to. And it's like even it's just hard to know where to lead the conversation because I feel like anything we— I'm like, oh, we could have just talked about LARPing for 2 hours.

Yeah, absolutely.

Um, but thank you so much for coming on.

Thank you. Thank you, Ashlee. It was my pleasure.

The Kore Memory Podcast is hosted by me, Ashlee Vance, and/or Kylee Robison, or both of us together. It is produced by me and David Nicholson. Our theme song is by James Mercer and Jon Sortland, and the show is edited always by the John Sortland. Thank you so much to Brex and Send Cut Send for all your support, and thank you most of all to everybody for listening or watching. We love you. Please leave us a like, a review, a subscribe, all those tremendous things. Thank you, and we'll see you again.