AI Will Cure Us All!! Or Not!! - EP 84 Abhi Mahajan

All right, here we go with one of the most important questions of our age. Will all of this AI technology that is unfolding before us improve biology and biotech? Will we be healthier? Will we live longer? Will we have cures to all kinds of diseases? Silicon Valley in the biotech world would try and convince you, yes. There are about 47,000 AI biotech companies. They say they're Folding proteins, coming up with antibodies, coming up with therapies faster than ever before. I think there's a camp of people that would say, no, biology is messy. It's going to stay that way. You've got regulation, trials, all kinds of things to do that will keep things running at the pace they just about always have. To discuss this, I am stepping aside. I'm turning the microphone over to Aaron A. Marogi. He's on his way to becoming a doctor. He is an accomplished scientist, and he writes for Core Memory quite frequently. And he is going to be interviewing Abhishek Mahajan. He now works at the OpenAI Foundation. He has also worked in biotech. He writes the Owl Posting Substack, where he, he writes in detail about biology and biotech. Wonderful detail. So these two guys are going to go at it. Aaron wore a suit and everything to be respectable and show me up. I can't think of two smarter, better, more knowledgeable people to be discussing these issues. I think you'll find it fascinating. I should note their views are their own and not representative of the OpenAI Foundation or whoever Aaron. is beholden to in his professional capacity. It's just these 2 gents thinking big thoughts and walking us through this incredibly important question that may be one of the most profound and important topics of, of this century as we know it. So I am Ashlee Vance. This is Core Memory. Here are 2 other guys on the show.

I'm Aaron Amorogi. This is Core Memory. I know it's not Ashlee talking, but we have a special episode today about biology and AI. Our guest is somebody who is quite well known in the space. Some people know him as the owl poster. I personally know him as Abhi Mahajan, who I worked with at Dyna Therapeutics for a couple of years before he went off to a company called Noetic. And now most recently, he started at the OpenAI Foundation. So it's a pleasure to talk to you, Abhi.

Pleasure. Pleasure for me as well.

So I guess my goal from this episode is to engage with some of the ideas, I think, around whether sort of maybe develop a taste for whether I actually believe that AI can accelerate the pace of biology by throwing unlimited intelligence at the problem. So can you just sort of think about stuff really hard, and does that get you far enough along to move faster than the speed at which we've been moving the last maybe 500 years or so?

Sure.

sounds like an interesting discussion.

Okay. All right, so just starting us off here, then I have a pretty maybe simple question, which is, do you think that the bottleneck in biology is intelligence or a sort of physical grounding of reality through experimentation?

I guess it depends on what stage of— at least for the problem of drug discovery, I guess it depends on what stage you're operating at. I think at the preclinical state, I feel like you can make an argument for any— every side has kind of a place to stand on. I think for the intelligence maximalists, they very well could say that, yeah, doing things in the physical universe requires time and people, none of which are replaceable by intelligence. But at the same time, you could imagine superintelligence could state what experiments are most worth running. And that perhaps lets you do the preclinical experiments that are perhaps best. At the clinical stage, probably intelligence is not all that useful. It's primarily like a human coordination problem, which, I don't know, maybe in the aggregate, superintelligence helps plan out the built environment in such a way that it helps. We had a friend of ours who went to China and said the big advantage of running clinical trials there is that you can recruit from kind of anywhere in the country because there are trains that will get you to the major medical centers. And so in that case, maybe superintelligence helps plan that out better. But you can make kind of contrived arguments as to where intelligence is useful.

I think you could say the physical grounding in that way is like, how many trains do we have running between Chicago and MD Anderson in Texas?

Yeah, I think for the most part, it is primarily a human coordination problem. I think when you get to post-approval, I think superintelligence actually is pretty useful in discovering What was that drug? Like Vioxx. It was an antihypertensive that turned out to have major cardiovascular events a few years after its approval. I think in cases like that, maybe intelligence really is good for roaming through— what's that self-reported database? Apheresis, something like that.

I don't know about that one.

It's something run by the government where people can— it was a big thing during COVID where people can Oh, self-report, like, the side effects?

Like what you're supposed to do in phase 4s?

Yes. Yeah. And maybe superintelligence is really great at roaming through that and discovering things that humans could discover, but we just simply don't have enough epidemiologists on hand.

Yeah, I guess a follow-up to this. I'm not a mathematician. I don't think you're a mathematician, but I think you probably have more of a sense for math than me. So I think a couple of days ago, maybe it was yesterday, I'm not quite sure. I don't quite remember. The days blend nowadays. There was a big GPT, I think 5.7, SOL thing where they had 10 theorems that they solved, or I don't know. I'm sure mathematicians are going to be really upset at the way I framed that. But supposedly the really big deal, and I was sort of reading something on Twitter, which we know is like a grounded reality, that Any number of these could be like a fields-level discovery. I guess in my mind, seems like superintelligence, at least in a mathematical sense to me, at least definitionally, people have always thrown around, could you have 1,000 von Neumanns in a data centre? Kind of seems like we did. And so we got this outcome, right? I guess the question is, does 5.7 do anything to biology? right now. It feels like we actually do have it. Maybe you can make arguments that it's not quite superintelligence, but we have something pretty close to something that looks like superintelligence. Do we think this is going to— like tomorrow, right? We should be picking up the pace really quite rapidly if superintelligence is a bottleneck.

I think the common counterargument here is that math is an extremely verifiable domain. And so your feedback loops are extremely tight. And actually, I think the essay I think a lot about is written by an ex-coworker of ours, Stephen Molina, who wrote this thing about fast feedback loops. Oh, that fast feedback loop. And I think if you can imagine a future in which there are robots in a lab that can plan out their own experiments and learn from them immediately and then plan out the next set, maybe that is really the substrate for genuine superintelligence in biology in the same way you see it in math. I think today those feedback loops are roughly as slow as they were 5 years ago, whereas in math, the feedback loops are always tight. It was just like intelligence genuinely was the bottleneck. Now the bottleneck is both intelligence and also human hands to do everything. And from my conference, I wrote this article about lab automation a few months ago, and I got the impression that the problem with aggressively automating a lab is simply that most experiments are not worth automating. There's a few select experiments that, like mixing compounds, doing hepatocyte drug screening, those things, sure, you can scale up an infinite amount of times, but they also aren't super informative. The most informative experiments are kind of just like you tinkering around and trying things. And maybe we'll eventually arrive to the point where a robot in a wet lab has the dexterity necessary to do all those superfine mixing experiments and swabbing a cell plate with stuff. But today we're not really there.

Yeah, I guess. So I read that article that was fairly recent. I want to say like 6 weeks-ish, maybe. When I read that article, I think in my mind it's probably worth considering there is lab automation, which is maybe translating, making a PCR strip that you're putting all the reagents, whatever, translating that process into a plate-based assay that a CNC machine basically is doing, or liquid handlers, we call them in lab biology. There's that kind of lab automation, which I think speeds you up maybe on the order of less than 10-fold. Maybe if you run things 24/7 all the time, that gets you some acceleration. But actually, what I think about is interesting. So one problem we worked on together at Dento Therapeutics was the idea of, can we speed up the evolution of proteins, specifically viruses? Prior to, I would say prior to next-generation sequencing, this was something you did manually, right? So you had to make a virus, you had to You have to design the virus, you have to make the virus, you have to transfect the cells, you have to infect the monkey or mouse or whatever, get the data out and be like, okay, did it work or did it not work? And you're doing this maybe on the order of dozens-ish viruses because you have to have some reporter in those senses. Maybe you attach a GFP to one, RFP to another, something like this, maybe immunostaining, but ultimately you're gated pretty quickly by the assay. So not the The automation, you could have automated that process, and maybe instead of doing 12 viruses, you could get 48 viruses or 64 viruses, but you're not getting 11 million viruses. But the thing that kind of changed that was next-generation sequencing. All of a sudden, you could get, in some cases, with some of our viral libraries, billions of viruses in one pool. That, to me, is actual data acceleration.

Yeah.

That has nothing to do with automation.

But at the same time, then the bottleneck moves to a new place. Now your problem is that the mouse blood-brain barrier receptors are not the same as the ones in NHPs, which are not the same ones as humans.

Right.

And so even if you scale up an infinite amount of the NHP, you don't get all that much closer to solving the delivery problem for humans.

Yeah, I mean, that I'm completely with you. I feel like maybe that's actually a good segue into my next question. So reading one of your essays, I actually didn't know about this history, I've read the word. I've never said the word before. I'm going to try. Tigit?

Tigit?

Is that how they say it? T-I-G-I-T?

Oh, Tigit. Tigit.

Tigit.

There we go.

Tigit. All right. Okay. You learn new things every day. So, like, looking back at Tigit, so there was kind of like a— maybe to put some context, there was a sort of sense maybe in 2010s, around 2010, 2015, that there could be this new pathway for killing cancer cells. where there's this thing, TIGIT, and if you knock it down, you could unlock the sort of immune system of the body to go and kill cancer cells. And there was some interesting preclinical data, maybe like 2020, and this then moved into a bunch of companies investing into clinical pipelines in this. And all of the preclinical data was actually pretty rich and was pretty concrete. in saying, oh, this is going to work, this is going to be it. We're going to be able to move past checkpoint inhibitors. This thing is going to change the game for cancer. And all of the trials failed. All the phase 3 trials failed. Maybe one way of putting this question is if you took that data, all the data that existed, both maybe all the preclinical data, and you fed it to GPT-5.7, or maybe Mythos. Mythos, I don't know. I have no idea how good that one is. But you feed it to the most frontier genius model with unlimited credits or API tokens. Do you think we get a different assessment for whether Digit is going to work for cancer in a human?

I forget which article I did this in, but when O3 preview came out, I ran this exact type of experiment. where there was this major drug failure that I forget the exact name of it, but it happened in the early 2010s. I basically asked the model, pretend you don't know anything about the future. Was there sufficient evidence for you to predict whether this was going to fail? And I did not find the answer very convincing. It was kind of like a lot of— it was in the pre-training data that the drug had failed. So it a little bit distorted itself to say, Yeah, there was sufficient evidence, but it didn't really seem super convincing. Maybe 5.6 or whatever the Astra version of 5.7 is, maybe that changes it, but I'm skeptical. And I think this is also true for Teget. I think the argument that superintelligence would have against Teget is that this is mouse data. Be careful. Don't spin up 12 clinical trials based off of this result.

Right.

I think if you were to put yourself in the headspace of someone at, I think it was like Roche.

I think Roche was the first one to do this.

This is like the golden goose. The whole point of blockbuster drugs is that, or I guess a large part of pharma is like venture-style bets and that you do kind of need to go all in the second you discover something that has some promising preclinical signal and and it empirically has just a massive market. And so maybe from the standpoint of Roche, even given the failure of Tigit in the years since, it wasn't actually that bad of a bet from an expected value standpoint, and maybe it'll pay off someday. And so I remember I wrote this article about will AI be all that useful for clinical drugs currently in clinical trials. And I talked to a lot of clinical trial consultants for that piece. And this was last year or so before the current state where the models are at. And the general impression I got was, no, they're not going to sway anyone's opinion for drugs already in phase 1 through 3 because they've already made a bet. They're not going to shift away from that bet just because of what a model tells them. And you could argue that maybe 2, 3 years down the line, people are sufficiently convinced by the model to turn away from it. But a lot of the times these trials are ran more on emotional conviction rather than scientific ones. And I guess my overall point is, one, I don't think superintelligence would have predicted the failure. Two, even if superintelligence had predicted the failure, I don't think people were well rewarded for actually trusting it, and they would have just ran it anyway.

I see. Yeah, it is interesting to consider whether you would have been rewarded for going like, oh, they're pointing out something very useful to consider, that this is a mouse, and so maybe we shouldn't do it. I guess there's a component. The dangerous part about the superintelligence thing is people can always change. They can shift the goalposts and be like, well, superintelligence will simply be able to know what the relationship is between preclinical data and how likely it is to succeed in a human. So I should say, maybe from this point forward in our conversation, I'm going to assume that that's not going to just happen by just training on more internet data.

Yeah.

or more RL training. I don't think it's going to necessarily work out in that way, but maybe I'll be wrong. To maybe sort of change the direction of the question, then, in this case, based off of your little experiment, and maybe we can apply it to Digit as well, what kind of data do you think was missing that would tell you that this actually was not going to work in a human? And maybe the obvious answer is, like, human data.

I think the obvious answer is human data. I think it's a little bit of a siren song to think that you can just magically discover human-relevant data from these super scalable assays, either in vitro ones or ones done in animals, and it just immediately transfers to humans. Empirically, that just rarely turns out to be the case. It did vaguely turn out to be the case for PD-1 drugs and regular checkpoint inhibitors. It did turn out to be the case for chemotherapies. Not too insane to imagine that TIGIT was also there. And I don't know, I think the original thesis for TIGIT was, one, it seems to be highly elevated in particularly aggressive tumours. And 2, knocking it down in mice seemed to lead to some responses. Not a terrible pitch for a future cancer drug, especially if, like, I'm sure if you looked up TIGIT around the time prior to this pivotal paper being published, you would have found a bunch of like, hey, TIGIT is implicated in all these other immune processes. And I don't know, your eyes kind of filled with dollar signs at that and you just go forward with it. I think it feels like rarely the case that blockbuster drugs have this super clean story to it. And in terms of what data would have actually convinced people, I'm not sure anything would have sufficed other than taking it slow and doing a phase 0 with an academic center first. But then there's the obvious incentive against that is maybe you run that, you get positive results. This was done in 2020, so maybe this wouldn't apply. But if you do it today, now China is running multiple phase 1s and phase 2s at the same time as you, and they beat you to the punch. Back then, your competitors would be still slower, American and European pharmas, but still you'd be worried about them catching up to you.

And yeah, I guess it sounds like the thing that protects you as an American company in this case is that a Chinese company can't just take a drug and say, like, oh, we have all this data that works in China, we want to sell it in America now. You have to actually run the trial again in America. At least today that's the case.

Yes.

I expect this will probably remain true for a while. So there's some, maybe call it like the— we will once again say the verification layer is the thing that protects AI from, or whatever system, from really accelerating us. But I think the thing I'm personally just trying to develop a sense for is what kinds of problems actually are interesting for AI when it comes to biology. I know that you particularly worked on something at Noetic, so the company that you worked at after you left Dino. And I'm going to try to explain it to my understanding. I'm kind of like, I'm on the left tail of the IQ curve, so correct me where I'm wrong. My understanding of things is that the belief system here is you can take cancer cell lines and you could introduce drugs and develop these Perturb-seq type models that basically tells you generally how a particular cancer cell line is going to behave to different perturbations. And so when you give a new drug, maybe you can actually do some latent space if you have this model, you can actually just explore similar areas around the drug that you are testing, and you can get a sense for, like, actually, this will work or this won't work. And that might speed up your ability to maybe speed through some of the screening of the preclinical. Do you feel like— okay, I guess the first question is, do you think that's an accurate depiction of what you did and what Noetic is all about?

I do not think that's an accurate description. Okay, let's hear it.

I want to hear from you then.

The thesis of Noetic is you can take a bunch of biopsy samples from a bunch of real cancer patients, like resections. You can collect a bunch of high-dimensional data from them. So specifically spatial proteomics, spatial transcriptomics, H&E, and an exome panel. And you can feed all of that into a model as a self-supervised learning task. So The end result is that you have a model that can create a sort of embedding space of all the tumor samples that it sees. And one way to ensure that this embedding space is well organized is that, hey, maybe squamous carcinoma is here, adenocarcinoma is here. They're separated by known legible human categorizations, plus within them they also contain categorizations that are only known to the model itself. And so once you have this, now one interesting thing you could do is fine-tune it on real clinical trial response data, because ultimately clinical trial response data is the rarest type of data that's out there. And so unless you have a model that already has a pretty good understanding of cancer, it's difficult for it to just generalise to this new space. And so what the company is interested in is taking biopsies from patients who have been through this investigational phase 1, 2, 3, running through the model and seeing if it's fine-tuned on the set of 7 responders and 6 non-responders, can it learn some notion of what was so special about the responders? And can it place responders here in the embedding space and non-responders here in the embedding space such that there's a clean separation between the two? You don't know where this inclusion rule comes from. Only the model can really know. But Maybe there's clues. Maybe you can see that as particularly high levels of this particular protein, high levels of these set of transcripts that are immune related. And so from there, you can do kind of one of two things. One, you can go to the company and say, hey, you should probably run this in whatever future clinical trial you run, and you should probably add in this inclusion criteria to ensure that your objective response rate in future trials is really good. 2, if the drug has failed and the company simply doesn't want it anymore, we, or they, Nordic, could simply buy the drug and run it in a clinical trial that has a really high expected value of success. And there is this separate perturbation platform that is, you do the exact same thing as I described, but with mouse data instead of human data, and you apply perturbations. Because it's a mouse, you can apply perturbations to the tumour microenvironment and you can, similar to Dino, you can multiplex them. So you can have, I think, dozens of tumour subpopulations inside of a mouse lung. You can take that out. And one of the interesting— we wrote a blog post about this, but one of the interesting things we learned is you can take this model that has been pre-trained on only human data and run the mouse data through it. And if you run the mouse data through it, you kind of get this projection of mouse data into a human landscape. And there's a few reasons why this might be interesting. I think the value proposition is still a little bit fuzzy, but it's interesting that you kind of can learn, because what comes out of the system are human transcripts. They're not mouse transcripts. Mouse transcripts have been mapped to some degree of humanness. And when you mentioned this thing about, oh, you can perturb the model itself, one interesting thing about this whole setup is that it's not just like an autoencoder, it's a generative model. So you can take the input and say there's an elevated level of this particular gene, maybe it's TIGIT, maybe it's something else, and you can see how the predictions themselves change. Maybe if you elevate these 5 genes, you can see that a bunch of these other inflammatory markers also go up. So probably a good thing. And so maybe those represent really good targets, and the spatial mouse perturbation platform acts as sort of like a ground truth for how it actually affects at least mouse tumours. I think the perturbation part is the least well understood. I think the The pre-training to fine-tune on clinical trial stuff is like the most like well-fleshed-out part of it. No.

Okay. Yeah, no, that's it. I appreciate your redirection.

All right, what do we do at Core Memory? We cover innovative, fast-moving, forward-thinking companies, which is why Core Memory is sponsored by Brex, because Brex is the intelligent finance platform for many of these companies, 30,000 companies from startups to the world's largest corporations. Rely on Brex's technology for their finances. They've got smart corporate cards, high-yield business banking, and expense automation tools that are fantastic. I hate doing my expenses, and Brex's AIs and software run right through those expenses, figure out where we're spending money, and take care of so much stuff for you so you don't have to waste your time on it yourself. Go to brex.com/corememory to learn more and just, you know, get with the program. Let's get going. Let's get out of this archaic finance software and move toward the future. Core memory and Brex.

I'm wondering if you— in my mind, it makes sense to at least approach the problem. If you're going to try to address the main issue, I guess what we're trying to say is the main issue, which is a translation from preclinical into clinical. It sounds like really you're just trying to map mouse, which we'll just say broadly is the most common preclinical data you can have, if you're trying to map that into existing clinical datasets, so like phase 1, 2, 3 trials, 4 trials data, it would make sense you might learn something and it might increase your hit rate. I don't know. Do you have a sense for maybe roughly this is increased from like 5% is kind of the number that people throw around when it comes to preclinical to clinical?

So importantly, the mouse data is almost entirely for novel target discovery. It's It's not used to necessarily connect it, because if you have clinical response data, you don't need to even bother with the mouse data.

Of course. Yeah.

In the case where there is no investigational drug that hits these particular sets of targets, then the mouse data becomes valuable. So it's almost like a backup.

Yeah, I see what you're saying. So you're saying it's to explore a new space, which I get, and you're hoping that you can sort of learn these representations of maybe drugs or some molecular interactions.

Oh, and so your question is, in the case of novel targets, how well do you expect them out to do?

Yeah, exactly.

I think the answer is unknowable. I think it's like, as with all things, it's unverifiable, maybe difficult to verify. I think this is the problem with AI and bio. And I think especially, I think the protein people and the small molecule people are very lucky in that they have identified interesting hills to climb. So improving IP in the protein world, good. It's a good metric. It seems to correlate well with in vivo efficacy to at least some degree. So yeah, let's pump those numbers up and we'll get somewhere interesting. I think with regards to in vivo efficacy of immune IO drugs in general, I think the hill to climb for in silico models is not well defined, so it is very much reading tea leaves. And I think that's kind of the reality where you have low N, you do kind of need to squint your eyes and hope for the best. And this might sound like a horrible thing, but the counterfactual case is not having the model at all. So maybe it will be better, and then the worst case, it does nothing at all, which is exactly where we begin.

Yeah, I actually think the counterfactual is like, are you better off taking $1 trillion of investing and putting it into something else? I don't know. I don't know.

That's true.

That to me is actually the counterfactual. But to be clear, I actually do. I obviously believe in AI. I do work in an AI and bio company. I've kind of worked within AI and biology and medicine, so I do believe in it. I think the more interesting question, I think it's undeniable that you can find narrow verticals where it's like a useful thing to work on. Protein binding and making binders. You, I think, were like the first one to really convince me that we solved maybe binder generation because now people can make binder libraries that have like 10% hit rates, which is way better than like the 0.1% hit rates we were getting 10 years ago. So there are these narrow slices where I think you totally can apply ML in very, very accelerationist directions. At Dyno, we were making extremely good viruses much faster than, say, the 25 years that preceded maybe the AI and bio landscape or field. I think the challenge is, you and I had a discussion, I think at dinner earlier this past week, where we talked about, is text actually just the only thing you need? Is text data actually the only thing you need to learn about stuff?

Yeah.

And in the case of biology, that text data can be protein sequences, it can be DNA, RNA, it can be transcription data, it can be a lot of different types of text. But does this text actually tell you everything you need to know? I don't know. I'm not really sure. It's hard for me to say. Seemingly, this is the most obvious, but it is the important question, really returning to the foundations of, are we generating the right kind of data to actually make massive leaps in biological discovery?

I'm not sure if anyone has necessarily tried doing this yet, but I've long been— because drug discovery existed before computers, and there were these very smart med chemists who had very strong intuition as to how drugs ought to be designed. Like, what was it, Lipinski's rule of 5? I think there are a bunch of these tiny little chemistry rules that seem to directly map to, like, you're supposed to have these particular number of hydrogen atoms, you're supposed to have these particular chemical motifs, and that seems to work really well in practice. And I think it is a really fun idea to imagine making superhuman medicinal chemists. Who, okay, maybe they aren't superhuman, but they are these scaled-up versions of these extraordinarily rare humans. And so imagine, I've never met one of the Baker Lab protein wizards before, but I have to imagine that if you had just a billion of those in a data center, you would get something useful. Maybe human intelligence, human intelligence filtered through an LLM, is like a very good regularizer almost in the sense of a pathology of a lot of small molecule foundation models is very easy to get them to overfit on a particular slice of chemistry. But the LLMs have almost an incentive. This is almost like a tortured analogy, but they have an incentive to be broad and they don't necessarily overfit in the way that we think of overfitting. And maybe if you encode medicinal chemistry knowledge, if you encode drug campaigns as you start with a SMILES string and then you include some reasoning text as to why you need to move on to this different SMILES string, then you move on to another one. Maybe encoding reasoning traces like that into natural language models is a really interesting direction for how you make superhuman medicinal chemists. And maybe you get really crazy leaps as a result of that. I'm sure there's a ton of things in structural biology and toxicology and on and on which are not encodable via this format, but it feels like there's an underappreciated fraction that is.

Yeah. Okay, so my knowledge of chemistry sort of ends after physical chemistry in college. There's like a limit to what I know. My sense from taking chemistry in college was that we kind of answered so many of the interesting questions there were to answer. And maybe physics, we actually did this as well. And now we are returning to a place where, like, it's harder to come up with simple rules for these things. You know, like, a good example is, like, if you think about small molecules, there are rules for synthesis that you can just apply.

Yeah.

And they're actually so simple that we teach them to every 20-year-old pre-med who wants to become a doctor in this class called organic chemistry. And part of that is just the nature of, like, small molecules being simple. I mean, they're flat. They're obviously not flat, but like, they are kind of flat. You can really flatten how much complexity there is to them. And yeah, you will have difficult things here and there, but so many So many of the principles of chemistry are actually genuinely just things you can think kind of hard about and then be like, yeah, that kind of makes sense. Biology feels weird because even proteins have this sort of— yeah, there are rules to maybe how they fold, but that doesn't always tell you how they're going to behave. And maybe we will figure out that, oh, it's actually this really simple thing. You just got to look at the 14th amino acid and how it relates to the 3rd to the last amino acid, and that tells you how it's going to move with a ligand or something. I don't know, whatever. Maybe there is a principle and maybe we just don't know it, but it seems unlikely to me that there will be one that's this simple. And so your only other option is to invest all this money into, say, molecular dynamics and just be like, oh, well, we're just going to generate all this cryo-EM data and all this MD representations of what proteins look like when they flop and fold, and then we'll finally understand Well, you'd be able to predict with 10% accuracy instead of 0% accuracy how an antibody is going to function in a plate. And so there was a lot of words there, but maybe the question that I'm trying to ask you is whether you feel like there's been something that you've seen that you're excited about that points at the direction of actually maybe we can just learn principles. And that is not AlphaFold, because AlphaFold is like the one that everyone goes to because it's like, oh, AlphaFold, we solved protein folding. As you know, we didn't protein folding, because obviously we don't have just antibodies you can print off the shelf.

There's a friend of mine, a bunch of collaborators made this really interesting benchmark that is basically trying to assess how well models, natural language models, can interpret variants or SNPs in particular genes. And the way that the benchmark is set up is I think they provide the wild type sequence and they provide the mutated sequence and they ask some particular question. I think it's like, how much does this break the fold of the protein? And they compared it to leading models and it gets pretty good.

What do you think that tells you? What do you think that tells you? Do you feel like there's a lesson in there?

Well, okay. They were still not as good as the models. And I was initially super skeptical of the results when I first saw them. And I asked Codex, use simple metrics to see how well— use BLOSUM64, use just more substitutions is worse, more deletions is worse, and just create a linear predictor of that. See how well it compares to the models. And it's still worse than the natural language models. So clearly the models understand some vague sense of what makes for a good protein and what makes for a bad protein. And it is at least somewhat at the level where in an ideal world, there was like a Baker Lab person who could also run through this and we have the peak human benchmarks to it. But I don't know, that result convinced me a little bit that the natural language models are actually doing something really interesting?

Nope. There's no argument for me in terms of their ability to give you something interesting. And there's tons of evidence for that, right? I mean, yes, we haven't solved zero-shot prediction of antibody behaviour, but maybe we've improved it sufficiently that, like, AbSci or all these antibody design companies, Nabla or whatever, are going to start printing antibodies in the next 5, 10 years. Viruses, obviously, there's been a lot of progress there. Enzymes, maybe we're getting some momentum, but actually, it hasn't gotten that far. And a big part of that is, like, it's not just about the prediction, it's about predicting how things are going to work in dynamics.

Yeah.

And that's the part that I feel like I haven't seen too— I've obviously seen academic papers here and there where people, like, We predicted how this one cofactor is going to be involved in this enzyme, but we haven't seen necessarily people— I mean, you and I wrote something for Asimov Press about this, but there hasn't been so much where I'm like, oh, yeah, you know what? We're going to start zero-shotting enzymes that can turn a piece of wood into hard plastic if you just add it to a bucket or something.

I think enzymes is particularly hard, and it's actually one amongst many areas in biology where despite having written that article, I feel like I have a very bad understanding of enzyme ML. But I do think at the end of the day, like Nabla and Chai, I think if they end up failing or if there are any major pathologies in the company, it's the fact that the stuff they're predicting is a few steps removed from what actually matters. And so maybe, yes, they could create really binders and they happen to have really good development developability properties by virtue of what's deposited on the PDP. But there's a bunch of these other PKPD steps and admin properties that they're not really optimizing for today, but eventually they will get there. Eventually the data substrate necessary for that will be created and then they'll climb that hill. And maybe there'll be another step after that of can you get cells to manufacture this at high scales? And maybe they'll hill climb that as well. I think one of the adages of biology is there's many bottlenecks present throughout the drug development process. But I also don't think any of those bottlenecks are impossible to hill climb. I think those are just difficult to. And right now, people aren't optimizing for the things that are at least— if they can raise money today on one particular climb, they'll raise money on that, then they'll move on to the next one. I do think whenever people are pessimistic about the future in BioML, they often point to these virtual cell papers that are like, they're no better.

We don't need to name anybody. Friends of the show and all that.

They'll point to these papers that are like, oh, it's no better than linear. the near baselines. And I think those are all correct papers. I think they were very good papers, but I think it's a little bit like, it's almost like, of course they can't be that good. They're barely measuring the cell. Their input is like a SMILES string and their output is like, or sorry, their input is like the cell line, the chemical they're applying to it, or the genetic perturbation they're applying to it. and the output is single-cell RNA readouts. There's a lot that isn't being measured there.

Yeah.

And maybe if you did actually accrue all of those measurements, both input and output, and train a model on that, you would actually see this crazy out-of-generalization performance. I think for a long time I looked at the virtual cell literature and thought, what a misallocation of resources. But I think it's just early in many ways. The problem space itself hasn't been fleshed out enough.

So I kind of have an answer, but I want to hear your answer first, which is why do you think that there's all this data that's not being measured?

I think it's just hard to measure a lot of it. What do you need to measure a cell? Single-cell RNA readout is one. You'd probably benefit from ATAC-seq as well. You'd probably benefit from was being expressed on the surface of the cell and the intracellular proteins as well. You'd probably benefit from knowing the glycans that are floating on the cell surface. You'd probably also benefit from understanding, if you had a bunch of cells together, you'd probably benefit from understanding how cells are communicating with each other, like the secretome, I guess. Yeah, that's the full set I can think of.

Yes, I think you're right. I think your sense is correct. Maybe from our conversation, it feels like the aspect I'd be most excited about seeing AI help with is actually maybe the same thing that I expect a lot of hardware people are hoping as well, is that extremely cheap intelligence allows you to make new ways of working within the environment. And maybe put in a more specific way, I feel like the most exciting problem you can apply AI to is, can we actually just find new ways of creating assays that collect a tonne of different data?

Right.

And so the specifics of what you just discussed, when you have a cell, you can actually only, just by definition of how different assays are done, there are limits to what you can do at once from the same cell. So obviously, you can do batches, right? You can produce, you know, 10 litres of cells, and you can partition one quarter of it to, like, RNA transcription. One quarter of it, you can be like, I want to look at the, like, DNA here, and maybe some of that stuff can be shared. Maybe you want to do, like, a quarter of it where you want to do, like, live imaging within the cell of, like, compartments. Maybe you have to do some immunohistochemistry, you can't get all this data from the same exact cell. You can get this data from batches. And maybe the thing we don't have right now is really exceptional assays similar to next-generation sequencing. So NGS really unlocked a ton of stuff in genomics and DNA, working with DNA. Obviously, a lot of RNA technology really relies on Or I should say, like, RNA-seq really relies on the basics of what we learned from NGS. And then protein engineering, as I discussed with viruses, can really take advantage of NGS as well.

Yeah.

But it's like a specific assay that really broke that open. It wasn't like, yeah, let's just buy more OpenTrons OT2s, and let's just make them screen more plates. That will get it for us. And that's kind of where we are right now. People are like, well, all we need to do is just scale up the number of OT2s we have running, and make it sufficient so that they can run 24/7 and you don't need people to be introducing things. This feels like a losing strategy to me.

Yeah.

It feels like the interesting problem is how can we apply intelligence? How can I get a co-scientist who's actually like a co-engineer to help me? I'm a biologist, I work in medicine now, but I don't really know anything about hardware. I've done things in ML, but as you know, I'm not some exceptional software engineer either. there are limits to my skill set. Good thing Claude Code is an excellent coder, and Codex is also really great at this. Can it actually also be really great at helping me make a new device that doesn't exist right now? That seems like the actually important problem to solve. And if you can throw unlimited tokens at something, that feels like the thing you want to throw unlimited tokens at.

I directionally agree with everything you said. I think—

Oh, that means there's still something you disagree on. I say that, and I think everyone says I would actually disagree with.

No, I agree with it from the standpoint of I think better or more clever instrumentation or assays is probably the single most important thing you could do if you were trying to improve the state of biology R&D. Better methods will beat scaling up bad methods every day. I think what I'm most unsure about is whether the intelligence that is available via the LLMs today are all that useful for that. I have no idea. I do think instrumentation assay research is one of those black magic bio R&D areas where it's a little bit unclear to me. I think the only ones that I've really deeply studied is that yeast mating thing that A Alpha Bio does. And I'm not sure if you've read that paper, but it's a very clever idea that takes advantage of the fact of this is how yeast cells mate with each other. They're male to female. You can take advantage of that property to do really large n by n protein-protein interaction screens. Would that pop out of the models? I don't know. I genuinely have no idea. It very well may be the case that the models are so capable of, they're tireless, they are relatively creative, that maybe If you did try to churn out a billion memos or maybe a few hundred memos of propose this interesting assay to measure this part of biology that's very difficult to measure, and you just hired a bunch of postdocs to look through all of those and figure out which ones are most promising to move forwards, maybe that's actually a really great use of resources.

The good news is I actually think this is kind of a verifiable theory, or at least, right? So what you could do is you could take whatever your favorite flavor of frontier model is, and you can try to see, can we— some of these are very recent, right? It's not like these are 70-year-old technologies that we're applying. You could say in the last 5 years or maybe even 10 years ago, could we have taken everything we know and predict the next step in the technology jump? Could we have learned whatever they're doing from the data that existed 5 years before they started doing it? I think this is not necessarily a fresh idea, but I'm actually surprised that people don't try to explore this in a serious way. My guess is actually you would be able to, because you ever talk to a really bleeding-edge biologist? And I had the pleasure of working with many of them in the Church lab and also at Dino. There were plenty of them as well. you can kind of feel like their brain is kind of always like, you read a paper, it's maybe like a kind of mediocre paper, maybe it's a really great paper or whatever, but you can always have this takeaway of like, what does this matter to my work?

Yeah.

And it feels like that is a mindset that an LLM could actually take on where you tell it, like, I work in this space, and you are just giving it papers and you can kind of say, What ideas do you have to help me from this field? What ideas do you have to help me apply to my field? I don't know. These feel like the kinds of things that are interesting to do with LLMs. And truly, if you had an unlimited token budget, what could you actually do in an interesting way? This is what I would do. I do hope someone actually does these things rather than me talk about them on a podcast.

But I feel like I have had where we're friends with a few people who run semi-instrumentation bio companies. I feel like I have seen them say a few times, like, oh, Claude Code 5.6 just helped me solve this particular problem in the wet lab. And it does directionally feel to me that— I know when I say directionally, it makes you think I'm disagreeing with you, but—

I directionally understand what you're saying.

But I think whenever I hear that, I do get brief glimpses into this future where the models genuinely are aiding the future of whatever interesting instrumentation is about to come out.

But are those like big leaps? So I've used models for troubleshooting things, especially when I'm doing an assay for the first time, but they're not really making these massive leaps for me. It's like my Protein Express. I've never really used this column before. What do you think is going on? It's like, oh, you got to flip the switch or you got to run the buffer through it first. I'm like, oh, duh.

I'm sure there are some assays that are a little bit functionally impossible, maybe not impossible, but at least very difficult for the model to come up with. And maybe one example of this is PCR. You needed the Taq polymerase, you needed the existence of a heat-resistant polymerase to be able to attempt that assay at all. And would a model have been able to say, go find a polymerase that is heat resistant so you can perform this thermocycling? Maybe it could have, but—

Well, you need the idea of the polymerase. It's not even just heat resistant. You have to know polymerases matter. Polymerases are actually the thing that string together DNA. But it's actually not even just that. You need to understand that, well, actually, we build these things one nucleotide at a time, and there's this templated process and blah, blah, blah. There's so many of these steps It's hard to know exactly where, like, you can hit start on and say, like, progress forward. I do take your point. It's hard to know, obviously. Like, so much of this discussion is like, who knows? Maybe. I do think a lot of aspects of our discussion here are actually verifiable.

All right, pod friends, I am here to tell you about SendCutSend. They are a manufacturing phenomenon. Whether you're working on your own car or you're a giant automotive company, a rocket maker, If you need metal cut, machined, or bent, you're going to SendCutSend. They'll do it for you right, and they will do it for you fast. Go to sendcutsend.com/corememory for a 15% discount on whatever you're trying to make. We love SendCutSend. Go visit them today.

There is something more specific I want to ask you about, which is related to a recent article you wrote with organoids. So switching a little bit to models. Do you feel like predicting organoid behaviour perfectly, say literally 1.0, right? Do you think that's a $1 trillion company?

It depends. It depends on, like, yes, yeah, I do. Because empirically, it seems to me that I have an upcoming article about the clear success stories of organoids, and there aren't that many, but there are at least a few in the cancer space where patient-derived organoids do genuinely seem to be pretty predictive of patient response. And I think the reason cancer organoids, I think I could be very wrong about this, but I think the reason they aren't heavily used is because there's this sort of lag time between collecting the patient's cancer cells, growing them into an organoid, running the drug over them and screening them, And using that as treatment benefit stratification for the patient, I think that lag time is on the order of weeks to months, and that's kind of too long. And I think if you could compress that to a single yes or no answer, then I think that is maybe not trillions of dollars, but it is quite useful.

Well, I guess trillions of dollars. Everyone's worth trillion dollars these days. So maybe it is trillions of dollars. Yeah, it sounds like yes. It seems like your sense is like, yes, that is worth trillions of dollars.

I think if you take the maximal view, there's this really interesting line of work from the Mark Davis lab out of Stanford, which basically said that they can model some interesting facets of immune elicitation via tonsillar organoids.

Tonsillar?

Tonsillar organoids.

Yeah.

And so it's a little bit like a torture definition of organoids because they They're neither iPSC nor tissue stem cell derived organoids. They're more so biopsies of tonsils.

So they take the biopsy and they expand them?

Yeah. And those contain germinal B centres, so they are able to model some facet of immune modulation. And if your question encompassed that as well, then I think that does become a trillion dollar company.

Yeah, my question does encompass that. So it sounds like it could be a trillion dollar company. It's an interesting insight. I guess I don't know that much about this. My sense for organoids has always been like there is so much batch-to-batch variations.

Yes.

And not even just that, it's like within a lab. Actually, I was just talking to a graduate student at Sloan-Kettering yesterday, or I should say on Friday. She is an NK cell person and she's working with a new organoid model that she's never worked with before. And she has to learn how to layer the cells in the right way. And even though it's like, yeah, it looks right, her results don't necessarily match what a postdoc who's doing the same exact thing does, which is troubling.

It is bizarre that your primary functional readout for organoids often is either these very coarse set of protein markers, Or it's just like they look wrong.

Yeah.

And I think that's a bad thing.

Yeah, I guess what I'm trying to get at, where it seems like an interesting problem, is could you actually turn the model's attention to resolving particular steps in the process from going from like, all right, man, we want to make a model that's able to tell us if a drug is going to work or not. in a human before we do it?

Sure.

What do the steps look like? Well, you got to have a good computational screening, some sense, maybe you have to have a good receptor knowledge. Okay, so maybe we've actually solved this problem to some extent with all these different PLMs and whatever. Okay, great, we solved that problem. All right, the next thing you need to know is, can we predict mouse behaviour? Great. Can we predict mouse translation to an organoid, or can we skip the mouse and go straight to an organoid? And so on. It feels like you can actually split up the burden of superintelligence into these discrete tasks and then attack it that way, as opposed to we'll just dump a tonne of biological data into the largest model we can and hope that it can just figure it out. I guess maybe the high-level question is whether you feel like the lessons of whether the bitter lesson truly does apply to biology in the same way. To me, it feels like it could. The ultimate thing that I'm not as quite sure about is with transformers, it was just like, we need more text data. With biology, it's like, well, what actually is the data that we need? What's actually the important data? And I don't know that anyone has necessarily found that.

Yeah, I think biology— I've been trying to work on this essay for months now titled Trusting the Biological Exponential, where I try to ponder what is the hill we actually need to climb with regards to both biological datasets, but also the instruments that are necessary. And I've been struggling to finish it because it's hard to come up with a right answer. I do think the sort of breaking things down into discrete problems and then throwing intelligence at that, solving that problem, and then moving up the hierarchy, is a good way of approaching it, like the decomposition. And I think going back to the organoid problem, maybe you are able to perfectly predict, like, you have a model that can perfectly do in silico organoid behavior. Then the next step becomes how well does this perfect replicability and perfect ability to replicate organoids transfer to human behavior? And now you need to set up an experiment where you have organoids derived from humans in one branch and the humans themselves in another branch, and you have to compare those 2. So you need to have some notion of gold standards for every proxy dataset that you are creating. Right now, people seem to be exclusively either scaling on super coarse readouts from humans or they're scaling on super deep measurements from proxy systems, and there's never any connection in between them. So one example of this is PDB. Very deep biochemical knowledge is available there, but it has no knowledge of isoforms that commonly exist in humans, no understanding of the often massive multimers that exist in humans either. Someday that dataset will be created, but today it's not there.

Right. Yeah, I guess everybody's trying to figure out what their dataset is, the dataset that they think is the one that's missing. And maybe there's not one right answer. And I suppose maybe the Anthropic Life Sciences team thinks it's one thing, maybe Noetic thinks it's a different thing, at least for their problem set. Maybe DeepMind thinks it's a different thing. Maybe they actually all think it's the same thing. I don't know. But ultimately, I think your point is one that I agree about or agree on, which is you can just separate discovery into discrete steps, even though we don't like to think about yogurt spoiling leads to fixing metabolic dysfunction in a baby born in Pennsylvania. Maybe actually you can just break that down into discrete steps. And if you think about it hard enough and you had enough people to think about it with, like, say, in a data centre with 10 million similar instances of yourself, maybe you could actually just get there. I do want to shift gears just a bit to talk about my favourite topic, biosecurity.

Okay.

Because you can't talk about AI without talking about biosecurity. I'm going to ask you a very open-ended question. I will just say, I'm going to say this first. My personal stance on the biosecurity question is, is really just that for some reason we have, we have really lost the plot a bit, and we are focusing on the wrong thing with biosecurity. And maybe definitionally, to me, biosecurity should not be like, can we prevent bad actors from doing bad things to us? Maybe biosecurity actually, to me, the definition should be something like, Can we as humans have agency to prevent biology from harming us in whatever capacity that is? And so my first question for you is, do you feel like that definition is a fair one?

I guess I struggle to see the difference between the 2 definitions.

Well, one of them is about humans hurting humans. The other one's about humans maybe getting hurt by humans, but actually could be about humans getting hurt by natural evolution of viruses.

Yeah.

It could be about insects harming other insects, insects harming cattle, like with screw worms. It could be about fungi ravaging tree populations in the Appalachian Trail. It could be about all sorts of things that have nothing to do with AI.

Yeah.

The trouble for me is AI just takes up all this oxygen, When it comes to these things?

I agree more with the second definition, but I guess my stance towards biosecurity is like, I am aligned to your point of probably good to focus on almost the pathogen agnostic tools. I think far-UVC, concretely a good thing, if not for someone releasing mega Ebola, just to reduce airborne infections in general. Good thing. I do think the people who are working on these sort of being worried about mirror bacteria, being worried about the mega evolved viruses that maybe Evo 3 or Evo 4 will bring about, I don't agree with their worldview, but it's the sort of thing where I'm glad someone is thinking about it.

Yeah, I get your point. I'm glad someone's thinking about it. You and I had dinner, we talked about something that actually I feel very passionate about. I worked on gene drives, right? I worked on gene drives. I worked on this idea that we as humans can have agency over biology to eliminate something that's very harmful to us called malaria. And unfortunately, that work hasn't necessarily translated into impact, not because the science doesn't work, but I think because there's a lot of regulatory— there are obstacles over it sort of preventing that. I wonder if it is fair to say that the screw worm problem gets not enough attention relative to, say, like, all the red teaming projects that, you know, get— probably tally up, like, over hundreds of millions of dollars towards how many college undergrads can we put in a room to see if they can make a virus. And actually, there was something I learned specifically from your article. I read it. I read it with, with, with the hope of being convinced that I'm wrong. Um, or I should say, like, I should reframe. And there's, you know, there's a study you cited in there, uh, where they're like 150 kid college, you know, college undergrads, I think, but it's like novice biologists who are split up into LLM versus internet. and their task was make a virus. And the citation was like 5% success rate for LLMs, 6% with internet. And I'm not saying that obviously LLMs do nothing. It's just that seemingly doesn't make any difference whether it's the internet or the LLM.

Sure.

And then the point that people bring up about Ohm-Shimrikyo and like, oh, they produced the wrong thing. It seems like the thing that actually stops all this from working is actually like, physical reality. You have to move stuff with pipettes, you got to order the right stuff, you got to have it come to your lab. There are reagents you can't order, you simply cannot order them. We have screening, which is good. I'm glad someone's doing viral or nucleic acid screening. To me, it feels like this is actually a solved problem. We don't need to think about it too much anymore. Let's move on. Let's actually put money into stuff that matters. And I brought up the screwworm, Because something I mentioned to you, so there's like only one lab like in the whole country. We have a country of 300 million people, 2% of them have a PhD in bio or a PhD in something. Uh, so, you know, we have millions of people with PhDs, hundreds of thousands of them work in biology, um, and yet we have like one lab of like a dozen people that work on screwworms.

Yeah.

And they got like a huge grant of $500K to work on solving this screw worm problem. And then I see all these startups who kind of are getting dozens of millions of dollars each to work on AI red teaming for biosecurity. It feels like an asymmetry.

I agree. I agree. There's an asymmetry. I do think there is a sort of bias towards— I think there's 2 things. I think people like sci-fi things a lot. And I think it does influence a lot more of their decision-making than they want to believe it does. Yeah. I do think there's also this bias of, like, maybe this is connected to the sci-fi section, or sorry, people like sci-fi stuff. They also think the sci-fi stuff is more likely to be existentially risky than the stuff that's actually real today. I think if you really wanted to save a lot of humans, the most legible way to do that today would probably be to improve the number of vaccines that are delivered to people in lower middle income countries.

Right.

I think the biosecurity people would look at that and say, yes, but what if the megaviruses come about and they wipe out all of humanity? So someone ought to be working on that sort of stuff. Again, I don't agree with it, but at the same time, I also do view a lot of these things as dual use, where if someone is helping create something, I posited in the article that here's how AI might help with the defence problem in biosecurity. And a lot of what I talked about feels pretty applicable to a lot of therapeutic areas of, like, we probably should be learning how to FC engineer antibodies such that they last months in your bloodstream so we don't need to discover a new immunogen for every single crazy virus that comes about, which, yeah, maybe the crazy viruses are designed by North Korea, but maybe the crazy viruses are also just like tuberculosis.

Right.

And we probably want to invest in really good biomanufacturing, have a really good biomanufacturing setup in the United States so we can defend ourselves against bioterrorism. But we also want to have that probably so we can distribute these to other countries for further, more banal viruses. I do think the pure red teaming of how do you ensure the model doesn't teach people how to make bioweapons, I think I could go back and forth as to how useful that is. I do think the therapeutic biodefence companies are fairly dual use. So they are sucking up, I guess, the intellectual oxygen But the actual fruits of their labor, I think, apply in many different areas. I do take your point about agricultural biothreats are very underfocused on, and I talked about it in the article, and I would definitely put screwworm underneath that as well. I don't know why this doesn't get that much attention other than it's like wrapping it up to the sci-fi thing. It's a boring topic to think about.

Yeah, I take your point. Yeah. Maybe to be most clear, I am glad that there are people working on things. I'm glad that Stripe has their Intercept Fund, and they're going to try to work on many dual-use things that are about trying to reduce the global burden of viruses. This is good. I'm glad people are doing this. I am glad that there are some people like Red Queen, who are working on finding the boundary of what should and shouldn't be possible with AI. The challenge for me is really just towards that last bit that, and maybe it is just about sci-fi, maybe we actually just need to hire Hannu to only write books about fungi that kill all the trees and therefore we don't have any national forests anymore. And then people will care about it. But it is troubling to me that so much of the conversation— and I know that conversations happen online that are not necessarily every conversation that happens offline, right? It's not always the same people. But as we know, a lot of people who are about to become extremely wealthy and deploy a ton of capital are online.

Mm-hmm.

And they do really— they read all this stuff that I'm sort of referring to. And they take that in. I've certainly had many conversations where I'm like, wow, I totally cannot even fathom why you would believe that, other than, like, you read it on a sci-fi website.

Sure.

But people believe these things very deeply, and in such a way that it's dogmatic to— you can't convince them that you're wrong the same way that maybe you can't convince me that I'm wrong. At the same time, I do see there is some data, there's actual studies. So, like, Om Shum-Rikyo is an anecdote, but we now actually have studies where we've tried to compare the impact of LLMs on acceleration of bioterrorism, so to speak. Turns out it doesn't really accelerate. And maybe you can always say, like, oh, Well, that was GPT-5.1. Really, we need to use GPT-5.6. And then if it ends up being head-to-head the same, it'll be like, oh, we actually need GPT-6.5. And this is sort of the obnoxious component of this argument. But it feels like at some point you have to say, actually, the thing stopping all of this is that, one, people fuck up in the lab constantly. Even really highly trained people fuck up in the lab. 2, ultimately, state actors are going to be able to behave however they want to behave. Um, and you can't stop them. Uh, big countries, small countries, regardless, like, of their size and their wealth, they can do whatever they want.

True.

Uh, you know, Al-Qaeda was not necessarily a state actor, but they were state-level in terms of their financing. They could afford as much bioterrorism as they want to wreak, uh, if they want to do that, and you can't stop them, right? Um, so, like, what are the things you can stop?

I think The dual use—

like, finding those dual-use problems are the thing, right?

Yeah.

If you can, like, make a pill you can take, and it makes you immune to, like, not just this season's flu, but it actually makes you immune to, like, all the viruses from this year, or, like, it jacks up your immune system in such a way that it makes you extremely resilient to viruses, or something like this. Or maybe you can undergo, like, a genetic surgery, like, the most sci-fi world you undergo genetic surgery to remove all these viral elements from your genome. Maybe these things are really good. And I agree with you. I'm glad people work on stuff like this. Really, maybe the point I'm just trying to make, it's very selfishly on this podcast with a lot of people who are very well off who listen to this, is Is that, I think, really trying to explore deployment of capital to things that have nothing to do with AI but are focused on biosecurity is maybe where the attention should go and say, we've actually put enough money into AI and biosecurity, we can focus on other stuff.

I do think the frontier labs themselves have a pretty strong incentive to ensure their models aren't spewing out instructions on how to do biosecurity. Sorry, how to do bioterrorism.

Right.

So I would almost trust them to take care of that. And most of this third wave of philanthropy should go to places where it's kind of like cut the difference of it's useful for public health, is also useful for the edge case of some terrorist group decides they want to work on a virus. I do think I posted this meme on Twitter once about how I think biosecurity discourse has this repeating midweight curve of there are ways to think about biosecurity that are very dumb, that are paranoid, and there are ways to think about biosecurity that are very dumb that are also not paranoid at all. And I think the most correct way to think about it is broad security measures are always a good thing and broad antivirals, good. Someone probably ought to be working on phage therapy just in case.

You and I maybe disagree on this one a little bit.

We probably should be fitting buildings with far-UVC and also the boring parts of Right now, there's not really building regulations for what defines clean air. There are HVAC standards, but they're not super biosecurity pilled from even a public health standpoint. And it's probably good to develop those despite everything else. I think one thing I do want to say about the active site study is that people are split on what that study actually means.

What are they split on?

They're split on Similar to how we were talking earlier about are these AIs useful for developing new instrumentation technology? They probably wouldn't be useful for me who is using it, who's trying to think how can we make better mass spec? Because I barely know how mass spec works in the first place.

Wait, hold on. I actually disagree with this, but yeah, okay, I want to come back to this.

Keep going. I think AI for interesting scientific discovery, is usually best ran by people who are experts in that particular domain because they best know when the AI is leading them in a wrong direction or has bad taste in ideas. And I think a better active site study would've actually used experts for a harder wet lab task.

That, I mean, I think is fair, right? But isn't that kind of like the point where it's like, does an expert even really I would never do this, but if, let's just say there's an Aaron A Prime, he exists on Earth Prime, it's not me, not Earth, and he wants to make smallpox prime.

Sure.

I guarantee you I can make smallpox prime, and I wouldn't need to use ChatGPT to do it.

I guess like—

It might help me do it in a couple weeks instead of a month, but I guarantee you I don't need I don't need ChatGPT to do it.

What if you— people are going to get upset about the— Maybe we don't. Okay.

All right.

But I think it feels very likely to me that there are people who are kind of on the— so there are a limited number of RNA primes on prime world. No, I think there are a lot. Like I said, I'm on the left tail, but I think Most of the RNA primes are going to be working in well-to-do jobs where they have little reason to hate society, and they'll keep this knowledge in their head where kind of it belongs. I think there are going to be plenty of people in the world who have some reason to hate this particular other section of society, and they're kind of on the edge of knowing all the steps they need to do something that is very harmful for the world. I also think there is a sort of a— you can imagine scenarios in which you want to improve particular facets of the— actually, this is not a brain worm because this is documented in the essay.

Okay.

They failed with their initial virus release attempts because whatever encapsulation they used for their virus just prevented it from—

Yeah, they used the vaccine strain, I think.

No, that's one thing. But another one was they had the correct strain, they released it, but they released it in broad daylight. And so UV immediately neutralized it. At least that's what they think. Who knows? This stuff is a little bit unknowable. And maybe if there was an LLM helping them throughout that process, they would have very reasonably said, should probably release this at like 7:00 PM or 8:00 PM instead of at that time.

Do you think so? To be honest with you, I feel like the challenge with LLMs, at least with general use, and the reason why being an expert or at least knowledgeable in the domain you're prompting about is like, you kind of know roughly what— you might not know exactly the fact, but you might be like, doesn't UV kind of affect DNA stability? I don't know. And you might prompt it. Maybe you wouldn't ask it what time, right? And again, this is a little bit of an info hazard, but it is documented. But there's an element of the expert kind of knows enough. Sort of like, I'm not an expert software engineer, I used to do pair coding with Steven Molina, a friend, and it's like there are things that he would say to me where I'm like, oh yeah, okay, sounds good, because it just didn't occur to me.

But that's why I think an ideal active site study would focus on experts, because experts do kind of—

But do they need to?

I guess I view it as experts have a lot of things they need to be concerned with.

Right.

And maybe the LLM just helps as a thought partner.

I guess so. So you're saying the really good study, A really good study to assess the capability of a model is not a novice, but an expert, because that differentiates— call it like it differentiates the high-level athlete from the elite athlete, right? It differentiates the bubble roster player in the NBA from LeBron James. That's kind of what the study is. Can you actually just— is ChatGPT a steroid from already really good scientists?

I think of it as that. And at the same time, Aum was kind of in this unique position where they had a little bit infinite shots on goal.

They had a lot of members.

And the Japanese government didn't care about what they did.

Right.

But you can imagine the thing I said about the essay that I'm much more concerned about terrorist groups who have a limited number of shots on goal. They know every time they poke their head up and they try to do something crazy, there's a chance it'll get caught by someone. And so they really need to make sure each individual shot is as good as possible. And I think the LLMs are helpful for that.

Did you read the article about the terrorist group in, I think it was Burkina Faso, the ChatGPT, or maybe it was Mali?

Oh, vaguely. What did they use it for?

They were using ChatGPT basically to see, like, oh, can we do terrorism better? And I won't go into the details, just read the article, right? But there was a specific scenario where they described like, oh, we asked ChatGPT how to improve our ability to do this. We'll call it performance stunt because it really is like an Evel Knievel type stunt. And if you read the article, you'll understand what I'm referring to. And then I saw this screenshot, I was like, oh my God, it's like you're enabling terrorism. And then I was like, oh, did this increase your ability to do the thing? And it was like, oh, no. It didn't work out.

It was just fun to talk about.

It was just fun to talk about doing Evel Knievel stuff with, you know, with your boys and a bunch of explosives. And it didn't work out. I'm not saying— I understand what you're saying, right?

Right.

It is a very reasonable— and somebody should think about these things. Yes, it's possible that I'm actually wrong, and maybe we need to focus more on screening. My knee-jerk reaction on screening stuff is like it just introduces— screening specifically, meaning nucleic acid screening. My knee-jerk is that whenever you introduce regulation, and if the regulation is not doing a lot, it's probably not a good idea.

Yeah.

Which is why, generally speaking, I'm kind of against regulation for DNA synthesis beyond, like, the companies already doing what they're doing right now.

Yeah.

Like, why introduce regulation? They're already doing it. And the ones who aren't doing it are in countries that are not our own country, so we can't stop them from not doing it. Right. Maybe you can say we can only do business with countries that do it or whatever. I don't know. So it's possible I'm wrong in this case, which in that instance, I would say we've addressed some bottleneck, which is synthesis. And maybe the next bottleneck is the fact that you can order a benchtop synthesizer. Maybe we should make licenses for this. I don't know, whatever. I disagree with that. But maybe this is actually a reasonable middle ground. What else do you need to do?

Okay. I also don't think it's like the most onerous thing in the world to have some degree of know your customer.

I mean, I think that's fine, but this does exist, right? Go try to buy right now. Go try. I would like you to buy HGH. Go buy HGH. Go start Owl Posting LLC. And spin it up and go buy HGH.

But I guess you can buy that off the deep Silk Road.

You can do it illegally. Sure, you can do it illegally. So there are illegal markets for everything.

No, I get that. But if I try to go buy a pound of uranium or enriched uranium, probably a bit harder to do, right? Because we have really strict standards as to if any uranium is ever lost, there's a worldwide manhunt for it.

So you're saying we should have a worldwide manhunt for my primer that amplifies attack polymerase?

I think for at least—

So I can— never mind, go ahead.

I do think the whole thing where you stitch together pieces of plasmid fragments to create horsepox virus, that did make me downweight the value of having any sort of regulation over benchtop synthesizers. But I don't know, I did get a little bit pilled on the security, like the Swiss cheese security model of you make it hard for people to do the dumbest version of their plan, and maybe that just prevents a lot of tragedy. And so I think we aren't there today, but I think the second enzymatic DNA synthesis really hits prime time, I would support a kind of be careful where these things end up.

So you're saying right now, I guess just for context, what you're saying is right now you have to go through these companies like Twist or Agilent to order your stuff. But maybe if people succeed with enzymatic synthesis where it's more like a protocol you can run in your own lab, then we should prevent people from doing that.

I think there's like—

Or we should find some way of controlling who gets to do it.

I think there's a very strong know your customer setup. Or there's some sort of hardware lock on it that has a very good— it needs to be connected to the internet at all times. It will screen whatever thing you ask to make via some dataset of both public and private datasets that have been curated by, say, SecureDNA, by whatever you're trying to create. And maybe that's not perfect, but it's also not that onerous for the people who are actually trying to do their synthesis. that's maybe completely legal. And it places all the onus on the people who are actually selling the benchtop synthesizer as, like, you have to have these things in place to ensure that your machine is not going to be used for evil in the most dumb possible way. I think it's fair to look at that and say, you've gotten rid of the worst possible cases. What about the more clever people? I think it's fair to throw our hands up at that point and say, we'll find other ways to deal with those people.

I guess, yeah. So towards the second group, I think, yeah, maybe it should come closer to your argument where we should do more to stop the dumbest person on the planet from doing something bad. Maybe we can do more for that. I don't know. It seems like screening kind of does that right now, and we didn't need regulation, or we don't need regulation to do that. Maybe with synthesizers this changes, in which case we need some regulation. That seems reasonable to me. The second group, and I've had this discussion with very many smart people smarter than me, but I feel like I can't quite get it. You don't think that the reason why there's so much of a fascination with Ted Kaczynski is because it's so absolutely unique to have somebody be both extremely intelligent and also evil? And like basically on a mission to like end society's, you know, stability and whatever. There are really not that many people like this.

I mean, I like— I—

and the ones who do will just like— you can't stop— you cannot stop people from coming up with these plots that like involve understanding social dynamics. It's just like, it is impossible.

I, I do think like As to the topic, as to why agroterrorism or biothreats to agriculture are less focused on, I do genuinely think there is sort of like an optics problem of like, it's a lot scarier to proclaim that someone's going to design a virus to make you bleed out of your eyes rather than someone's going to design a virus to depopulate wheat when you live in a first world country and think to yourself, like, America will find wheat somewhere else.

We'll just get it from another country, which, by the way, has a natural virus that makes you bleed out of your eyes.

I do think a lot of these things come down to optics, and I do think there isn't a strong rational reason to trust a lot of biosecurity. This is actually one of my favourite tweets. A lot of biosecurity sentiment has a sort of like Pascal's wager vibe to it of like, the downside is so infinite that you have to trust me regardless of how outlandish the thing I say is.

Mm-hmm.

I also don't think they're entirely wrong. I think there is a kernel of truth there.

Yeah, I guess so. I mean, the thing that's— and this is not rational of me. This is irrational of me. But I will say it is curious to me that we apply this constraint to biology when you can actually do so much more damage currently without any sci-fi scenarios, with really good coding models.

Sure. But I think we're plenty paranoid about cybersecurity.

Right. But we kind of let whoever wants to use Claude code use Claude code.

Sure. Yeah, I agree that it's weird.

It feels like an asymmetry. This is all about asymmetry. I'm not like, well, you let him do it, so why can't I do it? That's not really it.

I do think really part of this, and this is also something I don't understand very well, is Clearly there's probably a really huge economic downside to not allowing coders to just use cloud code.

You don't think there's an economic downside?

This is the thing I'm a little bit like, I do hear biologists talk about a lot, like biosecure shouldn't exist, XYZ biosecurity regulation shouldn't exist because it slows down my work. I've never worked in that particular type of wet lab. Does it actually slow down their work?

I will say, like, maybe it's hard because all these, like, biosecurity legislative pushes are newer. But there are, call them, like, health data and other types of legislation that are in the same vein. Like, it's all about protecting people from the frontier of, like, biology and medicine, or at least the downsides of those things. Those things legitimately do slow you down, not because in the moment they slow you down, but it's like in 30 years when it no longer is like a Like, I think, like, basically the argument that these people are making, and, like, I certainly would probably be one of these people as well, is that it's not necessarily that making it harder to order DNA today stops me from doing anything. Because ultimately, like, you know, at least I haven't, or I haven't ordered DNA in, like, a couple, you know, 18 months or so. But when I was ordering DNA, it was like, I'm making, you know, plasmid or protein binders. None of those protein binders are going to be things that I expect are going to be troublesome. And if it gets flagged because it's close to something that I missed, it's like, I shouldn't be working with that anyway. And that's totally fine. I think it's more like, what does it look like in, say, 10 years when the specifics of the law that you wrote now start touching on and some new technological development. And now you're doing funky things like, well, we can't collect data from this state because of the way that their law is written. Actually, a great example is like the state of New York, which we're in New York City right now, has quite backwards laws when it comes to patient biopsy samples. And a lot of trials just go like, yeah, we're just going to exclude New York, even though we have this massive city with tons of patients and tons of money and Ochsner and Kettering and NYU and New York Presbyterian. It's just a— it's a hassle.

Yeah.

We're just not going to do New York. We're out. That's really the concern I think most people have. Yes, there is a component of like the libertarianism that I think a lot of people are drawn to, which is like, I want the government out of my industry because they don't know what they're talking about. Um, that is maybe like the hard— the most hardened stance in this particular case. But I think a softer stance is like, fine, make the law, but make it expire in 5 years. And if you think it did anything, you know, renew it.

Yeah.

But don't have it last forever. And this is a general principle of regulation, right? It's not like a biosecurity-specific thing. And then the second aspect I'll say, and again, like, every, every person who holds these stances probably can speak to a different thing they care about. But, um, it's like, well, who's gonna actually dictate who is an acceptable person to order DNA? Is it the academic? Is it the industry person who has a PhD from an academic institution that's approved? Is it, uh, an industry PhD who's allowed to order these things? Is it gonna be an MD? Is it going to be someone who got their PhD in a foreign country but moved to the US? Is it going to be a permanent resident? These are all things that— these are details that matter, and these things actually end up decelerating biology or decelerating technological progress. And I think a good example of this is every AI person freaked out when the US government said, we're going to limit access to like, fable, I think.

Yeah.

Because Anthropic has a ton of employees who are not American citizens. And I think justifiably they're upset because this very specific instance of federal government legislation applied to their industry. That is why biology— biologists who actually do biology, who actually manipulate DNA, who actually do things and make drugs that help people live longer, get a little bit upset and paranoid because, yeah, I mean, it sucks when somebody comes into your industry and makes it harder for you to do things even in a theoretical way.

Yeah, I do think, I guess, like, there is the— today I think the biologists are absolutely correct. I do think if you have your AGI-pilled hat on, You can imagine a future where maybe lab automation really kicks off and the number of biologists necessary to have in the world to make biological progress move is actually far lower than it is today. And you can actually have these few trusted individuals who are shepherding in the future of biology. I don't think that's feasible in 5 to 10 year time horizon. But I imagine that's what's going through the heads of people who are very biosecurity pilled.

And it is worth saying that I absolutely understand the reason why people care about these things is because they are very worried about, as you said, the unlimited downside scenario. And I can understand if you can have any capability to model someone else's worldview, I can understand why somebody who has worked in AI, who saw scaling solve their problem that they care about, even RL with self-driving cars, whatever. I can understand how you'd be like, well, this is going to happen to every industry now. Obviously, all of the worst things now become possible. I can get that. I guess I don't understand why these people don't also understand the flip side, which is like, they're upset when their industry gets regulated. Why can't they understand that other people might be upset? Not even just biologists, right? Chemists, environmentalists, petroleum engineers, whatever. Everybody in their own field gets upset when regulation comes in.

I guess a big part of my worldview is that it is probably good the United States tried to regulate the models.

Yeah.

Purely because I don't think Fable or Mythos is necessarily destabilizing to society. But I think at some point the models will be. And so it's probably good for us to get some appetite for eventually the government's going to poke their head in and ask what's going on here.

Which is totally cool. I guess the difference is the frontier labs don't— well, the frontier labs were not really limiting who got access necessarily, and then eventually they did. The good news is Twist is perfectly happy to screen the DNA, and they didn't need the government to tell them not to do it.

But there are— and this is actually why I think there are pathologies to biosecure, but I think, sure, Twist, US-based company, they're happy to do it. Is the DNA supplier in China, are they willing to also do the—

Some of them do, but they also don't really believe in this as much. It's worth considering why they don't. And I think this is something you could probably talk about for hours and hours and hours, but it's worth considering why they don't.

I guess if I were to put the AGI pill hat back on and be in the position of someone who's actually reading the transcripts that people are putting into Claude or ChatGPT, I have no doubt in my head that some of those chats look really scary. And they do push you towards this maximalist worldview of we need to shut things down this instant. And I think in an ideal world, there is a biologist sitting alongside these people saying some of the stuff isn't as scary as it looks. But I can understand their reactions and I sympathise with them a lot. I think a lot, like when I wrote my biosecurity article, there were at least a few people who message me saying, you're not paranoid enough. The world's going to explode in a few years. And I don't know, I get it. I get where they're coming from. I hope they continue screaming as loudly as they can. I also hope the other side continues screaming as loudly as they can, and hopefully we will end up, I guess, it's a boring position to be like, everyone has a point. But I genuinely think everyone has a point.

I do think everyone has a point. Again, if you consider their mental model of things. Maybe the question is, is the thing we should focus on, like, stopping the low-hanging fruit, sufficient to also be stopping, like, people who are actually doing important work? Is that really the best approach? And I don't know. I don't know. And maybe it is, and maybe they have Maybe they do have access to all these prompts from people that I don't interact with who are asking ChatGPT to order them from bomb.com, order me a bomb. I have no idea. Maybe people are doing that. But in the absence of that evidence apparent to me, maybe my hope is that people can understand or at least build in an element of, call it, deprecation to some of these laws. And that's the thing that I think everybody wants about regulation in general. Let's actually do it with biology. You know what? We're all smart people. Most of them are, a lot of them are smarter than me. I think that they can understand the merits of that at least. So I don't know, there is a middle ground component of things.

I think a really interesting blog post for someone to write, and I hope someone does this, someone who's listening, is let's say biology did get heavily regulated in whatever way the biosecurity people really want. What actually changes to the bio industry? I have no idea. You clearly have some idea, But I would love to see a quantitative breakdown of here is what research immediately becomes almost impossible to do. And what is the economic impact of this?

Can I present the future that I expect to happen if you go with the maximal regulatory?

Sure.

It's the one where China actually just destroys American biology. That's the future, actually. That's actually the future because they don't care at all about this stuff. And you could argue that they should care a little bit. But they don't care.

No.

They don't even think about this stuff whatsoever. And maybe today it doesn't matter because their models are worse than ours. But you read the same stuff I read. Their models are getting quite good at some point, and they have a lot of highly technical, exceptional scientists. Maybe they start demolishing American biotech such that it doesn't matter that you solved AGI.

And this is why I would like to see that blog post. I could totally imagine that your version of the future where we pass all these regulations, it cripples our biotech industry even more than it's already crippled, and China takes the lead.

To be clear, it's not just regulations, right? It's like even the idea that we should screen bioprompts and not every other type of prompt.

Sure.

Yeah.

To some extent, I feel like China's a little bit already en route to the future that you mentioned.

We can hit the exponential if you want to.

Yeah, I do take your point, though, and I don't know, I hope the smart people involved in this discussion find some Pareto optimal place. I do wonder if there will be some flip in China at some point where they do start to care a little bit, because China wasn't really a big force in the world during the big anthrax attacks. But maybe there will be some big biosecurity event that happens in the next 5 to 10 years, and China does also start to get a little bit spooked about it. It feels unlikely to me that the paranoid people are so clearly off base and that there will never be some major biosecurity threat in the future. I think it probably won't take the shape they expect it to, but I think it will happen someday. And China will also be a similar degree of concerned, perhaps not to the same extent the United States is, but I think they'll lose something. It just feels impossible. One example of this is—

How about COVID-19?

That's a good point.

Right.

Do they do anything?

Well, they shut down the country. Their countries, their cities were shut down. But they also had a— maybe this is probably— I wasn't in China, right? I was in America. I was in Boston, which was completely dysfunctional. It was a mess. It was a nightmare. My understanding of things is that in China, they shut down the cities and everyone had to stay in their home and food would come get delivered to them. The army would deliver food and stuff. They at least took biosecurity seriously if the application of it is basic public health preventative measures.

So once the problem arose, then they implemented it.

And went into a sort of reactive mode immediately. And maybe in America we have to care about this more because we have poor institutions. Maybe that's what I'm learning right now. And maybe this is one of these like East versus West things where it's like, well, in the West we just think really hard about it and therefore it doesn't happen. And in China it's like, well, if it does happen, we know we can handle it real quick.

Yeah.

I don't know. This might be a— maybe actually this is it. Maybe this is it. And maybe, maybe this is like a philosophical thing. that is embedded in my mind, maybe in my genome, from my parents being maybe from the Eastern Bloc of the world.

Because I don't know the full details about this, but I do know China has made their investigator-initiated trial things a little bit more stringent over the past year. I'm not sure if I'm prompting any article. I do know it's not—

I mean, there was just the AAV9 trial where the kid died because the PI was being negligent.

And so a lot of regulations are historically written in blood. And I know at least in aviation, this is true in the United States. I think we're jumping the gun for biology. I wonder if China will learn from first principles, which regulations are worth taking seriously and which ones aren't. Whereas we're going to speedrun everything all at once and then we'll slowly dismantle the things that seem to be too onerous.

Yeah, I don't know, maybe one way of putting it is in America we trust Anthropic and OpenAI to surveil us, our behaviour. And in China, the government is like, we have the optimal surveillance machine, so we know how to stop people from doing bad stuff because we're always watching them. Maybe, maybe that, maybe that actually, maybe this is just it. Maybe that's like, maybe that's the answer. And maybe, you know, maybe we do just need to say, all right, I'll just keep using Claude Opus 4.1 while Iranian, like, hackers use, like, Fable 5 to destroy the American water system. And maybe that's the way— well, we generate a lot of money from writing software, so it's okay.

I don't like— I mean, I think, I think a lot of this, like, wraps around to Maybe regulation isn't necessary if we had a sufficiently good therapeutic apparatus for dealing with the problems when they arise, which is why—

This is the accelerationist view, right? This is like, AI should maximally accelerate us. And maybe this is the question is more like, is biology like attack or defense? What does it favor, attack or defense?

Well, it clearly favors this. Okay, actually, I think it clearly favours attackers. I think there are some defences as to why defence also doesn't have—

Human immunity is a pretty good defence.

Immunity is a pretty good defence. If you get really— there are such things as broad viral protectants. There are physical barriers. If you deploy glycol vapours and far UVC everywhere, maybe rendering the internal environment continuously hostile to pathogens actually is a really good defense and attackers would have to spend enormous amounts of money to hurt anyone. I think if we got that whole apparatus built out, I think defense actually becomes pretty easy.

Yeah, I mean, I guess I remember having a discussion with an OpenAI employee. I won't name them here, but about what happens if you just completely let down your guardrails and you say AI models are allowed to do whatever. And, like, let's really think of— let's just, like, digest this a bit, because this has— this has now been, like, something in the ether. So I don't think it's too dangerous to discuss. But let's just say there's a virus, historical virus, and it is, um, you know, some percentage of people who get infected with it die, and it's bad. Um, didn't we just learn that you can just quarantine? Maybe in America people don't quarantine because— didn't quarantine during COVID because of, like, all these public health messaging things and, like, the politicization of COVID But maybe if it was, like, Virus X, which I won't name, that is historically known to be super dangerous, people will stay indoors. And now you've stopped superspreading. And now we can wait, genuinely wait, for the spin-up of vaccines. I don't know. This is the other view of it. It's like, can you sufficiently capture— that seems to be what China's bet is. Can we bet on the same thing in America?

I think for COVID, yeah, it wasn't known to be incredibly dangerous, so people were willing to go out and about. I think if it was incredibly dangerous, people wouldn't have gone out and about, and maybe the virus would have burned through people quite fast, so it would have die out pretty quickly. I did have an interesting conversation with someone who I'm not going to name for the biosecurity article, who told me, thank God COVID did not target children, because if it did target children, people simply would not leave their house, one, but two, would also not be working on vaccines. They simply wouldn't leave. They would just stay in their home And they don't have the authoritarian government that is China pressing down on them to go.

This is not true.

Is it?

How many scientists have kids? At Harvard, none of them had kids. At Dino, I don't know how many. We had 150 employees when I was there. Maybe like 10% of them had kids.

But importantly, you're still cutting down on the workforce in some meaningful capacity because it's not just scientists. It's people who are Working at the GMP place. Yeah. And I think if you're cutting your workforce by a substantial fraction, I don't know what that fraction is, that does slow down therapeutic development a fairly high amount.

Yeah, you got me. I think biology is a sort of field of confrontation. This is kind of like the thing ultimately. It's like you have to either be ready to roll around in the mud and maybe accidentally infect yourself with some AAV in the process of making extremely exceptional therapeutic viruses, or you go like, well, we'll just operate at a speed that seems acceptable, which is always slow.

I've never spoken to someone who works in a BSL-3 or 4 lab. I'd like to get their thoughts as to maybe this is chill. They're happy with doing therapeutic development even in dire times when there is a superbug going around, and this wouldn't actually slow them down at all.

I don't know. N95s are pretty good. Wearing gloves is pretty good. Using Purell is pretty good.

It gets into your head, I imagine. I'm not even talking about the physical impacts. I'm talking more about emotional impacts.

Yeah, man.

You can imagine viruses that have much more physical displays of suffering that actually does, again, cut down on the workforce again.

Yeah, maybe it makes you bleed out of your eyes. Yeah. To be clear, when I imagined this podcast, I actually didn't imagine us talking about biosecurity this much. I really was like, maybe make this a platform to talk about screw worms. Screw worms. Like, a scientist whose lab I think is highly underfunded. Max Scott at NC State. For any listeners who have the means, by all means, please do look into his work. I think it's extremely important and extremely underfunded, especially right now. I didn't expect us to go into all this cybersecurity stuff, but I'm glad we did it because I have versions of this conversation with so many people in AI. And I would say And I don't mean this in an insulting way. They know as much about biology as I know about Frontier LLM infrastructure, which is nothing.

Have you spoken much to the life science people at the Frontier Labs?

Yes, I have. And I would say even then, they don't have that many people who actually have put their hands on a pipette before.

Yeah, I'm also reasonably far away from a pipette. My impression is that things are getting better over time. I think when the LLMs first popped up in 2022 to 2024, I think people were freaked out all the time for reasons that I couldn't quite grasp. They didn't have a good theory of change for how this stuff actually ends up impacting humans negatively. I think as time goes on and the frontier labs start to gain a stronger life science team who has actually worked in a lab before—

To be clear, I think they do employ very strong people, and there is a question about whether you need to actually work in a lab to be good at biology. But I think that there are issues that crop up that require you to do them with your hands to even know. But go ahead.

I think their opinions are becoming more grounded in reality as time goes on and they're learning what is a real problem versus what isn't a real problem. I do think there has been this, maybe you don't see it and I'm talking to a very select group of people, but when I talk to people in biosecurity, they do seem pretty— the people who work on pure LLM work, they're freaked out about everything. The people who are working on biosecurity x LLM, they do generally agree with broad defences are better. Let's not get too paranoid about mirror bacteria. Yeah, I think as time goes on, the culture in certain fields becomes more and more sophisticated and matches up with what actual domain experts think.

Yeah, absolutely. And final note here is I'm not trying to strawman the position of frontier labs. I think the final thing is just, I'm glad that people are thinking about these things. I hope that there's more room for the integration of differing opinions because to some extent right now it does seem a little bit of like, well, we were right about scaling.

Yeah.

So we're right about your stuff too. And I think that can be a kind of it's both off-putting and I think also dangerous to thinking in general, in the same way that I've shifted quite a bit towards the position of pro some measures through discussion. So thank you very much, Abhi. We discussed a lot here. Maybe the big takeaways are that we hope that people will work a little bit more on tooling and instrumentation. to generate data. We hope that people do a little bit more when it comes to environmental hazards and funding goes towards that stuff, which is also really important. And also that you should give money to Max Scott's lab at NC State. Do we agree with all 3 of those things?

Yes.

Okay.

And then, okay, so you're leaving New York And so we have something, you know, you're a guest of the show. So, you know, we want to make sure that you remember New York when you're in sunny San Francisco. So here you go. If you want to just open that up.

San Francisco.

Oh, you're going to be disappointed by the bag.

Did you pick this up on the way here?

I picked it up yesterday. I picked it up yesterday.

Wonderful.

I passed by. I passed by. I was like, you know what? He needs one of these.

I feel like I lose hats very frequently, so this is a welcome addition to have. Thank you. Thank you very much.

Thank you. All right. Well, thank you very much, Abhi.

It was a pleasure.