Is The Era Of AI-Designed Drugs Actually Here? - EP 50 Josh Meier and Jack Dent

I always wanted to find one, some way to bring these things back together, right? I loved programming, I loved biochemistry. How are we gonna bring things together? And language modeling seemed to be the solution. It was, I was sitting at OpenAI, you know, seeing these language modeling results. I was like, if a computer can understand how you and I communicate, why can't it understand how cells communicate, right? Proteins, DNA. What if we just feed all of that into a language model? Something as interesting as bound to happen.

We are here today to talk AI and science and health with Josh Meyer and Jack Dent from Chai Discovery. Hello, gentlemen.

Hi. Thanks for having us.

Thank you. I know you had to— dragged you through Bay Area traffic to get here.

Someone's got to solve that problem.

So I'm told people are working on this. We'll get tunnels like Elon promised. Yeah, well, thank you guys so much for coming. I, as I was texting Jack, you know, my inbox these days seems inundated with companies telling me they are breaking new ground in AI, in science, and medicines are being discovered left and right. And it's like gotten to the point where it's impossible for me to even sort of have any bearing on assertaining what is real, what's not, what's actually going on. But I had the luxury of meeting you guys a couple months ago, and then I have tons of people I respect who said, "You guys do, you're legit, and you're doing some groundbreaking work here." And so I just thought this would be a chance to like tell the world sort of where we're at with AI and medicine and discovering all these molecules, just to kind of like get updated on where we're at. So, Why don't you— does one of you want to do the honor of trying to explain just first off to give people a baseline of what you do?

Go for it.

Well, at Chai Discovery, our mission is to make biology look less like science and more like engineering. So we start with that with pretty much everything we do. It comes out to the way we do our work, the way we present our work, and the kinds of challenges that we're going after.

Okay. And, and, and In the baseline, on some level, is that you're applying AI to biology, I guess, also across everything you guys do.

As you might imagine, AI is a really good tool in working on that mission.

Yeah, yeah. And then sort of, I would say you've had two very large announcements over the last, what, 3, 4 months? And they were in particular around using AI to discover things like antibodies, like new, new molecules. Is that fair?

That's right.

Yeah. Okay. And, and, um, okay. And then, so, all right, let's get into the backstory a little bit because you guys have known each other for a long time. I like the journey of how you even got interested in this. Um, Josh, for you, I mean, you've been doing biology work since high school. I think you were in this strange position of having a stem cell lab at your high school, and that's like what first drew you into this.

That's how I got into this space. Like Jack, I grew up as a programmer, so I was coding since I was a kid and I went to this high school because of the math team. But a couple of weeks into freshman year of high school, I learned that we had a stem cell lab in our school and I was like, how am I not spending all my time here? So I got hooked on biology early on.

And it was like a public school.

It was. Yeah, it was a magnet school. So you had to take like a test to get in, but completely free, you know, public school and people from all kinds of demographics. Graphics. It was a really, really interesting place to grow up.

In New Jersey?

New Jersey.

New Jersey.

Yeah. Right.

Okay. And then you, were you 17 when you first started working at Stripe?

I think I first—

Is that legal? Stripe?

It was not legal because I actually couldn't get a visa for the first time I wanted to work at Stripe, but ended up going to Stripe the summer of my freshman year of college, having first applied out of high school and I was always sort of considering college, not college. Do I, uh, 'cause so much was happening in Silicon Valley, but ended up, yeah, working there as an intern at the age of 18 and thinking about dropping out for a good few years. But, uh, finally went back there when I graduated.

Okay. And I'm gonna thread these two stories back to how you guys reunite, but you end up meeting at Harvard, I think, right? Your freshman year?

First day of classes.

First day of class. Okay. Well, you, but you were not roommates, you were just— No. Okay.

There was some, Harvard was trying to create some new courses to make computer science difficult for people. And one of those courses, it was meant to be an accelerated curriculum. And there was a small group of people doing that. And Josh was one of them. And so we met in first day in one of those classes and then spent a lot of late nights in the library together getting through those problem sets after that.

So you kind of bonded over computer science nerdery, is that?

Pretty much. I'm ashamed to admit it, but yeah. Okay.

And then you end up going to Stripe after school.

That's right.

And then you go on this like OpenAI, Facebook sort of journey. And you're at OpenAI in pretty early days. I mean, it's like GPT-1-ish.

GPT-1 came out while I was at OpenAI. Yeah. So I joined them to work on reinforcement learning and then language model became all the buzz while I was there. So we did GPT-1 and GPT-2 and it was really a front row seat to all of the exciting things that have kind of come out since.

All right, what do we do at Core Memory? We cover innovative, fast-moving, forward-thinking companies, which is why Core Memory is sponsored by Brex, because Brex is the intelligent finance platform for many of these companies. 30,000 companies from startups to the world's largest corporations rely on Brex's technology for their finances. They've got smart corporate cards, high-yield business banking and expense automation tools that are fantastic. I hate doing my expenses, and Brex's AI software runs right through those expenses, figures out where we're spending money, and takes care of so much stuff for you so you don't have to waste your time on it yourself. Go to brex.com/corememory to learn more and just, you know, get with the program. Let's get going. Let's get out of this Archaic Finance Software and Move Toward the Future, Core Memory, and Brex. And this, I, I think this is what starts to kind of like bring us back on our journey. I mean, so you, you're looking at language models, you have this comp sci background, but you also have this interest in biology. But oh, actually, I mean, you majored in chemistry.

Yeah. So Jack and I took all these computer science classes, but I formally majored in chemistry. Right.

Okay.

I always wanted to find one, some way to bring these things back together, right? I loved programming. I loved biochemistry. How are we going to bring things together? And language modeling seemed to be the solution. It was— I was sitting at OpenAI, you know, seeing these language modeling results. I was like, if a computer can understand how you and I communicate, why can't it understand how cells communicate, right? Proteins, DNA. What if we just feed all of that into a language model? Something as interesting is bound to happen.

And so you're like bouncing this idea around in your head while you're— were you, were you like an OpenAI employee or were you there for just some limited period of time for a project or—

Yeah, I mean, OpenAI back then was a nonprofit. Yeah. So it was a pretty interesting environment. People were kind of coming in and out. So I was there as an employee, but at the same time, you know, it wasn't this like 5-year commitment kind of thing that people were necessarily even expected to do. AGI was supposed to be a 20-year journey. Maybe we're getting it sooner than that, but it was a very long-term oriented place. So people would come in, they would work on research projects. So I was working on reinforcement learning, tuning of language models, which did not work very well with the GPT-1 backbone. Yeah, it works very well with the GPT-4 backbone. It turns out that's all the RLHF stuff. But OpenAI at that time was really doubling down on that AGI focus. And that's what led me actually to Facebook of all places.

Yeah. So and I was going through my notes and you had this interesting quote. I think it was right. I was reading it very fast last night. But you were— your Facebook also had a research, AI research arm. But you were even saying, I think, and when I went over this, that it was even more open-ended than OpenAI at the time. And OpenAI was meant to be a research organization. But so you start to have this idea, how can we apply this new technology back into the things that I'm interested in? And so what, you like pitch Facebook on this idea or they were already also exploring this?

So this gets back into the, the really early days of FAIR as well. It was Facebook AI Research where there was a team called the Garage Team, which was about how do we take AI and apply it to big challenges. And science was one of them. Okay.

And so they're open to this idea when you're knocking on the door?

Yeah. Like, let's do something with protein folding, essentially. And there were a couple of folks that were thinking about this area. How do we do biology? How do we do games? You know, these kind of these big ideas in AI. And that felt like a really good fit for the kind of research program that I was really interested in building up.

And like when you, when you say you saw what was happening with LLMs? I mean, back then we're talking about vacuuming up all of the internet's words to feed these models. And so how is it like obvious to you that that could be applied to proteins?

And to be honest, it wasn't that obvious to me at the time it started. These days we would never fund a project like that at CHAI because it was almost like, hey, we've got all this biology data, we've got LLMs. Why don't we just throw the thing together and see what happens?

Yeah.

And again, it was a research lab, right? We weren't building a product or anything like that. So it was a bit of a curiosity view. There was a sense that something like this could work. If you go back the past 20 years of bioinformatics literature, the way people would do protein folding is you would take a sequence that you're interested in folding and you would look through these massive sequence databases to try to find other proteins that are similar to one another. And then from that, you would actually fit a really basic machine learning model. You would call it a POTS model. It kind of comes from physics. And it turns out the math that people were using for those physics models are basically the same math we use for language models.

Okay.

So there's some deep connection there to try to exploit.

And like, and AlphaFold from DeepMind is happening at the same time, or?

AlphaFold 1 came out, um, I think, uh, 2 or 3 weeks before I started at Facebook.

Oh, okay. Okay. So it's all, this is like close.

It's all happening at the same time.

Yeah. Yeah. Okay. And you're, and you guys are friends, you're obviously both in the Bay Area and you're, I, I was also going through my notes. I mean, you're kind of like keeping tabs on each other.

Josh, every couple of months we'd get dinner and he would show me, once he was at Facebook, some of the interesting work that was happening. We've just trained this model. Here's what we're seeing. We found this really interesting result. And actually, even when he left OpenAI, I got a Facebook message, because that's of course how people communicated back then, from Sam Altman saying, hey, this guy Josh is leaving OpenAI. I want to do some proteomics spin-out. Should I work with him? And I responded to him saying, yeah, Josh is amazing, but I'm pretty sure that he's pretty set on this Facebook thing. Good luck. And to Josh's credit, you know, it was just way too early to be starting a business at that time. And so Josh knew that I think a good another, you know, half a decade of research needed to be put in before any of this was really commercially viable. And so we were chatting a lot throughout that time and, Yeah, really too. I mean, we can keep going on, on Josh's story because there are a few more turns in the road there. But it was really in only in the past few years where this space made that transition from being something which was really the domain of a social media company's advanced research lab to something which could actually change the way we do science.

And so, and like in this time, I mean, it's like 2019 when you leave. So I'm just trying to set us a little bit.

When I leave, so I joined Facebook January 2019.

January 2019. Okay. And you and Sam, you talked on Facebook. Did people actually communicate on Facebook?

Uh, maybe not everybody. Sam certainly did. Um, yeah. And then, you know, when we, when we started Chai, we, in many ways we just picked up on that thread and said, hey, you know, 5 years ago wasn't the time to do this, but, uh, yeah, now, uh, we're, we're, uh, doing this and we're doing it together.

Okay. Which is— and this is where I want to get to, but like over the course of the time you're at Facebook, I mean, what specifically are you working on and how far do you get?

So the first thing I did when, when, when I showed up at Facebook is we started building the language models again, right? BERT had just come out, AlphaFold had come out, GPT-1 and GPT-2 was soon to be out. So we had to start just by building infrastructure there. We built that really quickly, probably like a month or two to go and re-implement a lot of that. And then we just trained on a big protein sequence database. And within 3 months of being there, we had our first result out. This was the ESM-1 model. It was the first transformer language model of protein sequences. And it was really interesting to see what it could do.

Like, I mean, for people who have no idea, I mean, what are you asking the model to do specifically?

So back then it was really basic things. It was like, do I even know the difference between a positive and a negative amino acid in terms of the positive or negative charge? Can I predict the impact of a mutation on a protein? If I introduce this mutation, does that make the protein more active or less active? Can we do some basic protein folding? This was my new AlphaFold 1 level days, so the protein folding was really bad compared to today's standard, but it was better compared to the methods that we had at the time.

So just like you're asking an LLM to finish a sentence or finish a paragraph, you're sort of setting it up and being like, do you know what happens next or how this protein can be tweaked?

Essentially, yeah, we used a variant called, called BERT. So instead of finishing the next word, you play fill in the blank. So you erase an amino acid, you try to predict what's there. And then how do you use that to predict the impact of a mutation? You just leave one out and then you ask the model like, hey, what would have been the impact if I use this mutation versus this one? And it turns out that's a pretty good proxy of what ends up happening in the lab.

And you're like, you're leading this program.

So it was a, it was a research lab back then. So it was a couple of us kind of coming together from multiple offices even, and just pushing the research forward. There was a pretty informal structure. Around the time I left, Facebook kind of doubled down on that and, uh, and started to put more of a structure in place on these big projects. But back then it was a couple of us sitting around just trying to push the project forward.

And how was Facebook feel— I mean, were they impressed with what you were doing? Could you kind of tell how it was being received?

Yeah, there were, there were some funny elements to this. So, uh, again, Facebook went through a couple of these waves of figuring out like, what is even the right, like, promotion structure, right? How do we incentivize people to do good research here? So they had to come up with a way to identify what were the top papers coming out of of the research lab. And it was funny where they actually put in tier 1, the representative paper of like a tier 1 research result was the protein paper, but literally this ESM work. And to me, this was like so funny reading. Obviously I was, uh, flattered to see that, but also I'm like, this is not what you should be incentivizing people to work on at Facebook. You should get them working on like natural language models or computer vision because that's what you need. Uh, but I think it kind of just came to this point of really thinking big about what AI could do, thinking about the future, you know, 5 to 10 year time horizons. And that really got people, I think, excited in the Research Lab.

Okay, and we're gonna get to try in one more turn, but I mean, you, you take a pit stop before doing your own startup. So you went to— what was it, um, AppSci?

Or I went to AppSci.

Yeah, okay. So a big biotech company, you're like 26-ish, something like that.

Yeah.

And then so you're getting yanked out of Facebook to go— you have this big company that's asking you to figure out AI and, and what it can do.

Yeah, yeah, that's right. So got this call from AbSci, uh, and it was a, it was a really interesting opportunity. So AbSci had built up, uh, a sophisticated lab, uh, to be able to do lots of different experiments with, with antibody sequences, with protein sequences. And the call was, hey, Josh, like you're doing all this work at Facebook. You don't have access to a wet lab. Why don't you come here and kind of push that research program forward? Because the future of drug discovery is going to have AI in the mix.

And how long did you do that?

So I was at Appsy about 3 years.

3 years. Okay. And then this is somewhere along here, you're, you're, you guys are chatting and things are, are lighting up. Is that generally what's going on?

That's right. Yeah. I think the, as there were so many, there's really towards the end of 2023, early 2024, there was so much happening in in AI in general. And I think both of us came to the conclusion that if you look at, there are many opportunities, I think that we both had to build companies along the way. And I think we both honed in on wanting to build something which was higher technical risk, that really in many ways gave, build a new capability that just didn't exist before. So in the early days of Chai, we, and when we were brainstorming, we spoke a lot about do you commercialize GPT-2, that level model, or you try and push to the GPT-3, 4, 5 levels of building something which nobody's ever seen before. And in terms of that style of company building, it had a real appeal to me and both of us in a way in which I think that there's, if you think about the two axes of any company, which are the principal components of risk space, let's say. Yeah. Being market risk and technical risk. I think if you can play in the sort of moderate to high technical risk, but low market risk segment, it's a really interesting place to be because you really get to focus on just building the best systems. It's also an incredibly hard path compared to maybe in just, you know, having a product day one and going and landing commercial wins. But in terms of, you know, what one can spend one's life on, I don't think there's anything more exciting than that.

Not one. I was curious about that for you in particular. I mean, so you were at Stripe for a number of—

how many people were at Stripe when you When I interned, maybe less than 100, 80 maybe. When I joined full-time, maybe 500. And when I left, probably 8,000. Yeah.

So you're, you're on this rocket ship in, in, in helping build this massive financial, you know, juggernaut. And, and so, so you, you're doing that for a few years. I mean, you already got at it a little bit, but I was curious of your motivation for wanna change directions and head into something far afield from finance. Yeah.

I think we'll look back on this decade as the decade when everything just started working and that the, I think the choice for me in terms of what, you know, domain to go into, there were, you know, once I'd honed in on the sort of the nature of the company I wanted to build and as I was especially at Stripe, one of the things that it taught me was that, that, you know, good engineering can take you a really, really long way. And that also there is a distribution of engineering talent as well, and that you can move really fast if you get a group of really amazingly talented people in a room together to build something. And so when that translated into what came next, it was really like, okay, well, let's look at an area where that there isn't clearly a really rigorous engineering bent in this field right now. If you look at, you know, the biotech industry, it really hasn't attracted that many software folks to it. And so, you know, you put that together with, you know, massive opportunity, one of the largest industries in the world, which is on the precipice of profound transformation, and that it's sort of, that it's calling for more people with an engineering background to go into it. And put that together with also the fact that when working on this problem, you, in the best version of it, get to touch so many people's lives and improve those lives. There didn't seem to be many other options in terms of what to work on, frankly.

Yeah. Yeah. I remember, I mean, I guess I did a story, it was probably like 2012. I mean, it was sort of like during the ad tech boom, and this guy Jeff Hammerbacher, who was at Facebook at the time, um, I think he pioneered the term data scientist. There's somebody in his cohort did, if it wasn't him. But, you know, he was kind of bemoaning the amount of talent that was going— I mean, this is like the cynical take on this, but going into ad tech. And, you know, he had this pretty famous quote that I put in the story. It was, he was like, the best minds of my generation are thinking how to make better ads. And, and, you know, it was really just a comment on the vast amount of human capital going into this one area. And then today, I mean, look, you can use AI systems for all sorts of things, but you could make a similar argument that we have so many intelligent people working on chatbots and, and, uh, coding helpers. And, and, you know, there, there is all this excitement in these other fields where this could go. This sort of gets like the heart of really why I wanted you guys to come in. And I should have done a better introduction on this, which is, it's funny because like, I feel like the attitude and the press change every 3 months. So, you know, maybe this won't hold very long, but, but Torkesh did this, that interview with Andrej Karpathy and was sort of like, oh, AGI is 10 years out. And then everyone's like, yeah, felt like we— this goes up and down, that we're in this low moment of, of this stuff is cool, but it doesn't really do anything. And where is the scientific discovery? Where are the new ideas? When will this technology figure out something a human can't? And then when I see what you guys are doing, I'm just like, well, either that's what you're doing, or I'm, I'm, I'm like, like crazy or something, you know? And so this is why I want to talk to you is just to level set and try to figure out if this if this is new. So, okay, so let's go back to, um, when did, when did you guys first announce Chai 2? It was end of June. End of June. So yeah. And try to explain it just in a way that just, just about like everybody can understand. I mean, what did you show with Chai 2?

Yeah. So, uh, maybe it's worth taking a step back and just talking a little bit about medicine as a whole. And so if you look at All the medicines that are approved today, you maybe get between 40 and 60 newly approved medicines per year, give or take. Roughly half of those medicines are things known as antibodies. And antibodies are a slightly larger molecule. They're actually what your own body produces to neutralize various threats which enter your body.

And then, but these are like synthetic antibodies that the pharma companies—

That's right. So some decades ago, Somebody had the really smart idea of, hey, your body already seems to have a defense mechanism against things that can cause you harm. So why don't we try and make antibodies ourselves such that we can then go and give those antibodies to everybody and not have to rely on the immune system itself to come up with antibodies to fend off whatever it is, you know, the invaders. And so antibodies have really nice properties. They can be highly selective. They can be highly targeted to to certain things in the body which you're trying to drug, which is really what we're doing when we create a medicine is we're trying to identify a target for the most part. Maybe that's COVID, or maybe that's, you know, some cancer mutation on a cell. And we're trying to find, you know, an antibody to kind of stick to it. And antibodies will then recruit, you know, other cells in your immune system to go and, you know, clean up that issue, let's say.

Yeah. Okay. So you can do an antibody, you could do a small molecule. I mean, there's these different therapies and they're going to go into the body in different ways. I guess something like a drug, a small molecule is sort of brute force in some ways, right? I mean, it's just like going throughout your body. Is that fair? Like, whereas an antibody is maybe a bit more targeted?

That's right.

You're like, I'm full of shit. Yeah.

I mean, getting selectivity of both of these molecules is actually still like a big challenge.

Okay.

Um, that's why one of the things that the selectivity of an antibody is often really good for that reason.

Yeah.

Uh, we just put out some, some new work last week actually, uh, showing that you could get this specificity that literally requires the model to understand things at the level of an atom. There's like a couple of added movements between like a cancer form of a protein versus a healthy form. And the question was whether we could use AI just out of the box to get a molecule that can bind one versus the other.

To tell the difference.

There are other properties as well, like antibodies don't tend to internalize into your cells. They tend to, you know, go float around in your bloodstream and only interact with things on the surface of cells, for example, whereas small molecules can really, you know, get many more places.

What's the most successful synthetic antibody, like Keytruda?

Keytruda.

And so Keytruda is used to treat cancer. And so it's, it's signaling to the body, there's a tumor here. I want you to come take care of this thing. Fair?

Yeah.

You can think of it as that.

Yeah. Yeah.

And so, so, and then, so like on some level, what you guys want to do is use AI to design something think like Ketruda, right? Which is, is we want to create, um, a compound that, that is targeting the right thing and kind of calling the body to action. And so you, you have to have it like bind in the right place and, and send the right signals. And, and it's like a puzzle piece that, like a very, very intricate puzzle piece that you have to get to fit right. And so like, in going back to your point on like how this is usually done, I mean I mean, we've been through this, like my impression of what's been going on is that we're spending more and more money to try to find therapies and they're kind of succeeding less often. Like it's getting harder to like find things that work. And so like, if you could, I mean, this is, this would be a massive deal because the biotech industry is spending billions of dollars on this stuff. And so you guys are in some ways trying to find, use, use this technology to help us get a shortcut. And so like, how would, if I am a traditional pharma company and I'm trying to find a new antibody, what does that process look like?

So using AI definitely is a shortcut, as you're saying, right? So you take that target, like Keytruda is targeting this target called DL1, and we might want to design an antibody to go and hit that target. It. Uh, now we can plug that into the machine learning model, and instead of spending a couple of months or even a year trying to find that molecule, we just get it right out of the machine learning model. We'll go test it in the lab and make sure it's functioning as expected, but, you know, 15 to 20% of the molecules we're designing now actually work as expected, which is, which is pretty crazy.

If you look at the traditional way that this has been done, uh, if you're a pharma company, it's pretty much up until this point, the way you might discover an antibody is either an immunization campaign where you might inject a mouse and, or a rabbit or a llama, and then go and extract its serum and get the antibodies from that. And then maybe screen those, do some secondary screening on those to identify the ones which you think might work. Or you might use a yeast display or a phage display where you are growing out tens, hundreds, millions of, or hundreds of millions or even billions of molecules of potential antibodies, really searching for that needle in the haystack to find something which just binds. And so it's this massive brute-force search problem, and that has been slow. It doesn't always find you things that you want to find, and it has various limitations in certain hard cases, you might have difficulty screening a— if you're going after a certain target, for example, which is very unstable or doesn't like being maintained in its soluble form, those sorts of problems have been quite challenging for the pharma industry. So not only is it, you know, can be relatively slow, and there are also things that you just can't you can't find using the existing lab-based technique.

Okay. Since day one, the Core Memory Podcast has been supported by the fine people at E1 Ventures. They are a young and ambitious VC firm in Silicon Valley investing in young and ambitious companies and people. Thank you so much to E1 Ventures for all your support. So you're doing all this wet lab work, um, even to, but like even to begin to somewhat narrow your field of what— like, if I'm trying to develop katruda, I mean, do I even have like a start of what I want that compound to kind of look like, like a theory, or I'm just looking out in nature for clues through these experiments?

You might be literally trying to copy katruda, which might be easier in that sense, but for many projects you're starting with an idea like, hey, I want to go and hit this target. And then this is where the machine learning model comes in and says, let me go from that idea to here's a molecule that you can test in the lab.

Yeah, no, I was just wondering, even like in the pre-AI era, yeah, is like, how do I, so I want to, I want this, I want to kill this cancer tumor, you know, how do I even, what am I first even looking for as to what that molecule will look like that could do that?

So we're looking for that marker that we want our antibody to bind to. And then we'll go make that protein. We'll go fish out among like a billion possible random molecules, try to find one that sticks. You'll often find one that does stick. Maybe it sticks weakly. Maybe it also sticks to other things. We'll try to clean it up. So you'll have protein engineers that work iteratively to make different mutations to that protein in order to make it ready that we think it's ready for a clinical trial.

But the start of that process has been quite literally random brute force, you know, as you get something which you think is maybe doing the right thing, then you might have people go in and start to optimize that and start to tweak the sequence of your antibody by hand or make subtle changes to it. But to get to the point where you can even do that sort of engineering on your antibody, on your molecule, it requires you to have a starting point. And that starting point has been discovered through brute force.

Okay. Okay. And then you mentioned LLaMAs. Which we talked about last time, which was a surprise to me. So llamas have some kind of like weird immune system.

So llamas make a different kind of antibody. They make these what are called single chain antibodies as opposed to two chains. If you look at the 3D structure of these things, a human antibody will have what's called a heavy chain and a light chain. It's basically two proteins stuck together, whereas a llama will just have one, and that has various interesting properties.

And we just discovered this and we use llamas for this type of of work.

And it's not just llamas, by the way. One of our board members, uh, Michael Dolsten, he used to be the chief scientist at Pfizer. And in one of his earlier jobs, he actually helped set up a shark colony because sharks also make those kinds of antibodies. Sharks are harder to take care of than llamas. So I think a lot of the industry has like moved to, to llamas or what we call camelid antibodies. Camels also make antibodies like that. Uh, but yeah, they're, they're actually really useful tools and we can make AI llama antibodies as well, it turns out.

Okay.

So there's just, there's like llama farms for doing this kind of work.

Yeah, some of our lab partners, you know, everyone sends swag, right? So, so we have a swag at Trik, bullish on biotech. We have these hats we give out. Some of our experimental partners literally send us plush llamas. So we've got a bunch of those around the office now.

Okay. Okay. Okay. So, okay, so this is the way things used to be done. And then we have this massive AI breakthrough. And so, yeah, I mean, you were hitting on it a little bit, but walk me through How much faster and same, same sort of thing. If it's possible to give kind of like a concrete example of this is where I want to bind. Now I'm telling the AI to do X to make that happen.

So if we just take ketruda as an example, process like that, there's various ways you could discover a ketruda-like molecule. You could do with the yeast display, like Jack was saying, you could do it within a mouse or even try to find something in a human. Um, overall, that process is going to take a couple of months at least to go through, often about a year if you do it in pharma. And it's both discovering that initial molecule and then you do a couple of rounds of optimization to make that molecule more drug-like. What we've found now, what we can do with AI is we can prompt the model with that target. And then within a day or two on the computer, we'll come out with a set of candidate molecules. We usually test like 20 molecules or so. And then we go in the lab, a decent fraction of those will actually have that function that we're looking for, and they'll be drug-like right out of the model.

So you, you get, you get the molecule, the model spits out a molecule, and then you are, then the next step is to like go test it in the wet lab. But you've, you've cut out all these, this blind grasping around in the meantime.

That's right.

Yeah.

The process before maybe to get to initial, an initial hit, just to find that starting point might take 3 to 6 months, depending on how fast or slow you go. Then to maybe take that molecule and maybe make it bind tighter to the target that you're trying to engage, maybe that's another 3 to 6 months. And then you've got a lot of engineering after to make it drug-like. Let's say maybe you want it to be highly stable, for example, so it doesn't melt, you know, because you want to be able to ship this thing around and you care about the conditions that you're keeping this thing in. And when we start to design these things on the computer now, we see that we can take that, that, that maybe 3-month to year-long process, cut that down into something which takes 24 hours on the computer, maybe 2 days, and then something which you can validate in the lab in a matter of weeks after that as well.

And keep in mind, even if you do the conventional technique, you're going to go discover this antibody, you'll anyways need to go and do all that characterization. If you're going to submit something to the FDA, you need to show that it's safe for patients. You need to show that it's binding the target correctly. So you just do that. All that screening step is now sidestepped with CHAI and with our— with AI models. And then you're just going straight to that characterization.

So and I know it's not like fully apples to apples. I just feel like people will be listening to this and it's like, man, anytime I ask ChatGPT to do anything reasonably hard, you know, it starts fucking up. It's like hallucinating. Like what you're talking about seems very hard. I'm pretty sure you could these complicated molecules and protein folding and all this. I mean, you know, like the odds that it's successful in doing something like this seems tricky to me. But you've obviously, obviously this is what you're focused on and your model is very tuned to this particular task. But yeah, I mean, can, does that resonate that it would be hard for people to kind of picture this working.

Yeah. And even though we're at, you know, 15% success rates, way higher than the, you know, a couple in a billion that you had before, it's still way worse than ChatGPT. If ChatGPT answered your question correctly 15% of the time, you probably would never use it. The nice thing in biology is we're doing that lab characterization. If you want to test one molecule or you want to test 20 molecules or 50 molecules, it's pretty much the same amount of work. If you're testing billions of molecules, different story. You need to get into high-throughput screening and all this fancy stuff. Stuff. But that means the models actually don't need to be at a 100% success rate. They're kind of in a zone now where they've become unbelievably useful because we can do things in that, that 20, uh, 20 scale.

Like, and when, I mean, what do you mean when you talk about this 15%?

So what I mean is that if we sample 100 molecules out of the model, 15% of them will bind as we expect.

Okay. And okay. And then, and so in that number, is fantastic for you guys, or that's something you want to go up and up and up, or that is enough to get what you need done?

Well, I think we all want it to be higher, uh, and we think that we can get there. But if you look at the areas that we're moving into next, it's not even just getting those molecules that bind the target, but it's how do we get molecules that bind very challenging targets? And also, how do we just make those molecules themselves better? So coming up with even more complex versions of those molecules that maybe we are that are not even able to work with today, or that are even more challenging to work with. Basically raising the bar for the field.

And like, yeah, I was just going to say that once you're at that 15% threshold, and in many cases, it's, that's just so far beyond the success rates for anything traditional where you're saying you're looking at 1 in a million or, and you're doing that faster and for less cost. So we are very happy with that as a starting point. And to be clear, this isn't, we don't need to take it from 15% to 50% for there to be immense value generated. That the value's already being generated today. And in fact, you know, it might cost $200 or something to test each one of these things, maybe a few hundred more to run it through the full battery that, that you might want to run on, on, on your molecules to make sure that it meets all these other properties that, that you care about. But in that world, spending, you know, 20, a few hundred dollars for 20 molecules, You're looking at something which is incredibly efficient to test in the lab. And that, to Josh's point, the more interesting challenges, and now that that's been solved to some degree, are pushing into things where it's, the success rate today is effectively 0%, where there has just been almost, it's not so much about efficiency and replacing some, making it faster, better, cheaper, but doing something which nobody could do with the existing methods. And that's what I think is so exciting about this and the biggest kind of market expansion, what will create new medicines that will, you know, end up getting to patients.

So there's like an astronomical number of combinations of molecules and you guys are helping narrow down the pool of what might be interesting that you want to spend more time looking on from a multi-month, multi-year experimentation process to kind of like I I get a command and 2 days later I'm getting told this is, this is where you guys wanna spend your time.

That's exactly right. And, you know, Josh likes to make this point that if you look at the ex number of examples of superhuman intelligence, there is no human in the world who you can give it a target and it can design, tell you here is a sequence of a, of a molecule or an antibody that will bind that target. And nor can they do that and have it also be be drug-like. So if you actually look at the, you know, there's a lot of talk of, you know, AGI and ASI, but if you look at the history of the tools and the evolution of the tools, okay, we had chess and then we had AlphaGo, but protein folding, when we solved that, and, you know, when AlphaFold came out, that was potentially the first example of a scientific system in many ways with superhuman, you know, accuracy. Maybe that's not quite fair to all of the protein models, folding models that came before it, because I think those were also pretty good. But these models can do things which a human just simply cannot do. It's a fundamentally different form of intelligence. If you gave me a pen and paper and asked me to work for years, I don't think I could design or fold a single protein to the same quality that these models can come up with.

Yeah. Okay. And then And like when I first heard this, and I feel like I did a bad job asking this question last time we were chatting. So like, you know, just as a relative layperson, I mean, this is not my area of expertise. When I hear you guys talking like this and you're like, we tell the computer to go look for these interesting molecules and we're finding, we're finding, you know, potential antibodies like right away, you know, in my head it's like, oh, so you guys are discovering novel, you just found novel medicines. Like this seems amazing. Why is this computer just not running nonstop all the time, spitting out new ones. And so, so like from what you guys have found so far to a person getting a therapy for a specific disease, what are, what are the steps? Like why? Um, it's just, that's immediately where my head, head went. And I, I still don't fully know the steps, um, that might be like preventing you from chasing something like that today.

Yeah. I mean, first of all, you can already chase things like that today with these models, right? And that, that's gonna start happening. A lot of our attention just even goes into what's next because we've talked about shortcuts a bunch here, and that's the most obvious thing to see. You have a process that took 8 months and now it takes 2 weeks, right? Nothing in AI has really come close to that kind of transformation. So that's the most obvious thing. But still, if you can get— take 8 months and then you can get a drug that's going to have $10 billion of sales a year and is going to change many lives, like you would probably still do it. What we're really passionate about is is how do you now create medicines that were just impossible to go after before? And that's what Jack was saying, these challenging targets that are difficult to work with in the lab, or even things that even if you can work with them in the lab, you just can't discover something with the right selectivity. Finding things that work for a, a mutant in a cancer cell versus a healthy cell. There's tons of examples like that that could just be transformative for medicine. And for some of these opportunities already becoming possible. So we just put out some work last week with about a handful of these really challenging examples and showing that the model could come up with solutions. There's a lot more like that. And that's really where we're taking the models next is to figure out how do we make more sophisticated molecules that can solve those really hard challenges.

So like the antibodies that you've found, um, was that kind of replicating known things and showing that you could find them or they were all—

No, they're actually all new. Yeah.

They're all new.

We needed to check generalization, right? Okay. You want things that aren't in the training set, for example, or in some cases things where you don't even have known molecules. But again, there's still— we think the models are going to keep getting better for the next couple of years and enable some really incredible kind of molecules that just weren't possible to be made today.

But like out of the ones you found, why they don't appear to be therapies for anything, you know, like why, why would you not be developing those into medicines right away if, if they— you can tell that they would be helpful for something.

Yeah. Yeah. I mean, we could. And this, by the way, is one of the challenging things about working on a problem like this. It's like the impact that you could make today is so large. And there's always this question of like exploit versus like explore, I guess, like the short-term, near-term thing. So we could start making those molecules today. But if we can make a molecule a year from now that's going to be 10x better, then like what do we do about that? Right. And what probably will end up happening across like the biopharma industry is that The models will continue getting better. People will be launching more and more molecules every year and like patients need molecules today, so get them out. And then if we come up with better versions of those a year later, okay, all the better, right? We'll just keep pushing those through the pipeline as well.

So the idea is to like keep pushing the technology forward and to, I mean, like, do you already have in your head, this could be better suited in the short term to cancer versus an autoimmune disease or, or like, what are the, where are the first places you guys are looking.

Yeah, I mean, the models don't seem to have an opinion on that. They seem to work across the board, which is really interesting. And again, going back to the company's mission, right, turning biology from science into engineering, the way you move to an engineering problem is you have basic primitives that can really be used across the board. So we don't think this is going to benefit preferentially one kind of medicine versus another. Maybe it'll— you'll actually see more of the impact in areas that again have been harder previously. So there you might have a bias in one area versus another potentially. But overall, we think like this is going to become the way that people design molecules the next couple of years.

And then, okay, I still feel like I'm stupid on this. Like if we're talking about some form of lung cancer that we have no treatment for today, like you could put, you could tell your model right now to go try to find an antibody for that.

So, so, uh, there are many different steps involved in creating a medicine. Medicine. The first thing is identifying the right target, for example, and then making sure that you know that this is actually what the— people talk about getting the biology you want, understanding that if I take this target and bind it in a certain way and engage it in a certain way, that that's actually going to result in the therapeutic effect that I want for my medicine. And so So there's that part of the process of still figuring out what it is to go after. Then there's the middle part, which is, given that you know what to go after, how do I get a molecule against that? And that's really where the models have got very good recently. And then there's the final step, which is actually then going and running trials and putting into the right patient populations and getting things into the clinic. And of course, that's still a long process which requires a massive amount of investment of time and resources to run those trials effectively. And so there will be, I think, a huge clinical trial bottleneck in the next few years as more and more medicines against this backlog of targets that have been traditionally challenging, that we start to— these models start to produce some of these. And I think there's an interesting question of, as a society, like, what do we do about that? Do we just accept the status quo that yeah, clinical trials take a long time and they're expensive, or do we find other ways to accelerate that? And you're already seeing things that the FDA is doing, for example, around maybe saying that at some point they're going to loosen the requirements around animal testing, 'cause that's a precursor to putting things into humans. But maybe that shaves some time off the process. And so I think there are very, people forget the historical contingency of the FDA. It's only has existed in its current form for maybe between half a century and a century. And I think there will be, as these models get better and as we create new medicines, I think there are likely changes. I'm not sure it's happening this year or next year, but we're already seeing that things are starting to move at least a bit there. So yeah, I think there's, developing a medicine is, goes beyond just the process of discovering a molecule. It has many other aspects to it. But if we take that targeted problem of just given the target that you are trying to address, can we design a molecule to that? That part is getting really, really efficient now.

So you think we'll have this technology will result in so many interesting molecules, it'll just be hard to test them all. And then, like, I hear your point on the animal testing, although also, I mean, the government has been trying to curb animal testing and in favor of doing like simulations, which sounds great. I'm not sure the simulations are up for the task, but then just as I'm like playing through what you're talking about in my head, I'm just picturing this flywheel of interesting molecules being tested in some simulated biological— I mean, you're nodding like that. That's kind of where we're potentially going?

Well, it sounds a bit bold to say this on a podcast, but if you play this out, like the trajectory of where things are going, you know, the accuracy of the models is approaching the resolution of the wet lab. In some cases, actually even better. There are times when we get weird results in the lab and then we do the experiment again. It turns out the model was right because there's this thing when you look at like one data point at a time, you know, there's noise with each one of those data points. But when you give that complete picture to a model, You know, the models can look at billions of data points and understand what's going on. It can really come up with that complete picture. So, you know, I could go test something in 10 different mice and I could go take their livers and I could, you know, go run some experiments on that. But if the model has seen that so many times, it might even be more precise than some of those experiments. It's a big idea, right? And it's a bit, you know, it's hard to say this without having that data today, but you can imagine something like that might be coming.

Like, it always seems like every awesome therapy always has side effects. And, and I mean, they really like, you know, just suck in a lot of cases. I mean, is the, is like the potential here that it just turns out maybe the computers are just so much better at this that we, we don't get as many side effects as a result of these therapies, or is that just kind of wishful thinking?

100%. Yeah, there's a study a couple of years ago, I think it was in maybe in Nature around just the number of off-target effects that drugs have. And I think if you look at the average drug, it has maybe 5 off-target effects. This is interacting with things that you didn't intend it to interact with. The aspirin, it's somewhere between 20 and 25. And those are the ones that we know about. And so I think if you Many of the medicines that we have today are just highly unoptimized in some way. Now, maybe some of those so-called off-target effects are actually helpful in some regard. Maybe they're actually important, and maybe it's doing something we don't fully understand. But in many cases, it's just something you'd really hope to avoid. And so I do think that we can, if you look at, I was chatting with the team the other day, we all believe that when we look back on the last 50, 100 years of medicine, that 100 years from now, we'll look back on this as almost the Bronze Age of—

Just like brute force.

Exactly, of how we deliver medicines to humans. And that if you think about, okay, we've got these models now which can build therapies with an atomic-level understanding by scaffolding things and really placing each individual atom in space. How can we extend that now, that ability to really program and engineer atoms into more complex therapies? And what will therapies look like 50 years from now? And will they be, you know, simple, just therapies which bind another therapy and another target in the most obvious way? Or are they going to be in some ways logic-gated or have some dynamic elements to them? And so I think as these these models continue to get better, the quality of the medicines, yes, the obvious stuff of avoid off-target effects, but also the complexity of the medicines that we'll be able to build and the circuitry we'll be able to build into those is going to increase dramatically.

So, and these models are being fed on what, just like huge volumes of DNA data or all kinds of datasets.

So that could be the sequences, right? DNA or protein sequences. That could be the 3D structures of what those proteins look like. That could be measurements about how proteins interact with one another or other various drug-like properties that we might have measured. So all kinds of things.

And mostly like when you guys trained your model, don't know if I'm getting into proprietary territory or not. I mean, it's like mostly public databases of this type of information or there are companies with vast stores of proprietary data?

I mean, there's, there's both. We also produce a lot of data at Chai. I think one thing people are sometimes surprised to see or hear is that there actually is a lot of publicly available data for biology. Uh, if you look at the Protein Data Bank, for example, uh, this is one of, this is the thing that gave us protein folding models. There are a couple hundred thousand 3D structures that people determined. It used to take an entire PhD to go and discover one of these things. That's estimated at like $100,000 of work. Network to go and come up with a 3D structure. Now it's a forward pass on a neural network, but the reason why that's possible is because all this data that's been collected over the past couple of decades of scientific research.

But like when Jack's talking about these, the models getting better, um, I mean, the same sort of question that the foundational model guys always get, it's like you've already hoovered up all of the internet's data and, and now we either have to work on pre-training or we have to refine techniques or somebody has to come up with something new. I mean, is it for the moment, have we already hoovered up all the relevant data you guys would want and we have to have these same sort of breakthroughs? Or since this is such a more, um, targeted field, you guys, there's plenty of data still to be had.

Yeah, I think we're still scratching the surface.

Okay.

Like, so how will we, like, what would you, what if you could have anything, where would you get, what, what kind of data would you want?

And Oh, I mean, all kinds of things. The data I think we're most interested in is, is what would help the model the most. So this actually launches an interesting question, which is like, now what we're starting to see is that the models actually help us curate or create the data. So when we go into the wet lab and we start to test new proteins, those are not just random proteins that we're creating to build training data. That's the model actually designing the experiments and then getting data that's fed back into that same model, which is like a wild idea, right? It's like an LLM. Actually, it is what's happening with LLMs. People talking to LLMs to kind of fine-tune them in a certain way. Like you have a conversation, you say that was a good answer, that was a bad answer. And similar kinds of things are starting to happen with lab experiments.

And you guys, it's a relatively small team, right? How many people?

We're about 20 right now. We were, we've doubled in the last few months, but a lot of the core R&D was done on a very small team.

And you started in like 2024?

Yeah, that's right.

I mean, this all feels very dramatic. I'm not trying to, I'm not trying to like overhype this, or, because it's like hard for me to even wrap my head around a little bit.

Um, I mean, it's dramatic to us as well. I think that the, the progress in this field, we were expecting it to take, uh, a number more, like many more years before we had results of this stuff. And it was, it was truly a surprise to us that models in such a short space of time could get to where they are today. And even if you look at, you know, CHAI-2 came out at the end of June and we published an update on it just last week. And it turns out that things went from being these kind of research curiosities models, which weren't that valuable to now being incredibly valuable in a space of a matter of months. And potentially the most interesting part of that is that, especially with the work that we put out the other week, is that these molecules are not just, you know, binders, but they look like drugs. And they don't just look like drugs, they can also address these hard targets. And so it feels like it's one of these things where nothing was working and then suddenly everything just started working in one fell swoop.

Okay. I don't want to— I don't— you guys can choose whether you want to do a tit-for-tat or not. I'm not trying to set one up, but there's a company called— is it Nabla? Nabla Bio in Boston. I mean, they seem— it was just either last week or the week before to put out an announcement similar time to you guys. I had same kind of thing. I had some of my smart friends pointing me to them. I don't know, you— the two of you seem like you were jumping out in the zeitgeist of, uh, of interesting things going on. But I mean, like, my bigger question is not really like, how do you stack up or what do they do? It's more, um, like, how do you, how are you, how do you stay ahead of or different from any competition? Um, with people chasing us, it's the same thing as like the foundational model guys, I guess, to a degree. Yeah.

Yeah, I think both, both companies have pretty small teams. I think focus is really what matters. You know, we spend a lot of time thinking about what's the problem we're going to solve, what is the right way to solve it, and then, and then what's the right experience to make that thing useful? What are the right experiments to do? So a lot of this work we announced the other week, I mean, we had done this over the summer and we were— it's the same model that we put out with Tritu a while back and it was more like, hey, it turns out the model can also do this and we haven't even talked about about, you know, the new models that have been in development. But yeah, I think like what the reason why our company has, has worked as well as it has is just that relentless focus on solving a problem. People don't leave the office until the experiment that they want to launch that day is done. Over the course of months, it's like we are just going to keep pushing this zero-shot antibody design because we think it can really work. And I'll tell you, the first couple of times we tried, it didn't work at all. Right. And there's a certain perseverance that I think we all have because because we know, or at least we think that there are some really big things that are going to be possible here in the coming years. And it's been happening faster than we expected. And that definitely helps as well, right? When you're ready to spend 2 years on a problem and you get the answer in 3 months, you know, that's, that's, that's very energizing.

Yeah.

Is it, is it crazy expensive to do these runs of the model?

I mean, the costs go up over time too, right? Because you put more compute, you put more data in all of that. So yes, we've had to raise some, some money to make this happen, but the value that's going to be unlocked here, like, makes it totally worth it, like, 100 times over.

Our biggest line item beyond people is compute.

Yeah. Okay. And then, I mean, this is, I'm still trying to wrap my head around this because I still don't feel like I understand it. If you're identifying these antibodies, why we're not running around making therapies. I mean, Jack, you, you said, I think this was in an article I found, we're not at a place yet where these are drug leads, for example, which somebody would put into a human. Okay, you guys are laughing.

That wasn't until a few months ago.

Well, I think, you know, that statement was true when we released CHAI-2, and that was as of a few months ago. And then fast forward a few months, and the paper that we put out, the technical report that we put out just the other week, is suggesting that these do have many of those drug-like properties that, you know, maybe in certain cases they need to bind a little more tightly, or maybe in some cases you want to make to optimize it so that it's a little more stable. But in the majority of cases, when we look at certain key drug-like properties, it's ticking all of the boxes. And when it comes to at least those developability properties, or me, I shouldn't say, my scientific team gets unhappy with me when I say all. So I should say many of the ones that we— The ones we've tested. The ones we've tested, yeah.

Yeah.

So, you know, I think that's part of what is so exciting about these fields is even a statement like that was made 3 months ago is now seemingly out of date.

Okay.

Okay.

I think this is helping me understand. So you're saying like when you were, when that first CHAI-2 announcement was coming out, you're finding antibodies and please for real, correct me if I dumb this down too much. So it's kind of like the puzzle piece is fitting, but it's still missing some of the characteristics you might want to see for some like long-lasting reproducible drug, but you, you guys can kind of like tell you're on the right track. And then now you're saying the progress you've seen is that now what the model is spitting out is looking a lot more like some of the very successful drugs, therapies. And is that, is that kind of right? Because I, I just don't know enough, you know, to know where to poke around on some of this.

That's why it's funny you mentioned that quote because this drug-like thing, that was the next challenge that they were trying to solve. And it was crazy to see how fast that was solved. Turns out that that model was already capable of doing that and we just never tested.

Okay.

Right. So we're like, we need to get drug-like molecules. And then we started to do the baseline. Let's go take those molecules. Let's check their drug-like properties. And we're like, whoa, like they actually have the drug-like properties already. So it's not even that it was out of date. It was almost just like we didn't really try and test. And then once we did, the data was just so clear that like we're even further ahead than we even thought we were.

No, I think it was just like the first time we chatted, I'm just like, well, I don't understand if you have all these antibodies antibodies you discover, why you're not just running around doing all this. I mean, I have done like a couple of pharma stories, like when you talk about drug-like properties, I mean, I know there's like some medicines, they just, they crystallize weird and they don't stay in the right form even though they do the thing they're supposed to do. They become kind of useless because you just can't like put them on a shelf. And I mean, is that kind of what you're talking about?

All sorts of things.

Yeah.

Another example might be maybe you don't want your antibody to bind to itself and aggregate. And This goes hand in hand with like viscosity so that when you, maybe when you come to dosing a patient, if the thing is really viscous, maybe you need to deliver the medicine in a different way. Maybe you can't do a subcutaneous dose, which is a much nicer way to receive a drug than something that like needs to be infused, for example.

Okay. And I get that you're very hardworking and very smart, but like why would every biotech company in the world, I mean, if you guys have done this with with 20 people, why would they not just hire 200 and, and just throw all the money they can at this and, and all have a department? I mean, I'm sure they will have some version of that at some point, but yeah, it just, it seems like, um, like all this stuff in AI, just very hard to kind of like stay ahead of people.

Well, there's, there's not many more than 20 people on the planet, I think, you know, capable of building something like this. Yeah, I mean, many of us on the founding team have been working in this space for a long time. Like our co-founder at, had built some of the first deep learning models for protein folding. It was actually like the AlphaFold 1 architecture essentially came out from his group. Jacques was building, you know, head of AI at this company Achemia, building AI for small molecules. We have a whole host of folks that have been, you know, building some incredible stuff here.

I think there are more than 20 people capable of building it, but I think also it's just a matter of when it comes to the quality of the level of rigor around actually building these models, the making the advancements to the next version. If you are in the same way that if you are a, you could say that you can train out an LLaMA GPT-4 style LLM now, or a GPT-3 style LLM with a few H100s and a small, all of the open source piece, the building blocks are there to make that happen is a very small team now. And people have done that. At the same time, there's incredible appetite for the latest frontier models. I think it's not just about capabilities today. It's about pushing into the next set of capabilities and the set of capabilities beyond that. For that, I think it takes a huge accumulation of talent and the talent density, a certain set of resources, because progress is not leveling out here. In fact, this is really just the very beginning for this field. We're 2 years in, and I think the I think the competition in this space is gonna be won and lost over decades, not in the next, you know, 6 months. And so at the same time, it is true that we like to say that the work we do is craftsmanship, not labor, in comparison to maybe building enterprise software, which is just a known quantity right now. And I do think that there is a, the talent pool in, I think in the domain and many of the domains that you cover is actually just far smaller. Yeah. That when we speak to many pharmaceutical companies, for example, they've made an active decision not to build these models themselves. They know that it's hugely expensive to get the talent and then even more expensive to get the chips. And that even if they were to invest in all of that and bring that together, that at the end of it, they might be left with results which are no longer cutting edge. And so I think there is a huge appetite and a kind of an acknowledgement within the the existing set of companies in the same way that the kind of the automotive manufacturers realized that they weren't going to necessarily hire the teams to build self-driving themselves. There is, I think, a certain acknowledgement of that as well in the pharmaceutical and biotech industries. You know, it's complicated.

And does the, I don't know enough about AlphaFold to be dangerous here, but obviously they've paid a ton of attention to that. Demis is one, another Nobel Prize and takes his science very seriously. I mean, are they, are they already working on things similar to you guys, or you guys are on two different paths?

I think it's hard to say from the outside because not everyone talks about what's happening on the outside. We've been a lot more public with the kind of work that, that we've been doing. So it's hard to say. I think on a surface level, we've been more focused on these antibodies where they've been focused more on the small molecules. But you have to imagine that, you know, everyone's going after this. Like, there's a really big vision that's in front of us. So it's hard to even draw like any lines in that sense. But, but look, it's also much more than just building a model, right? And I think, I think Pharma recognizes this too. It's one thing to build a model, but then there's all the validation around that model. We do a ton of lab experiments to understand in what forms does the model work well and what forms does it not work well? How do we use that to then come up with the next generation and make them better? Better? How do we even build the software and the APIs to use the models in a way such that you don't end up with some bug that just appears in the lab as like all duds of like nothing being active? So there's, there's the model is definitely the exciting thing to be talking about on the podcast, but there's a ton of other harness and infrastructure that goes into like really making this thing a reality.

Like going back to that question I was hitting on of are these models doing scientific discovery? Are they going to places a human can't? I mean, you said before, if you had to sit down with a piece of paper, it would take you a long time, but I mean, assume a human had infinite time already, like what you're seeing, you know, is this model doing, is it in your estimation kind of coming up with compounds that a human just would never stumble upon or sort of imagine? Like, is it doing scientific discovery?

We actually know quantitatively because we can take the molecules that come out and then we can compare how similar they are to other molecules that exist either to that target or to any other., and the molecules that come out of our models are actually vastly different.

So it's doing like the, like when AlphaGo started playing like no human had played before. It's just kind of going down a path that a human probably would not explore.

We actually have some of these like Move 37 style things in the paper that we put out the other week, just going into detail and we say, what is the model thinks going on here, right? What does the 3D structure look like of how the, the potential drug is interacting with the target, and it does some really creative things.

So like when you hear Dario from Anthropic, I mean, I just saw him like a couple of weeks ago and he's saying, I don't care what the rest of the world is saying about some kind of slowdown, whatever. In 2 years you'll see just incredible science. This is all over the place, like exponential. Like that resonates with you guys? Yeah.

Yeah. I think, you know, in the, I've been getting questions in the past couple of weeks from friends on the East Coast who work in finance and are worried about the bottom falling out of the market with AI. At the same time, you know, when we try and get our hands on some GPUs from cloud vendors, it's been a struggle in the past few weeks for the latest generation of chips because everybody's— there's so much investment happening now. So it's funny to hear the market's reacting in a certain way, then on the ground you just see this insane endless demand for companies like us to apply this. And I think my, you know, from our, you know, this is maybe not a comment about the whole of the field and I shouldn't, you know, take a view on like what, you know, the market as a whole, but certainly in this domain, the distance between what we were able to do 2 years ago and today is almost unimaginable. Unimaginable. The level of economic value that's going to be generated, and beyond that, the level of human value that's going to be generated by virtue of these advancements is immense. If that's any reflection for the broader ecosystem, I think we started here with one of these questions. I think that the reality is that, yes, AI is making scientific discoveries today. Maybe it's not quite in the realm that people think, or that, you know, maybe it's not ChatGPT itself doing that. Although I do think this year we've started to see the first signs that even reasoning models can get into that arena and be quite dangerous. But I think if you look at these domain-specific models, for example, those are, really doing things that are fundamentally new.

Yeah, it's such a strange time. Like, as I'm not— I don't really have a dog in this battle, you know, it's like, it's, it's— I would prefer not to live through some economic collapse, you know what I mean? Just, just, uh, on a personal level, I might— I generally think this is clearly some breakthrough technology. It just may be quick, it may take more time, but But things will happen. I, it's so, it is, it's an unusual technology in that you can both be like so impressed with it and underwhelmed at the same time.

You know what I mean?

Is it's like clear when I'm playing with any of these chat models, it's, it's like, oh, well, you know more about physics than I will ever even, you know, could ever even hope. But then you're so fucking stupid when I ask you to do do some really basic thing. And then I see like my— there's a couple friends in the more like mainstream media who do sort of like politics podcasts, and I just hear the way they're talking about AI. I mean, they already seem to think this is, is, um, it's all smoke and mirrors and, and they're just so stupid. And, um, I don't know, it's funny when I hear them talk about it because I find them pretty balanced on some other stuff. And I'm like, that's— I don't know, That doesn't seem like my experience. And then, and then yeah, I just saw these last couple of weeks, all the stuff coming out of the biotech world. And, and I mean, just to me it was jumping out like, if this is true, this seems quite, this seems actually quite remarkable. It is a weird, it's very weird technology to wrap your head around.

It's something to be really aware of in our space as well, because AI drug discovery as a field has been around for a long time.

This is what I was going to say. I've been hearing about this for like 8 years. I mean, this is like before, you know, it was just like, it was just big data. And it was like, we're gonna figure out every— I've been told this so many— this is why I'm like scarred.

Yeah, yeah, yeah. And, and, uh, and a lot of people have, right? And it's because it's been smoke and mirrors for a long time.

Yeah.

But this has really been the year when things have started to work.

Okay. I want to know if it's real.

I hope no one said that, uh, here 8 years ago.

No, but I mean, I— there was like so many— I, I'm not kidding, I just like stopped. I stopped like doing interviews about this stuff because it was just— it was I'm like, dude, you told me we knew everything about proteins ages ago and the computers were gonna figure everything out. And then, and then like, it, it does seem like what you guys are doing is different. I just, yeah, I'm just emotionally taxed.

I think that was good work that was done. Some good work in the, that, that time, a lot of it was taking an existing drug and redesigning it or optimizing it, for example. And I think that's why, you know, the fundamental leap here is into generating things that are fundamentally new where you don't have a starting point. And that's what's really started to work. And in terms of the impact of these sort of the hard-to-drug targets and potentially even the undruggables, I think we are— the previous work has all been building up to this and we stand on the shoulders of giants there. But I think in terms of the impact on real medicines and real patients, I think that S-curve, we're right at the start of that and it's kicking off right now.

What are things considered undruggable?

Yeah, you want to take that one?

Yeah, I mean, there's a bunch of examples. So when we think about undruggable, it's often things that are just hard to drug, hard to target. So an example category here is often what are called GPCRs. That's a mouthful. It stands for G-coupled protein receptors. But think of it as it's this protein that's embedded in this membrane of a cell. And one of the reasons why that's so difficult to drug is that it's literally embedded in the membrane of a cell. So if I take it out of the cell, it doesn't necessarily look the same anymore. Right? So it makes it very difficult then to go and, you know, put a protein in a, in a mix and like pull out the one that binds to it. Um, the nice thing about using AI here is that we can generate a small number of designs and then check them, check them directly on the cell. Eventually play this out further. Imagine testing this directly on patients' cells, right? You take a patient that maybe has some mutations on those GPCRs or, or has a certain signal that needs to be activated through those GPCRs. Take those cells, throw our drug on it, and then very quickly figure out if it would work. That's the kind of thing that starts to happen when we have these very high success rates of designing molecules. So it takes a process that was undruggable or hard to drug with the tools that were around before. And now it brings it in the realm of, of engineering where we can actually just make something to try to solve the problem.

So, okay. And then this is, all right, I'm gonna ask you guys to restrain yourselves a little bit, cuz this is not just to like pat yourself on the back or tell me, you know, how amazing everything will be, but like, okay, in the best case scenario, you keep improving your model. I mean, what would be some kind of realistic timeline for a therapy based off the work you're, you're doing?

I, I think we'll see the first de novo designed therapies, uh, from, from our models at least, uh, going into, uh, patients, I'd say certainly less than, uh, 3 years. Um, probably more than 9 months, I would guess.

Okay. Those are sort of my error bars because you have to do, you're gonna have to do some round of animal testing On all this stuff.

Animal. Yeah.

You'd be surprised actually that the slowest part of what we would have to do here next would be actually manufacturing the molecule and doing that, getting enough material to start testing it in animals and in patients.

Because it's slow.

Yeah. That's, that's one of the next problems to solve.

Yeah.

Because like you're sending off, you have to send this off to China for something or like, I mean, I mean, that's, that's easy.

You should send an email with your sequence, right? It's, uh, you put these things into these, uh, into these massive baths, basically. It looks like growing beer, actually. And then you're, you're growing up the protein, but it needs to go through, you know, very clean manufacturing processes. Everything has to be regulated because you're going to be putting this into a patient eventually. I think the bright side here is that one of the things we looked at in that technical report we put out the other week is the yield of the antibody. If we put this into a cell and then we have the cells start manufacturing the antibody, how much do we get? That. And things are actually looking pretty good right now. We think we can probably make that even better.

Okay. And then I know we talked about this last time, but I don't know if you changed your mind. Um, like as far as you guys developing your own therapies as opposed to partnering with people that do this all the time, what's your latest thinking?

The latest thinking is there's going to be a massive amount of impact that's going to be created by these models. And that's why the number one thing that we are focusing on right out is continuing to make these models even better. I think there's going to be a ton of exciting opportunities to really use these models to create drugs. And, you know, whether that is being done in different indication areas like we talked about or with different partners, these are all things that I think we'll figure out.

So you could either do kind of like chase your own blockbuster drugs or you could do some kind of like revenue sharing royalty type thing. Is that, are those There's a ton of models that are possible here.

Yeah. Broadly speaking, those are the two poles. And if you look at most biotechs historically, if you look at the so-called Wave 1 biotechs, many of them chose to make their own drugs and directly push through all the way through clinical trials and commercialization. There are other companies that have taken more platform-style business models and where they sell their technology to others, or partner with them. But even for people making their own drugs, most biotech companies tend to have a handful of pharma partnerships because the cost of developing a drug is so astronomical that you need to have a— you can either dilute yourself and raise a huge amount of capital, or you can do that but also bring in a source of non-dilutive capital by putting your technology to work and helping pharmaceutical companies go off and develop develop their own drugs, or maybe do those drugs in partnership with you. And so that's traditionally how the model has worked for most biotechs. I think that's starting to change a bit though. And I think a lot of what this technology's going to do is going to, I think, upend some of the traditional go-to-market, the commercials, the deal structures, because a lot of the hardest parts about discovery, if not this year, then certainly in the next few years, If you're just looking at designing a molecule, a lot of that's going to shift away from something which needs a massive, massive partnership to do.

And by the way, this is the thing to reflect on, like play out what happens when the molecules get so good that the molecule that comes out of the model is going into a patient. How do we even think about the difference of like, is that selling software or is that selling molecules? Like what happens in this world when things get so good? And I think that's why when you answer this question, like the lines start to get actually very blurry in terms of like what partnership actually looks like in this space. And that's why at the end of the day, what drives us is how do we create the best models and then how do we create the most value for the world with those models?

You seem so nice and so smart. Like when you say something like that, you make me believe this could happen. I mean, so you think, you think we could reach some point where we just, we know it's that good. That the computer spits this out and you take it.

Yeah, I think what's, well, maybe it's not just taking, right? We want to do some experiments as well. You probably don't want to be the first one to take this magical drug, even if, you know, the model tells you it's good. But one of the wild things you see with these models as well is that they're actually, they know when they're going to work or not. Like ChatGPT has gotten really good at this as well. If you ask it a question and it doesn't know the answer, it says, I don't know. What does that look like for drug discovery? And when we actually have that in our models today, also the model says, I know I can make a molecule that binds this part of the target. I'm not really sure about this other side of the target. And that's actually a very profound capability because again, it's, it goes back to this point about like simulating lab experiments. It's essentially the model telling you, I think that lab experiment will work and that one won't. And that means we can concentrate our efforts on the opportunities that are, that are really going to matter at the end of the day and the ones that are most likely to work.

Work. All right. I'm going to set you guys free in one second. Have you, is there like one molecule you're eyeing that seems the most promising so far?

I mean, the hot thing to do right now is obesity, but yeah. So I feel like if we don't have, if we don't, you know, work on that at some point, it's going to be a disappointment to the world at large. But no, I think there's far more creative things that you can do. Beyond that. And, and yeah, I'm not sure if you have any, any—

I think the stakes here are way higher than any one molecule. I mean, I'm most excited about what's going to happen.

There's not one where you guys are like, this is going in the mice first.

The thing is, it changes every day, right? Because the models are just getting better. We're trying more and more things. We're trying harder and more ambitious things every day too. So if you would have asked me that question last month versus this, literally this morning, I mean, it changes, right?

Yeah.

And, and again, that's why when we think about the value that gets created here, it's going to be much more than one drug. Look, and that's also, by the way, a really bold claim to make. If I told you that, you know, our life's work was creating a machine learning model that created one drug that saved one person's life, it probably would've been worth it. But I think the opportunity's going to be even much larger than that.

And that we, ah, this is a horrible question to end on. I feel like it's unfair. I mean, if you have some magic drug-making machine, I mean, isn't there a case to be made? This has to be like turned over to civilization., you know, in some open source way.

Well, you still need people to fund the trials, to fund the manufacturing.

Yeah.

You know, there would still be a separate infrastructure which would need to build up around that to commercialize it. 'Cause those parts are still incredibly, you know, resource intensive, for example. And if you look at the amount of dollars spent on developing a drug, the majority of that is in in the clinical trial when you look at an individual drug, maybe the amount of R&D spend as a whole in getting ourselves to interesting drug candidates is also pretty significant. But I think that the, you know, again, when we think about drug discovery, it is a big process with many parts of which we've, we're very close to, you know, solving in at least to some degree one component of that process. So I think, you know, if we want to, you know, it's the same argument for the pharmaceutical companies themselves in that, You know, if they were to cease to exist, the pipeline for new medicines would, right, would go away. And so, you know, what is the, you know, acceptable margin for those companies, for society? Well, it certainly shouldn't be negative or zero, 'cause like you do need some mechanism to fund that. And it's not at all clear that all of the public funding in the world, you know, if you were to add all that up, could get us the same, you know, quality of, uh, uh, and, and amount of medicine that we do get from, um, from these companies.

Well, and we're in this spot today where I think this is right. I mean, the US is just paying this incredibly disproportionate amount for like the world's R&D on these things. And then, and then it's funny, I mean, it feels like people are a little bit down on biotech overall in terms of investment. Drugs have been really hard to come by. I mean, this is like, this whole model is, feels like it's teetering on the edge all the time. And people are always chasing new blockbuster drugs and they just seem harder, harder and harder to come by. But if you like the most, I don't know if optimistic is the right word, but productive version of what you guys are laying out would be some massive disruption to this whole system.

Yeah, I think it's a platform shift. It's like web or it's like mobile or it's like AI. And we're at the start of one of these new S-curves and it's that, you know, part of what happens is when the technology stagnates, right? Or you've had, you know, you see this with everybody's no longer hot on enterprise SaaS because the time to do one of those was 15 years ago. And unless you have an AI angle, maybe the growth is leveling off. And it's sort of the same in biotech as well, in that maybe there was the first, you know, monoclonal antibody drug was, that was maybe 50 years ago at this point, but they really started picking up in the early 2000s. And now we're awash with certain kinds of antibodies, for example. But when you think about new leaps and the new platform shifts that can get us a whole new set of molecules, new modalities, molecules against targets that we couldn't go after before, that is hugely, or paints a hugely kind of encouraging picture for the ecosystem. And yes, you know, it's been the perfect storm for the biotech industry in the last few years. But I think what's really restored a sense of optimizing, this is the sort of the shining beacon of the industry right now. And I think is going to, you know, recognizing that we're still even in the earliest days of this, as this stuff keeps getting better, this is the path to, to building fundamentally better medicines, fundamentally more sophisticated products. And that's not gonna play out over a, you know, 2, 3, 2 year time horizon. That's gonna play out over decades and centuries.

It's, well, it's all very exciting. It's all comes to pass. The, if one of these models creates some like general purpose cure to cancer, like can a model win a Nobel Prize?

How will this work? Yeah.

Cancer is not one. This is like, do you guys get it? I know. Yeah. But I mean, do you guys get it or does the model get it? You know who I think the team gets it.

If you ask me, that would be the right thing.

Well, thank you guys so much for coming in and anything in particular people should check out. I mean, I know you've got your, the papers on the website and is that the best place to look?

I'd say that's the most exciting, right? That's hot off the press.

Yeah.

New stuff. That we thought was impossible until a couple of months ago. It's quite technical. There's a lot of details in there, but throw it into ChatGPT and it'll tell you what's happening. We have a lot of models with an explainer. We actually have some of these great like visualizations of what's going on too.

Yeah.

So to make a lot of these like drug-like properties clear, like what does it mean for a molecule to be stable? What does it mean for it to be polyreactive or not polyreactive for that matter? So we have all these animations that kind of explain what's going on. So check out our blog as well.

Okay. Okay. Okay. Um, well, thank you gentlemen so much. I'm sorry to, uh, drag you through some weird version of San Francisco traffic for this.

That was well worth it. Uh, thanks for having us on. It's been really fun. Yeah.

Thanks for having us.

The Core Memory Podcast is hosted by me, Ashlee Vance. It is produced by David Nicholson and me. Our theme song Editing is by James Mercer and John Sortland, and the show is edited by the John Sortland. Thanks as always to Brex, NE1 Ventures for making this possible. Please visit our Substack, YouTube, and podcast channels to get more of what Core Memory makes. Thanks, y'all.