Founding Unlearn and Revolutionizing Clinical Trials with Charles Fisher

Welcome to the Axial podcast. Axial is an early-stage investment firm based in San Francisco. We partner with great founders and inventors investing in early-stage life science companies, often when they are no more than an idea. Axial is fanatical about helping the rare inventor who's compelled to build their own enduring business. Boom. Hey Charles, uh, hope you're doing well. Uh, thanks for doing this. I'm really excited to just have a conversation with you and really walk through the story more importantly. And then we talk about Unlearn and all this stuff. Maybe for everyone, you've done this a bunch of times already. You've done like 5 podcasts. So you're going to be like a Joe Rogan pretty soon. So be prepared for this. This is preparation for Joe Rogan and a few other things, but maybe do a brief introduction yourself and then we can go from there.

Sure. I'm Charles Fisher, founder and CEO of a company called Unlearn AI. Before that, I was a theoretical physicist/ AI researcher for a number of years. So I've been working now at Unlearn. Started the company almost 6 years ago. So we're about 6 years in. Yeah, excited to just kind of have a conversation chat today.

Cool. Maybe I think what I like about these conversations is the story of understanding how people got from scientist or entry-level person all the way to like the best in their field. And so wherever you want to start, but how did you— how did you get your training as a biophysicist? More physicist than bio from my perspective, it seems like that way. How did you get interested in science in the first place? And then how did you get kind of roped into doing grad school and working with I'm blanking on the guy's name, Colin or—

Yeah, Colin, yeah. So everything is kind of just like a series of accidents. Like none of my life, none of my adult life was in any way planned. So I went to college at the University of Michigan. And when I enrolled, I didn't have any idea what I wanted to major. I didn't enroll with a major or anything like that. I just enrolled and I think my first year, my entire first year at Michigan was all taking all of these honors classes that you have to take, which are all humanities. So my whole first, like freshman year, taking like great books and reading. You read all these classics, it was like all classic, like the whole first year of college was like classic. And yeah, I had no idea what I wanted to do. I got a research job at Michigan State, 'cause I'm from East Lansing originally. So I went back home. For the summer and I got, I worked as a research assistant at the radiology department doing magnetic resonance imaging research over the summer. And that's really what got me interested in science. And it was also interesting because I originally did it thinking I was like interested in medicine and biology, but I ended up being much more interested in the physics of the MRI. And so when I went back to Michigan, I declared, I was like, okay, I'm gonna double major in biochemistry and math. And they came to me eventually after like, I don't know, after a year or so when they were like, Charles, we're going to make a new major called biophysics. And would you like to switch from these two majors into just one, just do biophysics? And I was like, yeah, that sounds great. You're half the classes. So I switched over and did biophysics. I was always the first graduating class. From that major. There were 4 of us. And was working on magnetic resonance spectroscopy stuff. NMR spectroscopy theory, actually, of NMR spectroscopy mostly. And then went to grad school at Harvard. Cool.

One quick question.

One quick question.

Because you grew up in East Lansing but went to Michigan.

Who do you root for?

Do you root for the Spartans?

I root for Michigan. No, no, no. I root for Michigan. Yeah. You're from Michigan? Oh, yeah. Yeah, yeah. My dad's a professor at Michigan State. So we have an in-state rivalry.

I know there's, I think Michigan State, this is being recorded March 21st, so I think Michigan State's in the Sweet 16 right now.

Yes, and Michigan did not make the tournament.

Exactly, so.

Yep, so yeah, I went to grad school, yeah, worked with Colin Stoltz at MIT, which is always a little confusing 'cause I, My degree is Harvard, but I actually spent the majority of my time at MIT. And then after grad school, I actually didn't exactly know what I wanted to do after grad school either. I flirted with the idea of going into finance. I was kind of interested in— I mean, generally, I would say I've been very interested in areas where mathematical modeling and computational methods are viewed as important. And this is when we get back to kind of this founding of Unlearn. But like the whole series of like being in biophysics, those all those years of doing biophysics. And then I did subsequent postdocs at BU and at University of Paris. It's difficult to be a biophysicist if you think that if your fundamental belief, which is mine, Okay, which is that mathematics and computation should be the foundation for biology. That is fundamentally what I believe. But that is a fringe belief amongst people who work on biology. The vast majority of people do not— the vast majority of biologists do not like math. They do not like computers. They don't want to have either of those things anywhere near them. And they don't think that they're important. And so it's actually very frustrating. So I always say that I am a frustration-driven founder, that Unlearn is an expression of my rage at how— at really the inability to get sort of modern mathematical and computational methods adopted within biological science. That's really fundamentally how I would describe it. And so that whole period of time of being in grad school I was thinking, I flirted with going to finance because I was like, oh, in finance they at least understand that computers are useful. They understand that math is useful. And so I have always had this kind of love-hate relationship with working in biology because I think it's, to a degree, it's the most important set of problems that we can work on, our problems related to medicine and human health.

One thing to back it up though, it was just like, Yeah, you love physics.

Okay, yeah, you love math.

How do you get to biology? What was the spark that got you like hooked? For me at least, you know, I took a seminar of Guido Guidotti and he kind of— I got— he told me these stories about biology, about like the discovery of like RNA polymerase, and that kind of got me hooked. For you, what was the spark that said, hey, biology is cool, I want to work here despite being frustrated?

That's not the spark. Yeah, I mean, that's almost not— it's not the way I would view it. It's not like I love biology. I don't. I think it's important. I don't necessarily find it personally interesting. I think it's important. I think that human health is important and medicine is important. And I think it's unfortunate that those sciences are not founded on— not built on a foundation of math and computation. I think that— so I think that creating a mathematical computational foundation of medicine is one of the most important problems that we have in this century. And whether or not I find it interesting to be working on biology is irrelevant because I think I find it important.

Interesting. OK, so you're at Harvard. You're in the biophysics program. That's a pretty sweet grad program. It's very small, intimate versus some of these bigger— My old mentor, Gary Ruffkin, was in the biophysics department for grad school and he recommended it. So you're in grad school in biophysics, probably having a good time doing research. Why didn't you just jump into finance? I'm assuming this is around 2010-ish, early, or maybe before that.

2012. Yeah, 2012.

So like, it wasn't a bad time to do finance.

Like, what a great time.

What made you—

yeah, it wasn't a great time.

It wasn't a great time, but it wasn't like— or I had around that time, I had a friend join Steve Cohen's fund. Like, she was a physicist and she joined SAC. So it was a—

I mean, things were recovered a little bit, starting to recover from the financial crisis, but they definitely weren't back yet. No, I mean, I don't know. I had this kind of love-hate relationship again with biology and biophysics. And I was kind of— I flirted a little bit with the idea of going into finance. But I didn't. I ended up going to work at— doing a postdoc at Boston, which I called a postpone-doc. Postdoc, I call it a postpone-doc because you postpone having to make a decision about what you're really going to do with your life. You just kind of— because you do a postdoc, you could still go into industry. But you can also stay in academia. But to a degree, once you go into industry, you've made that choice. It's very difficult to go back into academia once you've decided to make the jump. So you can do a postponed doc and you can continue to think about it. That's kind of the way that I thought about it. So I moved just across the river to Boston University in the physics department there working with Pankaj Mehta. And I loved it. That postdoc at BU was— like two of the most fun, interesting years for me. I really, really enjoyed being at BU. Cool.

I mean, okay, so you're— how did you— like, but when you're in academia, did you want to be a professor or— Yeah. Okay. You were like actively saying, hey, I'm doing research to become a professor.

Yeah.

Versus like, I'm just trying to like hang around. Like, I'm just trying to think about motivation. Like, how do you keep your— how do you keep like your, your like levels of motivation high?

Well, I mean, again, like, I'm motivated. I describe myself as a Darth Vader-like character. I'm motivated by the dark side of the force. All of the frustrations and all of the difficulties and all of the things that— those make me try harder. That's why I say I'm not motivated by interest. I'm motivated by working on problems that are really hard that other people— that people haven't solved. And frankly, I'm actually motivated by the fact that some significant part of the reason they're not solved is not technical. But social. I think a big part of the reason why we don't have like math— people are like, oh, biology is so complex, it's difficult to do compute. I don't think that that's really true. I think that it's mostly social, that the majority of people who work on biology don't do it this way and therefore, and resist it when you try to do it, they resist it. And so there's all these fringe— you could see it in physics. There's these fringe physics journals where people just publish stuff on other people's fields. But physicists together. There are— because they can't— they can't publish them in the main— in the main journals of the field. So anyway, yeah, so I originally was thinking, you know, if you think about sort of like at some point in my life, I decided that my life's mission was to try to mathematicize biology, that I was going to turn biology into— I was going to turn biology into a mathematical science, or I was going to die trying. But the question is, what's the pathway? What's the right pathway to do that? And so for the first part of my career, I was thinking that academia was going to be a pathway to do that, that I would go and I would become a professor, and I would work on research for mathematical biology, things like that. And so I pursued that for a while, did the postdoc at BU. And then moved to France to do a second postdoc at UNS. The France— that was basically like, I want to live abroad.

That sounds pretty sweet, to be honest. Yeah.

And so, but all of that was just like, you know, that was all the first attempt. Really what I would say that I discovered is I didn't feel— I ended up deciding to leave academia. I didn't feel like that was going to be a viable pathway for having the kind of influence that I wanted to have. You know, it's difficult to get a lot of resources. And ultimately academia is just people arguing about stuff. And that's what I found is like people, just a bunch of people, they all just argue about stuff. And I really wanted to, I don't know, I found that I didn't think it was the right pathway.

It makes a lot of sense. Yeah, I think, okay, so you're in France kind of realizing, okay, I'm not going to achieve my dream. Here? How did you— that's, that's pretty, that's pretty mind-blowing, to be honest, because like you spent all this, you spent like what, effectively almost 10 years, maybe like 8, like working towards something, and then you realize this is— this path I took is not going to get me to the end goal. How did you like reorientate yourself and then decide, hey, I'm gonna go into industry, join Pfizer? Was it just as simple as saying, how am I gonna get another job, or did you have to like reflect on the direction you're taking, or was it just kind of just, oh, I'm gonna do something else, and it wasn't that big of a deal at the time?

Yeah, you know, I, I wouldn't say it was a huge deal.

Yeah.

Um, I think one thing that's weird about being in academia is, um, you know a lot about what academic life is like, and they kind of teach you this. You're kind of being groomed to be a professor. Yeah. But you know nothing about what happens in industry. They even call it industry, like it's like all the same, right? So like if I had gone to finance or if I'd gone to work at some biotech, everyone would be like, he went to industry. Those are two completely different things, right? And so I didn't really know what it was going to be like, and I was like, I might as well try it. You know, there's no, there's no other way to really know. And a colleague of mine, Austin Huang, who we went to grad school together, he had had been working at Pfizer for a couple years, so he kind of convinced me to make the jump from academia and go move into Pfizer.

Okay, so Pfizer is definitely like the big behemoth, and when you look back at your experiences as a founder of Unlearn and CEO, do you feel that like joining Pfizer kind of set you on a certain direction or maybe imprinted you with certain values of business versus maybe say you joined some startup or a hedge fund maybe and then got your way to unlearn. Do you think Pfizer was kind of a— because at the time Pfizer also, I don't know if that was at the time or a little bit before, was a pioneer in new clinical trial formats. They had Craig Lipset there for a while.

Yeah. Yeah. Was that kind of—

were you exposed to ideas at Pfizer that you still hold onto and learn, or was it just kind of a job and kind of your first step on your journey to like ultimately get to Unlearn?

I learned a lot. Yeah. At Pfizer. I definitely learned a lot. I don't think that I could have been successful in starting Unlearn in what we currently do had I not had that experience. It also fed my frustration meter. Working at Pfizer was not— I didn't love it. So I learned a lot. But again, if you're a computational or mathematical scientist and you're going to work at a big pharma company, you're working at a biotech company, you're ostracized.

Yeah.

Right. You are a second-class citizen within that organization. And it's interesting because they're all talking about like, oh, we're going to build a data science team. And they say— I've written on so many things where people are like, data science, machine learning, AI, bioinformatics, statistics, it's all just the same thing. And it's like, no, none of those things are the same thing. Not to me. Every one of those words means something different. And people spent decades of their lives learning this skill. And then you go into these areas in biotech and they just like crap all over you and be like, your skills don't matter. Your area of expertise doesn't matter. People would ask me to like fix their computer. Some guy, I remember, he asked me to fix his computer. He brought his laptop over to me and was like, do you know how to fix this computer? I was like, I'm a machine learning scientist. No, I don't know how to fix your— first of all, I don't know how to fix your computer. But, right? Yeah, so it was a really, really interesting experience. And not to— I mean, in addition to working at Pfizer, I had interviewed at a number of biotech companies at that point, too, smaller biotech companies. I asked them, like, a big part of my interview questions were, well, how— how important do you think computational methods are to your area or to this company? And the answer was always, well, not that important. It's a support role. Not that important. It's a support role. And again, this is the exact attitude that my life's mission is to defeat. That's the way I would describe it. So I only sat at Pfizer for about a year. And it was very, very impactful. And I learned a lot. But I couldn't have stayed there any longer.

That makes a lot of sense. Me either. So you joined Leap Motion.

Yep.

Totally out of left field. Just like looking to hear the story. Yeah. Like, how did you get to Leap of all places?

And then, yeah, so I had a sort of a moderate— I talked about this frustration, and I was trying to work on the hard problems, and I've always had this love-hate relationship with biology. About this time in my life, had a bit of an existential crisis.

Oh, here it comes.

Whether or not I really wanted to stay in biology. Yeah. So it's interesting. I mean, not everyone knows the story. Some people do, because I've actually tweeted about it. But so a couple of things happened. One thing is that I actually accepted a job at the CIA. So that actually happened. So I had actually gone to an interview to be a— computational scientist at the CIA. And I accepted that job. But there's a long process of going— getting security clearance, like years. Okay. It was like— so it started— I was at Pfizer at some point when I had interviewed and accepted the job. And it was going to be a couple of years before I actually started it. So I had gotten connected previously with people at Leap Motion. And they offered me a job. So I moved out to San Francisco to try it. And my expectation was that I was gonna be at Leap Motion for like maybe like a year or a year and a half. And then we moved to DC to go work at the CIA. And then Donald Trump became president. And so I called up the CIA and I was like, I'm not gonna take this job anymore. So that's kind of how that happened. So then I was at Leap Motion. Leap's an interesting place. I also didn't work there very long, only a few months. I think that for me, it was very, very different coming from Pfizer, very, very different coming from academia, to go from academia to then one of the world's largest companies, then work at a really tiny company. Company. And that was very— so it was really eye-opening, I think, culturally and in terms of like what expectations would be like at a startup. But in the end, I wasn't particularly interested in virtual reality or augmented reality. It's not something that I'm passionate about. It was— I wanted to go to work at Leap Motion because I thought they were working on interesting engineering problems. Which they were. But, you know, when it comes to that particular industry, I just don't know anything about it. I don't play video games. I don't do anything like that. So I only stayed there a few months before, you know, thinking that, you know, maybe the right pathway instead of, you know, trying to go— if I really wanted to make change, maybe the right pathway would start my own thing. Cool.

Okay, so you're at Leap. What was the kind of the seed to say, hey, I can start my own company. Like, what was the moment, you know, like, maybe go back. Did you have some side hustle as a kid and in college? Like, did you ever make money or never?

No, no, no, I don't care about money.

Okay, so you never—

I mean, that's not exactly true. Like, like, obviously I want to have, you know, like, I, like, I want to be able to be financially secure. Yeah, but, you know, I have no desire to be like Bill Gates rich or something like that. Like, I don't know. That's not what I care about. And again, it's— I'm not like a vision-driven founder who's like this positive vision for the world. It's sort of like I am a frustration-driven founder that I am angry at the system. And therefore, this startup is a way for me to fight the system.

Okay, cool. So, okay, you're at Leap, between Leap and Unlearn. What was the process to start the company? Did you like listen to a bunch of proms and decide this is something I want to do, talk to a bunch of friends and get ideas, or do you always have this idea in the back of your head?

So no, I left Leap. There was a few months, maybe like 3 or 4 months between when I left Leap and when I started Unlearn. And so my co-founders were also all at Leap Motion, John and Aaron. We all kind of left around the same time. And I didn't leave saying like, oh, I'm going to start a company. I left saying, I don't know what I'm going to do next. But I needed to take— that's like I said, I was going through this existential crisis of being like biology or not, or like, what should I do? And so I just was like, I'm going to take a little time and just think about it. And basically myself, my co-founders, one of our advisors, Cami, we would just kind of meet up once a week in a coffee shop and just talk about things that we could do. And so this would have been in 2017. And what we were basically saying in 2017 was that no one was really working on research, sort of generative models. There weren't generative model startups at all. And generative modeling It was sort of an interesting area at the time with GANs had come out a couple of years earlier. But there weren't a lot of use cases for those things that people had figured out yet. And so the original thesis for the company wasn't even necessarily biology-focused at all. It was, we're going to do generative modeling. And we're going to figure out— we're going to have an AI company that's focused on generative models. And we are just going— so our first pitch is we went and we raised capital.

That was the first pitch, just like, yeah, yeah, first pitch was generative AI.

Generative AI is going to be a thing. 2017, that was the pitch to DCVC in 2017. We were like, in what field, what vertical, what the business model? We were— we listed out a few different— here's a few ways we think that generative AI could be used for. And we were like, what we're going to do is we're going to build this generative AI and then we are going to go try it on those things and we'll choose one. That was the first pitch.

Well, shout out to, I think, Matt or Zach and James and the whole crew at DCVC for betting on you. And so I remember reading the preprint in 2018 that you did the kind of proof of concept in Alzheimer's. So between starting the company and that preprint, you guys got to work. So how did you hone in on synthetic controls? Was it like— maybe you can describe what Unlearn does first and foremost, and then we can dig in deep in terms of how do you go through the idea maze of clinical trials and solving the control arm of it?

And yeah, yeah. I mean, so we made a mistake. So I'll start. I'll tell you, I'll tell you, I'll tell you, I'll tell you the way that it kind of came about. And you'll understand what you'll see it once I go through the story. So when I was at Pfizer, I had done a lot of work. Most of my work at Pfizer was kind of on two different things. Half of it was human microbiome stuff. And half of it was on clinical trial, machine learning in clinical trials. So I've gotten exposed to the problems with clinical trials and how people were thinking about applying machine learning to them. And so I had these thoughts that we would be able to— one of the possibilities for using generative AI was to create generative models of people. And then, basically saying, well, we can then create synthetic data and we could use these synthetic data simulated people in clinical trials. Now, the challenge there is that every clinical trial has two arms, right? So, because the clinical trial is just a comparison. You're comparing a brand new experimental treatment to a treatment that currently exists. And you want to know, is it better or worse than what's already available? And so if you just think about the data availability for that problem, you have two pieces to it. You have a new experimental treatment. Well, how much data is available on it? Maybe none, because it's never been tested in people before at all, right? So you have this one part where it's either none or very little. And then you look at the other part, which is, well, what about this currently available treatment that's been on the market and been given to probably tens of thousands, maybe hundreds of thousands or millions of people? Well, it's a ton of data, right? It's been given to tens of thousands of people. So if we just thought about it from a data availability part, we were like, oh, you can't do much. Machine— trying to apply machine learning to these new experimental drugs, to that part of the equation, it probably isn't going to work because there's no data. There's a ton of data on the control group. So why don't we set up— we set out to say, well, let's create these models where we'll train generative models. On data from previous control groups, and then we'll be able to generate new data that looks like it's from the control group in the new trial. And this is where we made a mistake, made a really big mistake. Thankfully, we fixed the mistake, but we made a mistake. We thought that a synthetic control group was a good idea. We had thought this. At the time, there were a bunch of other companies and people that were starting. Synthetic control groups was like a thing that people were talking about. And so what— first of all, when people say synthetic control group in biotech and medicine, they don't mean what I mean. What everyone means when they— the phrase synthetic control group has become synonymous with what, what is called propensity score matched external controls. So an external control group means that I'm going to take a clinical trial, I'm going to enroll, I'm going to run a clinical trial, I'm going to enroll in a bunch of patients, I'm going to give all of them my new experimental treatment. And I'm going to compare their outcomes to some other group of people. This could be like a previously run clinical trial. It could be just people going to the doctor, whatever, other group of people. And what people say, synthetic control group, when they say that, they mean they're going to take another group of people, real people, not synthetic, real people. And rather than using all of them, They're going to take a subset. That's it. That's all that they need to be totally clear. Synthetic control group means I have an external group of people I'm going to compare to, and I'm going to take a subset of those people. That's it. So, okay. But we were like, why do that? Why take this subset of people? Why don't we create computer simulations of people? And we can use these computer simulated actual synthetic data the way computer scientists mean synthetic data. But so the problem with any clinical trial with an external control group is that how do you know when you look at the end of your clinical trial, how do you know if you see a difference between the people you gave the drug to, your new experimental drug, and these other people you're comparing to? How do you know that difference is because of the drug? You don't. You don't. And there's no way to know. Any clinical trial with a synthetic control arm requires unverifiable assumptions. You just have to believe that it worked. That's the only way to know it worked, is that you just believe it did. There's no way to verify that the assumptions were correct. There's no anything. So they don't work. It doesn't work. And it doesn't matter. If you think about it, like suppose we have a model. We're creating a synthetic model that's simulating control groups. What happens if the model is wrong? And I create a synthetic control group. I simulate my control group for trial. If I compare this to my treatment, I see a difference. What happens if the model was wrong? What if the model made some errors?

Yeah, you have a lot of false positives and stuff.

Yeah, yeah. So now your clinical trial, you're going to incorrectly estimate the effectiveness of your drug. How would you know if it made errors? You wouldn't. Because that's the point. You have no way of knowing. And so this idea of a single-arm clinical trial in which you either use propensity score matching or if you use true synthetic data where you generate the data from a model, they just don't work. I mean, that's just fundamentally the situation. They don't work. They can never work. They won't work ever. There is just some fundamental problem with them that makes them not work. You need randomization. You need randomization as part of your clinical trial to be able to interpret it as causally. And so the FDA is actually pretty upfront about this. They're just like, external control groups generally don't work. Like, that's what the guidance says. It says don't do them. So anyway, so we had made this mistake. We were going after creating external control groups. With computer simulations, but external control groups don't work in the regulatory context. So this is kind of where we were around 2018, 2018-ish, beginning of 2019. But at this point, this is where we realized something interesting. So when you do a regular synthetic control, you take real patient data and you just treat it as if these came into your trial. You have no way to tell if that was a good or bad thing to do. But we're creating predictive models. So what if instead of replacing the entire control arm, we just make the control arm smaller? And now what we can do is we say, okay, well, for all of these patients in the clinical trial who actually were randomly assigned to receive the control, we are predicting what would happen if they were randomly assigned to receive the control. So now we have an internal reference point. We can tell how good our model is doing. We can measure if it's over or underestimating rates of progression in the control group, and we can fix it. We can recalibrate the model because there are still people who were randomly assigned to receive the control group. So we came to this completely different point of view. And what we realized is, we shouldn't be trying to replace randomized controlled trials. We should be making the next generation of randomized controlled trials. Now we want to reimagine the randomized controlled trial for the 21st century, a computationally sort of motivated RCT. So now what we do is when we work in RCTs where we are shrinking the size of the control group, but the way we kind of think about it is, We're going to say, OK, we're still running an RCT. You're randomly assigning people to treatment or control. We're creating what we call digital twins of all of these people. And then we are going to make the control group as small as we can while still being able to guarantee that that clinical trial produces unbiased estimates without making any assumptions about how our models work. So it's very interesting. That we have, which I think is incredibly rare. If you just think about the whole AI community and all of the safety arguments and all these, everyone is concerned about hallucinations. We hear about this constantly. Oh, the AI is hallucinating. The AI might make up stuff. And our whole argument is, who cares? Who cares? What really ultimately matters is that you're using the AI to answer a question. Do you get the right answer to your question in the end? The AI is just a tool that you're using to answer this question. In our case, which is How effective is this drug? And if we can guarantee that you get the right answer to that question every time, then it shouldn't matter what kind of AI we're using or anything. We should be able to guarantee it. So this was a huge insight that we had for the company. First of all, just the idea that we could do this and then figuring out how to do it and then going to the regulators and actually getting them to sign off on that this is a good idea.

So, okay, making that transition from external arms to external controls to kind of smaller trials, like when you think about, you know, maybe on this element of like AI hallucinations as well, at least for trials right now, do you need like, maybe to like explain a little further, you know, for an Alzheimer's trial, I'm recruiting patients and I want to treat them with a drug and then have a placebo arm, and I work from learn, and you guys will create a subset of—

the way it works is that every— we don't get to— because the trials have to be blinded. Yeah, right. So patient enrolls in the trial, so we'll have already done all this machine learning work up front. So we have this model that creates simulations of patients with Alzheimer's. And what will happen is a new patient enrolls in your study and you collect a whole bunch of data on that person at the very first visit. In this first visit, you haven't given them whatever treatment yet. You're just collecting this information. All of those data we get, that gets— I view that as like we have this pre-trained model. You're going to take the information from your patients. You're going to provide that to the model as a prompt, sort of, right? So you got to tell the model, here's what I know about this patient. And then the model will be like, okay, well then here is what would happen to that patient if they were to get the control. And we're going to do this for every single patient that enrolls in the trial. So it doesn't matter if that patient gets randomly assigned to the treatment group or randomly assigned to the control group. So if you're running a clinical trial, you have 500 patients in treatment, 500 patients in control, there would be 1,000 digital twins. Every— you get a digital twin of each person. So Steve enrolls, there's a digital twin of Steve. Alice enrolls, digital twin of Alice. Digital twins have to be of something. They're digital twins of the people in the trial. And so for each one of those people, we're predicting what would happen if they got the control. That's pretty much the way that works. What would happen to this person if they got the control? There's a whole bunch of interesting stuff that you can do once you realize I'm making these individual predictions. Of what happens if a person gets control. So the first thing is this— what we just talked about is that, well, for a patient who received the treatment, the experimental treatment, if I just subtract what was predicted if they got the control, that's an estimate of the treatment effect right there. And I can average that. But if I do that for the patient in the control group, now I'm measuring the bias in my model. And so now I have this ability to sort of after the fact correct. For correct for that. So it's not exactly after the fact. I mean, there's a way that we do the analysis that allows us to sort of come up with an unbiased estimate for the treatment effect from these studies. But there's all kinds of other interesting stuff that we're interested in work— that we want to work on. Our models are probabilistic, for example. So for each patient, we're not saying that, like, say you have Alzheimer's, you have a change in some measure of cognition. We're not looking at to say this will change by 3.1675 points over the course of this 12-month trial or whatever. We describe a distribution, a whole probability distribution for what the change in that score will be. And now you can actually think about computing like individual treatment effects from that kind of data. So we can look not only at how drugs work on average, but now we can actually start to dig into be like, well, how are these drugs working for individual people who are in the clinical trial? It's a very different way of thinking. As you move sort of from that population level down to the individuals, you have more and more reliance on the model. So at the population level, we're going to get the right answer even if our model is making mistakes. But as you go down to the individual level, the models need to be more accurate to interpret it. But yeah.

We can talk about what it takes to make those models, to pre-train models, and especially in drug development. I think you alluded to this. Is there some sort of like lower threshold of diseases you can go after? Like maybe a lot of rare diseases where there's a paucity of data. And also, how did you, at least the early days, learn, learn, how did you get access to patient data? EHR stuff, and that's really imprecise. How did you talk to pharma companies? How did you, how did you get access to data in the first place, which is often a barrier for a lot of these companies?

It's not actually that hard in a lot of ways to get access to the kinds of data that we want. We use anonymized data, mostly from previous clinical trials, but also from disease registries. We don't just use EMR data for a couple of reasons. It's really crappy. So that's one reason. EMR data is like the— anyone who's worked on any machine learning problem, you give them some EMR data and they're like, what is this stuff? Why did you make this data? It looks useless. So it's crappy. It's not consented, right? The fact is that if my EMR data are being sold to people, they probably are being de-identified and sold. I didn't consent to that, not on purpose, but that's just kind of the business model of some of these EMR companies. Whereas, so we focus on research data, research quality data that was collected for research. So there's a couple of things there. We say, okay, well, these data were collected for research in the first place. They're consented. The patients consented to take part in the research because they want someone to use their data to help cure their disease. Right. So we're starting already from that. We're working on consented research, consented data. But also researchers have designed the study. They have decided what to measure, how frequently to measure it, and they put a whole bunch of resources typically into doing it. Right. Way higher quality data when you look at research-grade data. So we made this initial sort of thought that we were going to go after quality data instead of quantity data. And that's been most of our work. This is true for probably any machine learning company. 80% of our work is data cleaning. That's right. Like 80% of what we do on a daily basis is that we take data They're usually from all these smaller studies. We aggregate them together and then we do a huge amount of effort to quality control to get those data into a format where we want our models to be able to learn from that.

That seems to be a pretty big decision. I think most startups would try to get access to EMR data and—

They shouldn't. They should— yeah, that's dumb.

Versus research studies. How did you think about So for high-quality data, what were the, like, the, the, how did you go about like curating them? Because a lot of that is kind of opaque. Was it like, how did you go about at least early days, like finding high-quality datasets and like, like, was it all open access or did you have to negotiate pricing? I have a few startups I work with and they have, you know, there's a lot of variability. Sometimes you have to pay for the data. Sometimes you don't even know how high quality the data is until you pay for it and get it. Curation seems to be your really, really hard skill.

I mean, the initial thing that we focused on was data from clinical trials. Yeah. And data from clinical trials, you have— it's still not great compared to other kinds of data and machine learning that you encounter, but it's basically the most high quality data you can get in medicine. So we said, well, we're going to start with data from clinical trials themselves. And there's a lot of those data are public. Some of them you can buy. So it's kind of a mixture. The public data are interesting because the government sponsors a huge number of clinical trials. The NIH pays for a huge number of clinical trials. And they are required to make data available. But there is no system for people to do it. So we have a team of people who just call up professors. And they're like, hey, you ran this clinical trial. Is there some way that we can facilitate your sharing of those data? Because they have to be anonymized and they have to be put into various standards and so forth. But so yeah, I mean, that was the initial thought. It's like we were like, we're going to start with clinical trial data quality. I mean, it was just the fundamental philosophy that data quality is more important than data quantity. That is, I would say that's just like a fundamental belief about machine learning that we hold. Yeah, I still believe that. I would, I would, I would, we reject a lot of data. We reject a huge amount of data is kind of what I would say. Interesting.

So when you think about this progression from, you know, like fixing a mistake early on, kind of maybe slightly tilting your product offering or value prop in a different direction, How did you end up going towards certain diseases like Alzheimer's? I think you have potential like in lupus and RA and stuff like that, more chronic diseases. And how did you think about, how did you like, what was the, maybe a better question, what was the sales pitch to pharma or whoever your customer is? Like, to your customer, what's unlearned sales pitch?

And why is it like— What was it or what is it?

Like, what is it? Like, you know, like when you go to Merck, it's like the sales pitch is, just run a smaller trial, more efficient trial. But then this next question would be like, for what though? Like, for what diseases would this be really, really relevant? Well, for what diseases could they have confidence that this could work for? And it seems like Alloren has kind of, you know, you're focused on nerve generation and then maybe autoimmunity inflammatory diseases after.

Okay, so we want to work on everything. Yeah, every clinical trial should be run this way. But we currently, we build models for individual indications. So we have model for Alzheimer's, model for multiple sclerosis, model for ALS, et cetera. And we're sort of doing indication by indication. We are interested in a long-term research program, which we have, which we're thinking about and working on for one model for everything. But that's harder. Yeah. So right now we have separate models that we would call experts, expert models for these different indications. And the pitch is really very simple. If you're a pharma company, every single additional week or month that your clinical trials take is one fewer week or month that you have upon patent sales. Yeah. So that's a huge thing. And for our work, we're basically saying, particularly at phase 3 clinical trials, uh, we'll be like, look, you're gonna run this massive phase 3 clinical trial, it's gonna take forever. Yeah. Um, and, uh, you're enrolling, I don't know, 5 patients a month, let's say. Uh, so if, if you work with us, you will need 100 fewer patients in your— in the control group of your trial, 100 fewer patients. And what that's going to mean is you're going to save had 5 patients per month, you could say 20 months. Yeah, 20 months. Maybe it's 10, maybe it's half that, maybe it's 10 months. Okay, that's like a, that's a huge, that's hundreds of millions of dollars, billions of dollars of Onpad sales, exclusive sales for that company. In addition, so it's great for the pharma company. So the pitch of the pharma company is like, well, that value proposition sounds great, right? And not only that, we're like, we can do it in a way that's It's totally acceptable to the regulators. Don't take our word for it. Here's this regulatory qualification we spent 2 years getting from the European Medicines Agency, right? So I think we are the first and only company, as far as I know, to have any AI, any qualification for a method to accelerate clinical trials with AI that I'm aware of. So again, the value prop to the pharma company is obvious. They get time savings off the clinical trial and they get to do it in a way that works. And they don't have to take our word for it. The regulators have already come out and been like, here's what we agree that this works. Here's why. The other part of it, of course, is that that faster clinical trial will be way better for patients because downstream you're getting a medicine. If the medicine works and we actually have nothing to do whether the medicine works or doesn't, we got nothing to do with that. We just help find that answer. Faster, right? But if it does work, that means that, well, that's 10 months faster people can get access to a treatment that could help them, right? And for the participants, the people who volunteer to take part in the clinical trial, most of those people are volunteering because they're hoping that they can get access to an experimental treatment. And we shift that balance. More patients in a clinical trial when you work with us get access to the experimental treatment. Not all, not all, but more, more than in a normal clinical trial. So that's like we have— it's an interesting point where what we're providing is a win-win for everybody. It's better for the pharmaceutical company, it's better for downstream patients, and it's better for participants. And we do it in a way where you don't have to interpret the results any differently. If you're a doctor and you see a paper that comes out with a single-arm trial or synthetic control arm, you're like, ah, how do I interpret this paper? What does it mean? If you are a doctor and you see a paper comes out from one of our twin RCTs, it's just an RCT. You get a p-value. You interpret it the exact same way that you would any other clinical trial. It's exactly the same statistical analysis. So you don't have— it's not different. So this is— it's just fundamentally an interest. It's a really, really, really interesting thing that these methods will allow us to just make clinical trials better. All of them, all clinical trials better, better, faster, more aligned with what patients want.

Okay, so maybe we touched on the— in September of last year, the EMA, the European Medicines Agency, kind of gave a framework to use your digital twin product for Phase 2, Phase 3 trials. What was it? What was the legwork to get that? I know a lot of conversations and work to get that done. How did you— you're the first to do this. So what was the— maybe any anecdotes of how to have success? What was the bar to get the EMA to accept your use of your software for RCTs?

Sure. I mean, if you have a methodology that you're hoping that the rate— suppose you have a problem and you're trying to develop a methodology. So the first thing that I would say people should think about is you gotta consider the regulations before you start working on your product. The regulations define product requirements. You need to be very familiar with what they are, and you need to build the product to the specifications of the regulators. That's fundamentally the thing. If you think, if you're gonna wait, you're gonna build the product you want, you're going to bring it to the regulators, they're going to reject it, probably. You need to think about what they want. So like I said, for us, we started off with this single-arm trial thing, as a lot of people did. And like I said, it was a mistake. We went and we talked to the FDA and we heard what they were thinking and we were familiarizing ourselves with the regulations and we were like, ah, that's not going to work. They don't want this. Right? So we said, okay, well, what do they want? We're engineers, we build stuff. Let's go figure out what they want and then we will build that thing. So the way that, you know, honestly, it was a long process. It took a long time to get through regulatory review in Europe. It was maybe 18 months, something like that, from start to finish, not including the pre-work time because we had to write the briefing documents and all that stuff. But it was smooth.

Cool.

It was smooth. Well, for one thing, the regulators were really quite sophisticated. We were really happy. First meeting with the regulators, they were asking great questions. They clearly understood what we were doing. It was on point. They were quite sophisticated, smart, and the whole process was smooth. We never got any questions where we were like, this is a weird question or dumb question. Everything was good. And I think that's in part, they even actually at one point, someone even said this to us, one of the people from the EMA was like, you all crafted this document, the scope of what we were asking them to look at really well, right? We were aware, we've read the regulations and we were like, we know where the gray areas are. So we're just gonna avoid all those gray areas. We're going to go for— this is the meat and potatoes thing that we know is acceptable. Let's put that and go get that reviewed. And so we walked in there confident. When we submitted it, we were like, it's 100% chance that they're going to accept it.

So that kind of touched upon something you defined about LEARN as you're a tech company, from just pivoting slightly to solving problems quickly and not maybe, you know, on a 10-year timeline, you're trying to get things done in months or at least a few years. What was the conviction that characterized Build and Learn as a tech company, not as a biotech company? And what has been the kind of the learning, so to speak, of building a software tech company at the interface of biotech and all this regulatory stuff that is in an industry? That kind of maybe like, yeah, that has it. Maybe a better question. How does it build a biotech tech company surrounded by slower moving biotech companies? Yeah. How do you—

how do you— yeah. I mean, you know, what I would say is my whole— what I described at the beginning, it's like my life's mission is to kind of mathematicize and build mathematics and computation as a foundation for biology. And I think that's the most important problem of the century.

Yeah.

So, okay. And I think AI is uniquely— it's like we, for a long time, we were like, oh, biology is too complex. It's too complex to mathematize.

Okay.

But now we have a whole new area of math called AI. And it turns out that this particular new area of math and computational science is a Perfect for a really complicated thing. That's what it does, right? So it's really— this is the perfect time to go after it. But how would you do that with a biotech company? Like, if you just think about it, who works at biotech companies? Biologists. That's who works at biotech companies, right? And are they going to want to— That's the whole point is that culturally, they traditionally don't want computers and math to be the foundation of the field. So fundamentally, the mission is incompatible with being a biotech company. It's just incompatible. We are an AI-first company. Unlearn is an AI company. As I said, we were founded as an AI company. Yeah, right. Generative modeling. That was, that's what we were founded on. And so there was never any, it's not even like a decision was made to be a tech company versus a biotech company. That question was never asked. We were always a tech company. The idea that we might want to be a biotech company was never even in the picture. Interesting.

So like, how does that manifest in terms of like hiring? And in terms of like, how do you think about like setting milestones? You know, sometimes biotech companies, you know, you're on like a 6-month timeline sometimes or multi-year timeline. For Unlearn, are your timelines more compressed or can you just do more things all at once? And then how do you think about hiring talent in terms of culturally? You know, most biotech companies do hire PhD biologists. Things I've learned trying to source talent from maybe different sources.

Sure. So I mean, on the speed, you know, we're running dozens of experiments a day. Yeah. Right. I mean, in a computer, you press the button and it runs, you wait until it's over, and then you look at it. Right. And so, you know, we're running dozens of experiments every day. I think that the cadence of product launches or product updates is, you know, not daily, you know, but you think about what's the cadence for new drug updates?

Yeah, never.

Decades, right? It's decades. But we're talking about, you know, we want to release, you know, so we're releasing a new version of our AD model this quarter. So just a couple of weeks, we expect to be releasing it. We expect to release another version, another update next quarter, right? So like we're trying to get into this like orderly update cadence, whereas you get new data, you have new methods, you apply them, you update. And so things can just continuously get better over time. I think that concept, that's going to be something that's going to be really interesting for all kinds of areas within sort of tech bio and the intersection over the next few years. For us, the fact that the use case is what we're focused on, from a regulatory standpoint. But we're able to actually continuously update our models in a lot of ways, as long as we're using the outputs through these kind of regulatory qualified frameworks. But that's not the case, like if you add a digital therapeutic or something, there is still this question about, do you just do over-the-air software updates? We don't have to worry about that, I guess what I'm saying. So we get to move faster because we have kind of the separation of concerns where regulatory is really on the clinical trial design and how you use patients' digital twins, not on the AI and where the digital twins come from. And then in terms of who we hire, the company, it's a mixture. Yeah. But the majority of the company are scientists and engineers. A lot of the background is math, physics, computer science. I mean, this is kind of always one of the things is for us, we want to be the best AI company in healthcare. And not just an applied AI company, but we do a huge amount of AI research. Actually, I am 100% certain that nobody is using the kinds of machine learning models that we use. 100%. Nobody in the world except us uses what we do. And that was the whole point of it, is that generative models is really still a pretty nascent field. And almost all of the research is on images or text. Yeah. So we're working on something different. And so if you start to work, what you'll see in a lot of areas of machine learning is that the architecture is driven by the data you have, right? So you've got convolutional neural networks for images, you've got transformers for language, but we're working on something different. We're working on something different. So you end up with different architectures. And I think that's— so for us, when we're going out and we're hiring people, trying to compete against Google, right, for engineering talent. That's the pitch, right? It's like, hey, you can work at Google. You can have a huge amount of resources. You can sell ads, right? Or you can come work at OnLearn. You have not so many resources, but you will have a mission that will improve people's lives. And you're going to get to work on even more interesting machine learning research here than you would at Google. That's kind of our pitch.

Awesome. And maybe any final lessons? You've kind of talked about product development, things for whom regulation, but any final lessons for outsiders? I don't mean outside, mainly engineers or people who want to buck the trend of biotech but still want to work in medicine and healthcare and biology. Things you've picked up in terms of how to pitch kind of your company or even just exist in biotech as a frustrated person and trying to change things? Like, how do you exist in biotech but trying to have a more tech mentality? Like, what kind of— how do you do it and not get like completely frustrated?

That's really hard. Oh yeah, I do get frustrated. That's the whole point.

Yeah, exactly.

You have to learn to love the grind. That's fundamentally what it is. It's not going to be fun. It's not. We have this, uh, uh, me and my co-founders— I actually don't know where the hell this came from. I thought it was a Leap Motion thing, even this designation of different types of fun that you can have. And when most people talk about fun, they talk about what we call Type 1 fun. Type 1 fun is something that you do and you enjoy it while you're doing it. It's fun. But there's also Type 2 fun. Type 2 fun is you do a thing and it sucks. It's actually not fun at all while you're doing it. You don't enjoy yourself. But after the fact, you're glad you did it, right? You think about like going through a super duper hard workout or something. It's not that fun while you're doing it. You feel bad, you're tired. But after the fact, later on, you're like, oh, I'm really glad I did that. That was fun. Right? And you have to learn to love that second type of fun, to be a computational person within, within biology, because a huge amount of your daily interactions with the rest of the field will be incredibly frustrating. And that's just how it's going to be. And it's going to be like that for, for a long time. You're going to be— you are going to be looked down on You're going to be treated poorly. People are going to ridicule your work, but they don't understand what you do, but they're going to do all these things anyway. And you have to look— take all of that as fuel. That is my recommendation. And if you can't do that, you need to get out of bio. If you can't, if you don't like that, if that sounds bad to you, you leave biology. Because you are— it's not a friendly environment for people who are interested in math and computers.

Okay, I mean, but get at it though. What's important?

It's up to you. Yeah, I mean, you got to decide whether— you got to decide whether you're a warrior.

Yeah, we'll end it there. Okay, Charles, I— thanks for doing this. I think I'm really excited for Learn to announce all the partnerships you have and, you know, kind of see the results of the software we've built and lead to drug approvals and all these new medicines. So maybe a few years from now, get the update as you announce a lot of the success you've had, uh, because there's a lot of stuff we're not talking about that maybe 2 years from now, as you kind of reveal it to the world, uh, we can touch base on. Uh, but thanks for doing this. This is going to be very useful to a lot of people, especially people who are, uh, outsiders or frustrated. I'm going to think of the title here. How to be a warrior in biotech, something, something.

Yeah, I mean, it is, it is, it is, uh, like, it is interesting, uh, you know, it's, uh, it's 100% my mentality. Yeah, it's like I'm at war here. I wear camouflage every day. It's actually my whole, my whole closet is just filled with camouflage.

One quick question. I'm the only child, so I used to love history. I used to love reading about wars just because it was a hobby. It was like, I think I would have I was trying to think to keep my time up. Do you have a hobby like that as a kid?

Like, do you have like a little—

like, uh, do you like playing sports or do you like—

yeah, yeah, I'm a boxer.

You're a boxer?

Yes, yeah, I've been, I've been doing boxing. Well, I mean, I've been doing boxing and martial arts and wrestling and stuff like that since I was like a teenager. Uh, I grew up playing ice hockey. In Michigan. Okay, I moved from ice— I transitioned from ice hockey to boxing, to boxing at some point as a teenager. Yeah.

George Creed 3, just more personal.

I have not watched it.

No, it's pretty good actually. I, I, I like, I like a good Rocky movie, uh, when I, uh, in terms of boxing. But hey, Charles, thanks for doing this. I know you're super busy, but I really appreciate you for doing this.

Yeah, thanks, Josh.