Data Driving Experimentation: Merging Biology and Data Science

Welcome to the Axial podcast. Axial is an early-stage investment firm based in San Francisco. We partner with great founders and inventors investing in early-stage life science companies, often when they are no more than an idea. Axial is fanatical about helping the rare inventor who is compelled to build their own enduring business. Hey Jacob, how are you?

Good, how you doing, Josh?

Doing pretty well, just busy today, a lot of work. I'm on like— I'm on— I had to eat some lunch real quick actually. I have calls for the next like 6 hours, but uh, yeah, really excited to talk to you today and just, you know, just generally just catch up, to be honest. But let's do some housekeeping first.

Okay.

You can click on my profile and make me a moderator so I can kind of protect you from the trolls.

Done.

Cool. And then can you hear me well? I got some new headphones. I should make sure the audio is better. My buddy Sean let me know that the audio kind of stunk last time. Can you hear me?

Yeah, I can hear you well.

Okay, awesome.

Well, great.

Really excited to have Jacob Oppenheim here, the director of integrative data sciences at eCRx. You know, Jacob's a good friend of mine. He actually was introduced to me through Brian Naughton, who, you know, we actually spoke to last time. Uh, yeah, Jacob, I'm just really excited to just talk about your career and just talk about the intersection of data science and biology and in the context of your work at eCRx and also just in in general. And so, yeah, just kind of a really interesting conversation. If people in the audience have any questions that are relevant to the topic, feel free to raise your hand and we can bring you up. And that's what's nice about Clubhouse. But Jacob, maybe you want to do a brief introduction and we can go from there.

Yeah, that sounds great. So like a lot of people who do data science, I'm originally some kind of physicist. I got really interested in biology back when I was an undergraduate. Mostly through, through a class I took on evolutionary theory. And I realized that you could use math to actually predict lots of interesting things about biology. And I started wondering what I could do there. And I guess, you know, everything kind of grew from there. And I went to graduate school sort of in the teeth of the Great Recession and spent a lot of time thinking about how brains process sound, and also what the structure of blood vessel networks looks like. After that, I spent some time developing algorithms in genomics and then sort of slowly moved my way from the more academic parts of the biotech industry to the more practical ones and drug development. Happy to talk more about that. I don't know, Josh, how much you want to hear about my background here.

Yeah, it's mainly a lot of founders, a lot of aspiring founders, you know, listen into this and just more kind of setting the stage of how you got into biology and how you made that transition from academia to industry.

Yeah, yeah, yeah. So I can set the stage a bit about that. So like, I finished my PhD in 2014, and I didn't really know what I wanted to do. I just knew that academia wasn't for me. The scales over which you got work done and the sort of practical import of what you did didn't feel right. Kind of amusingly, I didn't really know anything about the biopharma industry, just that biotech sounded like a cool word and it sounded like the natural industry to go into. And so I worked for about a year and a half at this place called GNS Healthcare. So GNS was kind of a CRO for analytics, basically, if that makes sense. Essentially, a pharma company or a disease research foundation would have a failed clinical trial or a whole bunch of samples, and they'd measure a whole bunch of covariates, everything from DNA sequences to protein levels to phenotypes, and they want to understand causally what's going on there. And so, you know, this is kind of a causal machine learning problem. And so they hired lots of people like me who had relevant backgrounds to basically develop algorithms and apply them to this biopharma data. You know, this felt very academic. You know, we had journal clubs, we did lots, we read lots of papers, we published things occasionally. And so it was kind of an easy transition. To the world outside of academia. The problem with this, as I quickly figured out though, is that when somebody is paying you to analyze their data for them, means that they can't find a signal in it, which means that if you're going to go do something new with it, if you're going to try to figure out some novel method or apply it, you know, you're not really going to be in a great place to do that because there's no obvious signal in the data. And it's not like they're going to go pay you to, to generate data for them. They're just going to, if they like the results, they'll make some decisions off of it and they'll never call you again. And they don't like the results, they'll never call you again either way. So you never really learn what works. And I think this is actually a pretty huge problem and one of the biggest problems bringing machine learning and computational techniques into biopharma. It's just that biological data is complicated. There's a lot of cool stuff you can do with it, but it's frankly pretty darn hard to validate a method, and you only have a limited number of experiments. And so, you know, over the course of this year and a half, I had a lot of fun building algorithms, but I sort of started realizing more and more and more that if I wanted to really be somewhere, Couldn't be from this data machine learning first perspective, but I needed to go someplace that actually had a biological platform that could generate a lot of novel data. And then from that sort of platform, we could do interesting data science on it, right? And we'd probably start with some classical statistics. And then on top of those statistics, we would start building more and more interesting machine learning, applying state-of-the-art techniques and really seeing how they'd work in the real world. Does that sort of thing make sense?

Yeah, it makes a ton of sense. You only want to hire the data scientist if you have the data, and if you're in these services or analytics, you have no way to generate your own data, especially in biology. Maybe we can shift gears and then talk about when I got first introduced to you, you We were at Indigo, and then you were exploring something new, and I made a bunch of introductions, uh, and you ended up joining EQRX, which is a great decision. And from my perspective, uh, you might be top 3 most important person at that company. How many people— how many people are you managing now? Is it like 40 or something, or 30?

Uh, we're headed into the low 30s next year.

So, geez, Jacob, congrats. Uh, congrats to you on that promotion too. I mean, You deserve it.

Well, thank you. Thank you. Yeah, I mean, so I guess kind of where this journey that I've been telling, was talking about kind of ended up was that really I realized at some point that you have to build biopharma to be digital native from the beginning. And that's going to allow you to collect and use these datasets. And it's not just enough to have a biological platform. And it's not just enough to have people who say, okay, data's important. And it's not just enough to do a bunch of good experiments, but you really have to be able to capture those data in their appropriate context and set up a culture that's completely data-driven if you want to use this stuff effectively from the beginning. And so that's kind of what I came to EQRX to do. Now, EQRX, in short, we're a fast follower drug developer. We're dedicated to bringing affordable medicines the fruits of sort of the precision medicine and engineering of biology revolutions to everyone at an affordable price. Now, why can we do this? Well, we know that we're going after targets that are clinically validated, which means that we can go bring a lot of process engineering and novel techniques to the biopharma industry. And a lot of those fall under the broad umbrella of what you might call being digital native or data science or something like this. So it's kind of an ideal place to start out and say, okay, what if we brought the best practices from the technology industry into biopharma? What are the types of efficiencies that we could gain in the short run? And what are the types of cool problems that we can eventually solve in the long run? That everybody says they want to solve but don't really have the system set up to either solve them or really take advantage of the solutions of them, um, if they even had the systems to do that.

Absolutely. And then maybe go to the, like, the really exciting part of the conversation, you know, your work right now at eCareRx. And, you know, over the years you've really given me some awesome ideas on, um, the intersection between machine learning and biology, especially your idea around training models with unphysical experiments. And so at a high level, how do you and your team think about using data, one, generating it, aggregating it, and then using it to do everything across the board from developing a drug to finding the right market to executing the right clinical trial? Ecurix definitely has a really, really bold vision and then data like you're alluding to, is essential to achieving kind of this fast follower model at scale, you know, doing it beyond PD-1 essentially, or PD-L1.

Yeah, no, I agree with you exactly right. You know, you talk to somebody about eQRX and we really got lucky with the first couple drug candidates that we were able to in-license. There was a lot of really great stuff out in the market and that's not a situation that lasts for very long. So yeah, that's exactly right. So how do I think about it? One thing is you have to understand what the real problems are. So I think there's a lot of interesting academic work in computation biology these days, but a lot of it is pretty divorced from the problems that you face in the industry these days. So there's all sorts of hype around, oh, can we use machine learning to generate novel chemicals and things like this? A lot of that work is super fascinating and very interesting from an academic perspective. The problem is, is when you try to apply it in a traditional drug discovery workflow, it just doesn't match one-to-one against problems that you're trying to solve. And this is a pretty common theme that you see over and over again, that what makes a good academic problem where you, you have some interesting data set and you want to learn something about it and you want to make some predictions that, that can actually be pretty far divorced from an industrial workflow where you're trying to develop novel chemicals that are going to really make a difference in somebody's life, right, and cure a disease. And that's because the datasets you're looking for can be different. It can be that in your— the precision at which you need the answer is just so much higher in industry because you're making compounds that are going to go inside a human. It can also be because you need to just work in a sort of design, build, test, scale cycle that is frankly pretty hard for any academic to be able to successfully simulate. Nobody's gonna go build an academic-scale foundry just to see what these things look like, right? You have to go out there and build it yourself if you're going to do anything like that. So that's a lot of philosophy. In practice, what it means is that I have a strong bias towards get the data in an easy place, expose the data to scientific experts, and see what problems they're trying to solve, and see if then if there are data scientific solutions that we can layer on top of it that are really going to solve those problems for people. So I like to say that, you know, a lot of people are pretty darn smart in this industry, They can be biologists, they could be people, regulatory specialists, they could be, you know, have any sort of role in an organization. But they generally know what the problems they're trying to solve are. And those don't map one-to-one on the types of computational problems that you might intrinsically expect to be interesting ones to solve. And so there's sort of this element of user research, if you will, of like, what the heck are people trying to solve? And which of these problems can we solve effectively with the tools that we have through modern computational techniques. That make sense, Josh?

It makes a lot of sense. I think you want to, you know, in some areas, you know, machine learning is obvious, whether it's protein design or, you know, virtual screening, but there's maybe non-obvious applications, especially as you get closer to clinical data. And so how do you think about what are the big challenges? You know, standardized data is one of them, like you said, but is it— what other challenges have you seen in drug development in actually using data to drive decision-making across the board from hit selection to target selection to then picking the right indication and then moving into the clinic?

Yes, so that's an interesting point, Josh. And I think I might say to that, that if there's a common theme across all the problems that we tend to see, it's that people can't see across complex multidimensional data. And that those data can be multidimensional because it's like a whole bunch of RNA-seq experiments and you just have thousands of variables and they follow these weird high-variance distributions, and it's really hard just to look at those data, understand what's going on. You have to process them computationally. Another way they can be hard is you can just have long time series of patient records, right? I got somebody with a chronic autoimmune disease. They're seeing the doctor a couple times a year for years, and they may never have been given a diagnosis that properly maps onto the exact phenotype of their disease. Right? And I think a lot of problems, if I had to classify them, where you can really use machine learning on these data are ones where you need to take this high-dimensional, these high-dimensional data and bring them into some lower-dimensional context so somebody like a physician can actually understand what the heck is going on, right? And you can see this even if you think about drug manufacturing and supply chains. It's great to talk about drug volume and how much drug you're going to need next year. But once you start having multiple countries and multiple products in multiple different modalities across small molecules, biologics, etc., it starts being incapable— the human mind stops being capable of capturing all those pieces and putting them together. And so you quickly run into the fact that run into problems where the ability to integrate datasets, to put on some machine learning, some scenario exploration, some optimization really helps you see further. And that's kind of a good sweet spot to be in because you can understand what the problem is and you as a computational person, you as a data scientist, you as an ML expert, really have the tools that a subject matter expert needs to be able to do their job better.

Yep. And so for eCareRx, how do you think about generating your own data versus acquiring it from like a biobank or a hospital through a partnership?

Yeah.

And then how do you also then think about— you kind of touched upon this, and I see portfolio companies with this issue as well— how well annotated the data needs to be. You might be able to sequence somebody, but if you don't have an annotation on like, you know, a follow-up over a course of at least a year, that data is probably useless. And so what have you seen in your role, eCareRx, in terms of the data quality available out in biobanks and the challenges and opportunities? And then how has that influenced eCareRx in terms of your strategy to generate data versus acquired?

Yeah, um, that's a meaty question. Um, I, I'll say this, I'll say this, um, you know, I think most of the data that you can go out there and buy is oftentimes unfortunately quite low quality. Um, you know, the data market and the data business are not really— are unfortunately an industry that in many cases exists due to inefficiencies in the healthcare system and the like. That's why you can go out and buy EHR data in the US is that somebody wants to improve their billing system or the like. And so you can go out there and buy the data that comes out of their EHR system or the like. So, you know, a lot of the data that you can access pretty effectively is not really designed for what you want to do with it. So I, you know, that's a pretty negative thought, right? You obviously need data to build your business, but it does lead to, I guess, one key conclusion, which is you have to understand why the heck data are being generated in the first place if you want to use them effectively. And this especially applies to clinical data and EHR data and things like this. And it helps you think through what the challenges might be with them and sort of what the intrinsic biases in a dataset are. You know, I have to say the type of ML problems I like oftentimes include things like trying to figure out how to integrate out the bias in a dataset. So that's one thought. A second thought is that if you really have a data-driven business, you should be out there generating your own data in the long run. But that requires a time horizon, you know, for the same reasons that you put, Josh, like, you need follow-up to make these data be valuable in the long run, right? And so we can talk about going out and generating our own data with partners in the long run and conducting prospective experiments. And that's gonna be, that's a really powerful thing that you could do in the long run because then you can generate the data that you need and nothing beside it. So you can, in some ways, increase the quality of data because you're only looking for things that, well, you know you need. And frankly, a lot of data is manually entered and almost always manually curated. So the simpler your data collection procedure is and the more that it's connected with what you're trying to do with it in the long run, the better chance you are of having something really valuable.

Yep, totally agree. I think generating data or at least acquiring it is like the first step. And then ideally, the moat is streaming that data from a given patient or set of populations, you know, be able to have constant updates. And I've seen a few companies, it gets kind of expensive though. And so, you know, how does eCareRx think about not only building the dataset, but then keeping it relevant across the next 5 to 10 years?

I can't say that we've necessarily figured out all of those pieces yet, Josh, but, you know, I think the core of our business model at eQRX is around partnership with payers and providers and national health systems and the like. And so when you think about what those actors are going to be interested in, right, those actors are interested in saving money and increasing access and keeping their customers happy., right? And so if you think about data generation from the perspective of we need to make this a living, successful partnership, right? You know, you start seeing the, the ability of us to work together to generate data that's meaningful, right? Because we can bring the right therapeutic to the right people, save people a bunch of money, save actors a lot of money, and really explore, you know, increase dramatically the access to affordable medicine. And state-of-the-art care, which is really what this is about at the end of the day.

Yep. I think that's what makes eCareWorks such a great business. It's just you've aligned your incentives with the people who have the data, who are the patients, and so they're going to give you that data versus somebody whose incentives aren't necessarily aligned. And so maybe we can shift gears into the— you have data and you have to go and get it somehow. Either you you make it or you buy it, and then you have to like kind of standardize it. Also on that point, maybe to wrap it all up, do you have any tips or tricks for just people in the audience and just people who are in your shoes at other companies to quickly assess a quality of a dataset? I think a lot of data science leads always talking to biobanks or companies. It's always a tough job to figure out like how good is this biobank? Do you have any shortcuts you do, any heuristics that have been useful for you?

Heuristics on data quality. That's an interesting question. Never been asked that before. I think the key thing you have to ask is, what are the incentives here? Who's generating the data and why? And I know that's somewhat of a non-answer, but if you think from the first principle, if your first principle here is, There is some data generating process that if I had enough data and I had enough knowledge, I could actually model out and I could write a likelihood down for it and I fit it completely. If you think from that perspective, then you can start understanding where the data quality issues might be. And there's things that you can start spot checking. Of course, I work with a lot of healthcare market data. I'm blessed to be the son of a physician and related to a couple others. So I usually have a good way of spot-checking data because I always look up my dad, but, and my first cousin in these datasets. And I can usually start figuring out if something's wrong, if they spell their names wrong or get their occupancies wrong. But, you know, I'd say somewhere between those two, the philosophical method of try to model everything out figure out where the weaknesses would be. And the sort of brute force approach of look for something that you know is obvious that is not obvious to everybody else, you tend to start getting to the right place.

Makes a lot of sense. Incentives and befriend a doctor, try to have a—

I'm not saying a doctor, but I don't know, like let's think about a biobank, right? In graduate school, a lot of biologists studied one gene or one pathway, right? Chances are you're going to know a lot more about it than whoever is curating the bio, right? And so you're going to have a way of double-checking everything because you're going to have some weird expertise that whoever is running their quality system does not. And that's always useful to think through. Having some diversity of thought and perspectives on your team definitely helps there too.

It's almost like the Van Halen thing, you know, you make a really crazy request I only want M&Ms. I only want red M&Ms.

Right. Yeah, yeah. The no brown M&Ms thing.

Yeah, exactly. Something like that. And so then you go and make that request. If they can't do that, it's probably a— you probably shouldn't even talk to them. That makes a lot of sense. But maybe we can shift gears and really then start discussing how to build data science teams. And you built eCarX team. And so How do you think about one, hiring? It's so hard to hire. Like the biggest barrier to startups is hiring now, not money. It's crazy. And how do you think about one, finding the right talent? And then, and how do you then think about merging the two disciplines of say software data with—

Yeah, so a couple of thoughts on this, you know, I have a perspective here from having worked for close to 5 years at a plant microbiome company called Indigo, where I built out the data science team. And we were in this really weird field. We were developing microbial therapeutics for plants, which quite frankly work really, really well because plants really need to rely on their microbiomes for health in the long run. So one of the problems with this is I was operating at this interface of plant biology and agriculture and the microbiome and trying to hire data scientists, right? And frankly, these are 3 fields that either didn't exist in any large way or form 20, 30 years ago, or are fields that are unfortunately backwaters within the United States. You know, plant biology and especially agriculture and the biological side of agriculture is not exactly a hot field, and you're not going to find many people at MIT studying it or with experience in it. So I realized pretty early on that if I was going to find data scientists who had any relevant expertise, I had to abstract, abstract, abstract. So, you know, I realized, well, this is some kind of co-evolutionary problem of plants and their microbial hosts. So, you know, we should probably find somebody who has expertise in evolution and quantitative models there because they'll think the right way about this. And you know, if they've even thought about a microbiome before, that would be a great plus. And so that's how we got Max, who had a PhD in evolutionary genomics from Chicago. Wasn't a plant person, was an ant person, but, you know, close enough. We needed to build bioinformatics out, and so I was looking for somebody who at least understood the complexities of when you had large numbers of organisms that you couldn't classify at all. And didn't really know anything about, and, you know, having reference genomes was probably not something that was going to exist. And so I hired Nicole, who had a background in experimental evolution and working with all sorts of really weird yeast strains that you develop when you're subjecting them to artificial selection and strange selection pressures over time. And we kind of started building up a team like that. So instead of saying, I need this person who has this exact experience, rather going for, well, I'm looking for somebody who has, who thinks about the same kind of general problem and has some good scientific experience and can bring it to bear to help us think about and see problems from a different angle, right? I'm some kind of washed-up physicist that gives me my own perspective on quantitation and biology and biological data. And really, the last thing I want to hire is a bunch of people who think exactly like me. So in general, I think the key with data teams is you need to hire good scientists first. That's why they call us data scientists. And the problem is somebody might hire somebody who codes really well, and that's great, but most data is messy, most data is awful, and frankly, it's kind of like a science problem a lot of the time, especially in biotech. 'Cause you don't really know what the hell is going on most of the time. And so you want people who've worked with messy datasets that have no idea what's going on and have had to sort of find their way out of some complex thicket in the past. And so if anything, I've kind of moved away from looking for exact subject matter expertise and more towards people who've had to solve some complex problem with data and sort of had to sort of hack their way out through from this thicket of, of a world where nothing made all that much sense.

It makes a lot of sense. And so from your experience of building out these teams and leading them, has it been harder for engineers to learn biology or has it been harder for like scientists, maybe a biologist, maybe a physicist to learn how to code?

Yeah, so getting more biologists to code, that is super important because the— obviously, if you know a bunch of biology from the beginning, that's great. In general, I think it depends on the phenotype of the person. Some engineers are never going to learn that much biology or really get the sort of the, you know, really grok biology, if you will, you know, sort of just get the way that biology works and the ways to think about biological data. And that's fine, right? The best data scientists are also not going to be the best software engineers in the long run. So you kind of need to balance your perspective from that side. You need to look for people though, who are going to have kind of a quick lay bio mind, who are going to think differently about problems and be ready to immerse themselves in something completely new, right, that they know next to nothing about, and then start making good decisions about it. So, you know, I've hired people who are ex-particle physicists to understand the complexity of America's health insurance market and how money flows through it, which is actually a pretty great data and optimization problem. I can't say it's more complex than particle physics though, but you have to have a certain mindset to be able to explore it effectively. And then you just have to start building up and learning from there. I do think that experience of any sort with biological data can be super useful there, even if it's not super deep and it's in a different context, because there's a world in which you almost sometimes— I think that biology is kind of out to get us and biological data is violates the statistical assumptions in a uniquely perverse and interesting way that makes all these problems harder. But at the same time, that makes the problems really interesting. So it makes it really easy to hire people in the sense of, heck, a lot of the problems we work on are way more interesting than the problems of selling ads on the internet or predicting credit scores and credit risk. So from that perspective, hiring can be hard, but it can also be easier because You're, you're the good guys on some level. You're working in an industry that everybody wants to succeed, especially, um, in this, uh, pandemic-haunted world. And in the long run, your problems are way more interesting.

I totally agree. And then, have you ever seen— is there some sort of commonality between some of the best data science people you've worked with? You know, for example, some of the best The best bench scientists I know are incredible cooks. That makes sense, right? You have to follow a formula and meet the people. I'm a horrible cook.

For data, is it just something similar? But I'm a horrible bench scientist, so I kind of violate your product.

I think you're violating my rule. Actually, that was my old mentor, Gary Ruffkin. That was his way to filter bench scientists is cooking a meal. And if it sucked, you know, he wouldn't hire you. That's hilarious. But for data, is there something similar? Is it like, do you look for like blogging or like do you look for hobbies or something small that has been a commonality between some of the best people in the field?

You know, that's really funny because I've never spent that much time trying to look into something like that. And I bet you it could be useful. You know, in practice, you know, we'll look at GitHub and things like this where people have it. You know, oftentimes it's just useful to know, okay, well, you talked about solving this problem. What did it actually look like when you solved this problem? And that sort of thing can be useful. But, you know, not really. I've never really looked that deeply into hobbies, come to think of it. And I'm sure that there's some weird correlation somewhere because I tell you, you get the data team out and we all seem to get along with each other in a way as if we share some sort of deep commonalities. I'll tell you something funny, actually, and maybe this is the answer. Every once in a while, eQRX has a trivia contest, and so we'll do some rounds of speed trivia or something like this. And the last time we did a trivia contest, of the top 5 people in a company of close to 200, 4 of them were on data science. And of the top 10, 8 of the top 10 were on data science. And so, you know, we're, we're, you know, less than 10% of the organization, but 80% of the top players at trivia. So maybe that tells you something about the type of people who are great data scientists and engineers.

Really interesting. Okay, that's something to follow up on. I think there's always— I think data science people, from my experience, just love playing around with, uh, anything they can get their hands on, any piece of information, and just trying to like, you know, maybe trivia is like that. I'm assuming like crossword puzzles, maybe.

That's probably—

they're probably— yeah, you're probably good at crossword puzzles.

Uh, I'm very good at crossword puzzles, but my wife is terrible at data and she's also great at crosswords.

I know, so it's hard. It's hard to, these are heuristics, not rules, but maybe we can kind of bring it all together. You have a company, you have your data, you built a team, and then talk about how do we implement relevant models. And you've given me an incredible idea that will stick with me forever, is training models with unphysical experiments. And so for yourself and for eQRX, how do you think about implementing the right models?

Yeah, yeah. So the unphysical models comment. So I guess when it comes down to it, a lot of the time in industry, you're not confronted with problems that all exist in the same domain over and over and over again. So you can imagine some data living in some high-dimensional space and, you know, And here I am on audio talking about multiple dimensions, a high-dimensional space. So this could go over poorly. But what you're trying to do is you're trying to say, okay, in some region of this high-dimensional space, I can make some approximations and some assumptions and I can build a model. I can construct some function that models some process with a high degree of fidelity. And the problem with that is always, and sort of, a dark secret in machine learning is that almost always when you do this, your models are not very robust to what is sometimes called dataset shift. So your data looks slightly different than you expect. It might just be that it's from a different source or a different experiment, or it might be that something dramatically shifted over time in the course of the experiment, or maybe you're just looking at a slightly different biological target or something like this. So in general, you need models if you're going to implement them that are quite robust to change and a larger space of things to be considered than you might initially have thought. And one way I like to think about this is that if I were to make a lot of predictions on a biological dataset, and I would be generating data, I'd want to, I'd want to generate some data about situations that I knew were not going to lead to anywhere good in terms of an output. I'd want to test some molecules in an assay that I knew were not going to be hits, but I'd want to test them nonetheless because I'd want to understand how my model performs on them, how it gets them wrong if it does. So I need to look at these regions where there's really high variance in a model if I'm going to understand how to build a better model someday. And these regions of high variance are probably quite strange places, but oftentimes they're likely to be the places between relatively interesting regions. They could be the places that we're going to look and need to understand or need to be able to predict if we're going to build a model that predicts binding accuracy across multiple different kinases effectively. There's going to be some weird gray zone between all these things wherein none of our individual models would work, but if we want a model that will generalize, we need it to be able to predict effectively. And so I think that can lead you to having a different approach to both conducting experiments and thinking about what needs to be done. Because your experimental budget is going to need to be somewhat misaligned with what, say, a biological expert would recommend.

Could you expand more on that? And so you're kind of alluding to the need for more data to really first map out these regions of high variance and then give you— give your models a shot to find maybe new little nooks and crannies to make better predictions.

Yeah, exactly. So the regions of high variance are oftentimes where your models might make some super interesting predictions in the long run. But you need to know something about them because a model that's generalizing poorly is likely— it's going to have all sorts of places where the variance is very high, and you want to improve your model, you're going to need to pin down the variance in those regions. Right? But those might be between multiple basins of attraction or something like this, and I'm going to want to be able to understand them better. I guess furthermore, if you can kind of imagine that you're in some mountain range or something like this, you're sitting at the bottom of a big bowl valley, right? You're trying to say, well, this valley might get deeper in some direction or the other. It might be useful to sample some of the points up the hill a little bit where the change is going to be pretty high, you can at least tell more about the topography by, by looking way, way, way up the hill instead of just sampling nearby points and trying to find a tiny little gradient in the slope. And the more that you think, well, there might be a valley over those hills over there and keep thinking about this and trying to map the full range, the more it's going to be important to possibly understand the peaks between those valleys. Even if that's not that relevant to your initial biological question.

Yes. And so that requires like just really comprehensive sampling or experiments to go through every set of combinations of sequence or whatever space you're exploring. And so from your experience, how has it been to, on the data side, to get biologists to do experiments that might not make sense to them, but like is a requirement for better models?

You know, I think that's really the trick, right? Is because to get to the point whereby data can really drive your experimentation, that's difficult, right? Because I'll give you an example. Of some people that we talk to and we work with, if you say to some chemists, I'm gonna present you with a problem. We have this molecule, we wanna make a better version of it. We want to make something that binds better and has better pharmacological properties. A good medicinal chemist can give you hundreds, if not thousands of molecules that look something like it. That should be prioritized for testing, right? And you know, you can do some in silico screening around them. You can predict some pharmacological properties with varying degrees of accuracy. You can predict some biological properties of them with varying degrees of accuracy, et cetera, et cetera. And then at the end of the day though, you're still going to be left with hundreds of these and you You can't test all of them. And so getting to the place whereby you can say, you know what, what we're gonna do is we're gonna select a diverse group of them. We're gonna select the 25 most interesting of them that is also a diverse set. So some balance between exploration exploitation. You know, I haven't even gotten to on physical experiments here, right? But even that can be a pretty tough sell on, chemists. And you can see why, right? The chemists are going to be like, well, this one is not necessarily the most interesting one. This one is not necessarily going to be the most interesting one. There's 4 others that look way more interesting than it. And you're going to have to go back and say, actually, yes, I get that there's 4 others that look super, super interesting here. But, you know, if this one that we're— that's already going to be in the testing queue looks good, we can test those 4 others and try to figure out which one is best. But we should test some pretty diverse ones as well. And so you have to get to this place where you can let data drive your experimentation as well. And that is And that is important. And that is sort of the first transition to get to. And then once you get to that point, you can start sticking some funky things in there because it's more about learning the space. But that first transition of going to, wow, I can really, I need to let the data drive the experimentation more than just sort of prior knowledge of a biologist or a chemist. That can be a really, that's a really tricky cultural transition to, to navigate. Ultimately, however, I think it's a really important one to navigate for data and computationally driven biotechnology companies.

Totally agree. And I think this is such an important topic. I think we can talk about this for a few hours. And to be honest, Jacob, you have— I've talked to a lot of people and you have the best ideas in this field. You should write a blog post about how Data drives experimentation. I think it'd be really useful.

All right. In my copious free time, Josh.

I'll help you do it. You just send me your notes. I'll write it up. Maybe we can shift gears to EcurX and what you're doing there and this concept of fast following and it's kind of revolutionary to to be honest. Maybe the first question is, why did you join eQRX? You had a bunch of options.

Why, why eQRX? Why eQRX? So fundamentally, sort of philosophically, I think that we're in this biotechnological revolution. There's all these great new technologies that are really starting to bear fruit from computational ones to things like cell and gene therapy that are just dramatic improvements in our ability to treat patients, right? Uh, precision medicine is stopping being a buzzword and more starting to be just how we treat oncology. Um, so one thing you can ask yourself is, well, if we're in this space, like, what are we doing with our time and with this technology, right? And I was very attracted by eQRX's mission of expanding the affordability and accessibility of medicine Because frankly, what are we doing with this technological revolution if we're just producing drugs that are way too expensive for anyone to afford? Like, quite frankly, like, what are we doing with ourselves? What are we doing with our lives? And that's important. Um, the second thing, the second thing is that the fast followers problem, I like to say, integrates out some of the complexity of drug development. So at the end of the day, When you think about drug development or really changing any process and bringing in new technology, you know, you could say, oh, I'm going to bring in this, you know, new AI-defined method for generating chemicals, for generating molecules, right? Or for, you know, for generating antibodies or anything like this, right? You could say, I'm going to be Novartis, I'm going to put this in, I'm going to see, you know, and we're going to see how this works for novel targets. Well, what happens when you go to lead optimization and it turns out you just can't get the thing to be good enough in the first place? You chose the wrong chemical series. Or, well, what happens if it turns out you get to phase 1 clinical testing, the whole thing just doesn't work, right? This is a big problem, happens all the time in biopharma. So when you think about introducing new technology, I think the place to do it is a company like eQRx. That's a fast follower, right? Why? Because we've integrated out some of the complexity. We know the targets we're going after are real biological targets. They can modify the work. We know some of the key properties that we need to look for in the clinic, right? So we can start trying to think about how we can use data to inform our trialing strategy and how we can understand the patient population to really hit the people who are likely to respond and get to them as as effectively as possible. And frankly, it's why you can change the business model of selling directly to payers and providers and really partnering with them to expand access instead of paying, you know, large sales forces, armies of salespeople to go out to convince doctors to expand to prescribe high-priced medicine. And so the same things that make something an interesting new business model and a way allow for process innovation, really allow for the introduction of novel data and computational techniques because you've integrated out some of that complexity. So I'd say that was another major driver. And the third piece was that, frankly, they were bought into integrating data from the very beginning and being digital native from the beginning. So I was employee number 20-something And we've been really from the beginning building up the right data systems. I have to make a big call out here to my colleague, John Weiss, who's our head of informatics. He and I have been working really closely together from the very beginning to build out systems. He owns the pipes and I own the water, as we like to say. And that pairing from the beginning really allows you to set up an organization and a culture that's data-driven. And so putting those pieces together, you know, made it seem like this is a really cool opportunity to try some of this stuff. You know, maybe someday in the future I'll have some strong opinions on how we can use data to really transform drug discovery and novel target discovery, but I'd sure as hell like to see, you know, how does data inform clinical development? How does data inform drug creation? How can we make everything single source of truth? Soup to nuts in the biopharma industry, that's going to tell me a lot. And it's going to derive a whole lot of value and make medicines a heck of a lot cheaper for patients all over the world if we're able to do this right. And so from a data computational standpoint, man, it was an offer I couldn't turn down. Yeah, by being—

you're one of the few, maybe the only company that's really taken on the stance of collecting and analyzing data from target get to commercialization. So you have a lot of water to manage. Jacob, I'm going to call you the water baron from now on. You're the water baron. And then maybe on the other side, the technical side, then there's also the business model side. And Ecurix is really one of the few drug companies working to delight the customer. Could you talk more about that and talk about your stakeholders? You know, you have like maybe like 6 different stakeholders. You have patients, you have physicians, you have payers, you have, you know, PBMs potentially. How does eCareRx think about all these stakeholders and making sure that you're aligned with the patient and you're delighting all these customers? And, and that's kind of one thing that really makes the company—

so, so you're soon probably going to be going beyond my pay grade here, Josh, and I'm going to plug that aligned with us going public. There's an excellent series of videos from most of the C-suite on a lot of some of these approaches here that sort of go into more depth about this. But fundamentally, you know, we're aligned around partnerships, right? And, you know, the people that we are partnering with early on and the news that has been publicly announced is really around partnerships with organizations that dramatically want to to expand access and improve patient care and lower the cost of care, right? And that's deeply integrated into our strategy from the very beginning, right? And there's a lot of people out in this world who want to transform the cost of care and bring life-changing medicines to people around the globe for more, for better prices. So really, it's in part, it's about being selective about that and taking the right opportunities. So that we can build partnerships that are both going to be long-lasting from the business point of view and enhancing our long-term goals and mission of our organization.

Really exciting. I think, I think the next couple of years, I think understanding Eco-Rex's next generation products I'm sure it's going to be better than the first. Maybe to kind of tie it all together, you know, what are you looking forward to over the next coming year? Is it expanding partnerships? Is it getting access to new datasets? Is it, you know, establishing more deals with payers? At least for you, maybe the company in general, what's kind of—

Yeah, so I mean, I think, you know, we're, you know, things don't change much until they change all at once, right? And we're going to be dramatically growing again. We're going to be getting very close to the market. We're going to be having a much larger pipeline of drugs at all sorts of phases. And we're really going to be, you know, EQRX has struck a number of interesting drug development partnerships with organizations and companies that have really unique and interesting approaches to technology, as well biological and computational, but also sort of theses on how we can do drug development differently.. And one of the things I'm excited to do is honestly see how the heck some of this stuff ends up working out. It's easy to say, oh, fast followers, whatever. We know something about them. Making good drugs is hard and seeing where we can do that effectively and seeing how that translates and how the data follows through all the way into the clinic. That's something I'm super excited about because I guess One thing I like to say to people is there's a lot of data problems in biopharma where I don't know what the value of solving them is. Digitizing everything and connecting, say, everything from early discovery data all the way through clinical data, and then eventually real-world performance. I don't know which of those problems, if we solve it correctly, is going to generate the most value. Some of them may not generate much value at all. But it's going to be really exciting to see where solving those problems really moves the needle. Because anytime I see a data problem where something is done with old-school technology, or things are written down on paper, or not kept in a lone system of record, anytime I've seen something like that, once you bring in sort of a modern digital native approach, you start seeing making all these unexpected connections and seeing these gains that nobody ever thought about in the past. And so I really think that pulling all these scientific sources together in the long run is going to be dramatically efficacious and really kind of a game changer. And seeing how that happens is going to be really cool. Absolutely.

I think I'm really looking forward to the market uptake for the anti-PD-1 inhibitor and see how that works, where, you know, you're going to be able to undercut branded drugs but, you know, stay ahead of the generics.

And I think that's gonna be— I think I gotta say that's one of the things that gets me up in the morning when we're talking with an overseas partner is at the end of the day, I'm working for the company that's working to bring affordable PD-1 treatment to people all over the world, which is really a fantastic mission. Everything else, everything else, no matter anything else, that's a pretty cool mission. And that's something that'll get you up out of bed in the morning.

Yeah, and I think the implications are broad ranging to increasing access to curative medicines around the world, to catalyzing drug development in places like infectious disease and areas that maybe have less attention. So I think eCureRx is definitely one of the most exciting business models in biotech right now.

Maybe to wrap Yeah, no, I agree with that completely. And I guess one thing that I'd say, one little thing that always comes to me is, you know, one thing we don't think about very often is there's a lot of countries in the world that aren't particularly poor and that have high life expectancies that just can't afford drugs given current pricing models, even the lower priced drugs, price, lower pricing that you see in Europe. And you look at, say, what's it like to have breast cancer in Bulgaria or in Mexico? These are countries with modern health systems, but frankly, you know, income's much lower than the United States or Western Europe. And these drugs are prohibitively expensive. And we're saying to people, well, sorry, PD-1's not on the menu for you. That's crazy, right? And that's something that we should fix. And that's something that is an enormous business opportunity. Out there to make this work. And that's one of the things that makes me really excited about what we do.

Yep, I wouldn't be surprised a bunch of other companies start copying EQRX. So seriously, maybe to wrap up the whole, this really great conversation, do you have any kind of last pieces of advice? Pieces of advice? Maybe how to set the right culture for data science teams, you know, how to find a good, you know, find the right job or anything else?

Pieces of advice. Okay, what would I say? First off is if you're a new graduate, you know, or you're thinking about what you want to do, we're living in a scientific revolution and I know sometimes it can be hard to see, but some, you know, PD-1 is really effective, Keytruda is remarkable, and so Some of the medicines that we've seen on the market, and I guess with the mRNA vaccines, this has become more public, is we're really living through this revolution in biotechnology and biomedicine that was sort of sparked by all the investments of the late '80s and early '90s. And so if you think about all the growth in science since there, it's only going to continue to accelerate. And if you think about that, you have a unique opportunity to be at the forefront forefront of something that's really, really cool and going to transform society and transform human health in ways that you would not have imagined, and that I don't think people necessarily can imagine effectively. And so I feel like one should always ask themselves, am I doing something that's going to be— that I'm going to look back on in 40 years and say, man, I was involved in something really, really cool? You know, uh, if I had been my age back in the '60s, it would have computer chips and semiconductors. But today, I think it's biotechnology. You should ask yourself, would you rather be selling social media for cats or selling ads on the internet, or would you rather be at the forefront of this technological revolution? So it's one thing I'd say. I guess shifting gears, if I want to talk a little bit about culture, I think there's really— when you think about data, it's easy to lapse into this set of thinking that, you know, you know more than everybody else. And I like to think, and I like to emphasize to people that the data scientists don't know— know less about everybody else's jobs than they do. And our job is to help them see further with data. And when you take that perspective, it doesn't really diminish the problems that you're out there to solve, but it helps give you a spin on how you should think about solving those problems for other people. And the types of solutions that are going to be efficacious and sort of build, get, generate buy-in and build that sort of data-driven culture you need to have in the long run. Also, frankly, lots of people, especially scientists and clinicians, know all sorts of crazy cool stuff that'll keep your life super interesting if you think about it from that perspective. So I like to think that every day what we're getting up to do is to help, uh, is to help the scientists see further and get the best drugs out on the market as quickly as possible.

Really awesome way to end it. This has been awesome, Jacob. We should— if you're in SF or JPM, let's go hang out. And yeah, I think I totally agree. Biology is a frontier that is only getting more important. And so I think more diversity and different perspective, someone like you who's a physicist going into biology. I think you're in good company, actually. But Jacob, yeah, I really appreciate you taking the time. Everyone, thanks for tuning in. And Jacob, let's just ping me if you're in SF and we can just get some coffee.

I think— I'll be in SF this spring.

I'll be there. Let's do it. And maybe Leon will be there too.

Yeah.

Cool. Let's do it, Jake. Let's do it, man. Well, great, great catching up, uh, and we'll talk soon.

Talk soon. Cheers.

Have a great day, everyone.

Bye.