Deep learning technology for drug discovery

People would come to me and they'd say, hey, I've got a machine learning system. I can predict yesterday's stock price to within a dollar. Give me your life savings. We're going to be rich. And you'd hear that and you say, okay, well, it's good that you can predict yesterday's stock price. That's, that's something. Have you ever predicted tomorrow's stock price? And they say, okay, tough customer. And they go in, they come back and they'd say, okay, now we can predict yesterday's stock price to within a dime. Give me your life savings. You know, we're going to be rich. And say, okay, well, that doesn't really answer my question. Have you predicted tomorrow's? And they'd say, I can tell you're, you're. You're an expert, right? And they go in and they come back and they say, now I can predict yesterday's stock price to within a penny. What more can you want? And basically you look at that and you say, I have less confidence now than I did at the beginning.

Bandwidth for changelog is provided by Fastly. Learn more at fastly.com Our feature flags are powered by LaunchDarkly. Check them out at launchdarkly.com and we're hosted on Lino Cloud servers. Get a hundred dollars in Hosting credit@lino.com changelog this episode is brought to you by our friends at O'Reilly. Many of you know O'Reilly for their animal tech books and their conferences, but you may not know they have an online learning platform as well. The platform has all their books, all their videos and all their conference talks. Plus you can learn by doing with live online training courses and virtual conferences, certification practice exams and interactive sandboxes and scenarios to practice coding alongside what you're learning. They cover a ton of technology topics, machine learning, AI programming languages, DevOps, data science, cloud containers, security, and even soft skills like business management and presentation skills. You name it, it is all in there. If you need to keep your team or yourself up to speed on their tech skills, Then check out O'Reilly's online learning platform. Learn more and keep your team skills sharp@o'reilly.com changelog again, o'reilly.com changelog.

Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show@change dog.com community and follow us on Twitter. We're Racticalai FM.

Welcome to another episode of Practical AI. This is Daniel Whitenack. I am a data scientist with SIL International and I'm joined as always by my co host Chris Benson, who is a principal emerging technology strategist at Lockheed Martin. How you doing, Chris?

I am doing very well. How are you today, Daniel?

Doing wonderful, Blessed as always and especially this week. I got some sunshine and snow melting, so. So that's always a good thing after a bit of a deep freeze in the States. For those that don't know, this is March of 2021 in real time. But yeah. How about you? How are things down there?

Oh, spring is sprung. It's glorious. I'm desperately hoping we don't fall back to winter. So looking nice outside, having fun, doing some AI. What else is there?

Yeah, exactly. So before last year, each spring here in Indiana we go mushroom foraging for various types of mushrooms. And there's a number of apps that do like classification of mushrooms based on like picture, you know, you can take pictures of them. None of them are very good in my experience. I haven't liked any of them. And so last year I was like, before, next mushroom season I'm going to make my own little, you know, mushroom classification app. But here we are. And you know, such is not the case, so such is not the case. So it'll be my continual side project. Maybe someone out there, one of our listeners, maybe if you're aware of any good apps like that, let me know. But yeah, it's a very interesting object recognition problem.

Absolutely.

Yeah, it's fun for anyone that wants a side project.

So that sounds good.

Yeah.

This week, since I work in the defense industry focusing on this new really long report that came out, over 700 pages.

No, it's not about mushrooms.

And it's not about mushrooms, actually it's the National Security Commission on Artificial Intelligence.

Sounds very official.

Yeah, it's basically saying we need to get with it, we need to get with it. And so I am consuming. I've kind of gone through the highlights and I'm going through the detail of the report. So it's an interesting read in some, completely boring in some, but lots of interesting tidbits sprinkled throughout. So if anyone has an interest in and things at a, at a government level, it's, it's, it's something, it's great bedtime reading. Okay.

Yeah. Well, Chris, I have not got my coronavirus vaccine dose.

You haven't either yet. I'm trying, I'm waiting.

My parents just got their first Their first dose. I imagine it'll be quite some time before I get my, my own. But in discussions around that and like health related applications of AI in our Slack channel, there have been a couple of people that have suggested that we talk at some point on the podcast about, hey, is it possible to apply AI to drug discovery or to like, you know, finding vaccines or how, how does that cross into that world? And because we, we had those prompts from our listeners, we're really privileged today to have with us Dr. Abraham Heifetz, who is co founder and and CEO of Atomwise. And this is exactly what they're doing. So I'm really, really excited to hear more from Abraham. Welcome.

Sounds good.

Thanks for having me on the show.

Yeah, definitely. You want to give us just a little bit of a background about yourself and how you ended up where you're at now.

Sure, I'd be happy to. So my background is actually in computer science. I did my undergrad, my master's at Cornell, where I was focused on what today is called good old fashioned AI.

Symbolic type things.

Symbolic type things. Graph search algorithms. For example. One of my big projects was world robotics soccer. So the cornell RoboCup Team 2000 World Champions. And so this was, you know, you build little robots that play soccer on essentially a ping pong table, but totally autonomously. And the stated goal of that, you know, of that team is to beat the human World cup champions with humanoid robots by 2050. And so that was my background. Right. Like you can hear that this is pretty AI and computation, but it wasn't a lot of drug discovery.

Yeah. So is the doctor in front of your name, is that doctor of like PhD in computer science or is that like medical doctor?

It's doctor in PhD in computer science.

Okay, gotcha.

Specifically in computational biology. So after Cornell, I went to work for IBM on what probably today we call big data. I don't think we had the term. So we called it high performance Data processing. Much more IBM Y anyway, lots of acronyms. Yeah, exactly, exactly. And so this was in Boston. While in Boston I had this. I got interested in medicine. I thought it was interesting. So I started taking for fun medicine, organic chemistry courses kind of nights and weekends at Harvard. And I thought that that was really, really neat and that there were these deep connections actually with how computers play chess is. There's this deep connection with how chemists think about making molecules is. You know, it ends up being in both cases these tree search algorithms that you can apply. And so I went back for my PhD and I did my PhD at the University of Toronto now in computational biology. And so I had the good fortune to meet my co founder, my eventual co founder, Ishar Volif. And his background was in protein analysis algorithms. He'd been working for a small pharma company doing protein analysis algorithms. And so you can see atom wise. And the technology became atom wise. He put together three pieces, my big data sets, his protein analysis algorithms. And then we had the good fortune where literally across the hallway from us, Geoff Hinton's machine learning group was inventing modern machine learning. And so, like, we shared.

How fortunate.

Yeah. Talked about being at the right place at the right time.

Yes.

But we were able to see, actually before Alexnet got published in these sort of hallway conversations, we kind of saw that machine learning was able to do things that it wasn't able to do before. And we figured out a way to apply that to the realm of biochemistry, the domain of biochemistry and drug discovery. And so that was the genesis of Atomworth. Witte's Atomworth.

Awesome. Chris, I don't know about you, but often my spare time doesn't involve taking medical and biology courses at Harvard.

Everybody needs a hobby.

Yeah, everybody needs a hobby, I guess.

You know, this is eerily familiar. I have a very good friend who is very similar in that way. Both a deep learning person and a chemistry. A Harvard PhD in chemistry, and he crosses that chasm as well. So I'll need to introduce the two of y', all, because, y', all, you. You're sounding frightening like him. And I've never met anyone like either one of you before. So there. There's one other person that you can talk about this with, and they'll actually understand the whole thing.

So. So it's interesting. It's interesting, actually. Like, I'll tell you about the. We might end up cutting this after the show, but I'll tell you about the connection there between the good old fashioned AI. It's not about atomoids, but there is a connection between the good old fashioned AI and how chemists think.

Okay, let's hear it. That's totally gonna be in the show.

Okay, so here's the problem. So, standard problem in organic chemistry, you know, tests and homework, let alone what they do, is, here's a molecule, I'll show you a molecule. And. And you come up with basically a recipe to construct it out of simple, commercially available pieces. And the thing you need to understand about the chemists is that they think the world is made out of Legos. So they'll take that molecule and say, okay, well, I don't have that molecule. But, you know, if I broke it into these two simpler pieces, I know a reaction that would stick those two pieces together. Or alternatively, you know, if I. If I broke it apart a different way, then I'd have two different pieces, and now I could stick those together. Right, with a different reaction. And so now you've got kind of the same problem. And so in our parlance here, we would say we recurse. Right? And so for each of those simpler pieces, you apply the same algorithm. You break it apart, and you break it apart, and you break it apart and you get this essentially expanding tree of synthetic possibilities where out of simpler and simple pieces, you can, you know, how to put each piece together. And so this is called by the chemists, retro synthetic analysis. Retro, because backwards working backwards from the goal. It was actually the topic of the 1990 Nobel Prize, which was given to E.J. corey at Harvard, was for elucidating this idea of retrosynthetic analysis.

So does that go all the way back down? Like, I assume if you take that all the way back down, you sort of end up with the elemental makeup of our universe. I mean, how far does that go down?

Practically, it goes until you can buy them out of a catalog.

Okay, right, right, okay, that makes sense.

But even here, you see that you need computer tools, right? Because there's like, you know, I think at the time I was working on it, 14 billion compound. 14 million compounds you could buy of a catalog. Today, there's. There's 16 billion compounds you can buy of a catalog. And I don't know about you, but there are some mornings where I can barely remember, you know, 12 billion.

That's most mornings.

Yeah, right. Like, this is why you have to have coffee.

So.

So my point is that, like, you need computer tools to do this stuff, right? Like, you can't do it all in your head anymore. And so your audience can appreciate that we would talk about this as backwards chaining reasoning or heuristic tree search. And it's kind of like when a computer plays chess, it looks at the board and sees every move that it can make, and then thinks about every move you can make, and then thinks about every move it can make in response. And so sort of similarly, you get this expanding tree of possibilities. And now it's plotting a course from the current board until checkmate. And that's kind of what the chemist is doing, plotting the course from the molecule you want to these commercially available molecules. And so those links are Actually very deep. And so that was what I went back to get my PhD to build.

That system is that assuming, you know, the compound that you want to start with. Right. The problem is how you actually get there.

Exactly, exactly. So this is the interface maybe between the work of Adamwise and the work that I was doing in my doctorate for the doctorate was you found the answer. Now you want to know how to make it. Right. So there's one molecule and the question is, what's the recipe? To bake it, to construct it. But there's this whole problem which is, okay, there's billions of molecules you could make, which one is the best one to make? Right. It's sort of the initial, the pre. Problem for what I was just talking about. And so that's the problem that we're focused on, atomizes. How do you make a molecule which is going to be safe and effective as a medicine? That's the problem.

So before we get into sort of how you're doing that, how traditionally is that done? Sort of absent AI driven drug discovery, how has that been done in the past? Maybe with computer methods, but also without them.

Sure, yeah. And people have been trying to apply computers to medicine and drug discovery for decades. I mean, it's. These are good ideas, but they're not new ideas. It's just that I think we all see that the power of machine learning, computation today lets us do things that weren't possible a few decades ago. Right. So that's where the excitement comes from. But I'll tell you the answer to your question about what's the still common approach today. It's really doing these experiments physically and relying a lot on human intuition. Those, I would say, are the baseline tools today. And kind of, if you think about it. Right. Like think about that because that is an incredible claim if you think about every major industry on the planet. Right. And I don't care if you're talking about, you know, Lockheed Martin. One of my, my second job in high school actually was working for Lockheed Martin missiles in space.

Okay.

But if you're talking about, you know, Lockheed Martin designing a new wing, you will test a thousand wing designs before you ever build the prototype. Now you still take that prototype to the wind tunnel, Right. You still do a test flight, I hope, before I ever get near the plane.

Right.

Like, but you do most, maybe 99% of the experiments computationally rather than physically. Right. Because you want those, those are hard and laborious and finicky and so you want those experiments to work. Right. And you rather run one test which succeeds rather than thousands and thousands which fail. And so there most of the experiments are in the computer. I'm right now in San Francisco, right? So out here in California, I've got a pretty good guess that this building will stand up in an earthquake. That is an inordinately expensive experiment to run physically. And so we rely on structural engineers doing computational simulation to give us the certainty that we should move into the building. So every major industry, actually most of the experiments are done computationally, but pharma still, the baseline is to run those experiments physically. And so that's what we're trying to do. We're trying to give the same efficiency of every major industry to the drug discovery.

Hey friends. This episode of Practical AI is brought to you by Kodesh, a podcast from the team at Heroku that explores code technology tools, tech tips and developer life. There's tons of great conversations on the Codeish podcast, so I would encourage you to check it out and subscribe. But in particular, I wanted to bring to your attention two episodes, episode 98 and 99, where Julian Duquet explores the ethical and technical sides of deep fakes. The rise of manipulated pictures and videos and other forms of computer generated media are able to cause uncertainty and doubt what we see and hear from online. So how are we able to use these tools for good, if at all? Here's a sneak peek.

Let's say we want to do a deep fake of my voice and we train the model and we have enough data and everything. This will be also able to imitate my accent, for example, like how I pronounce English and the strong pieces of my accent or is not there yet.

It really depends. If there would be a person with similar accent on the input, then it would be fine. But it's kind of cheating. You can think it's cheating because we're reusing accent of a different person that's similar to your accent. But if it would be like an, like an American native speaker or a British person with a British accent, or like whatever, diff. Whatever other accent, then it will kind of be a mixture on the output. So we're not there yet. In terms of converting accents, it's a little bit more difficult than we initially anticipated because like when we started the company we thought it would be, you know, we'll kind of solve it in a year or something. But then it turned out that oh no, we're here for. We're here for much longer.

Check these episodes out. Links are in the show. Notes to both Episodes or head to Heroku.com podcasts to listen and subscribe again. Check the show notes for links or go to heroku.com podcasts.

So. So as you arrived at that point where you were fortunate enough to be across the hallway from Dr. Hinton and his team, and you started applying such techniques, deep learning, as we're calling it now, to this, what specifically could you tell us a little bit about how you got into using these new technologies of the day and how that changed your workflow at the time, as you were getting started and as you took that turn with what we're now calling AI or deep learning at this point?

Absolutely. Yeah. Let me draw the connection for you between what maybe is more common, image recognition, speech recognition, and what we're doing. And here's the way I'll do it, actually.

I'll.

Let me talk about the history of computer approaches to chemistry, prediction. Like I said, this isn't a new idea. People have been trying this for decades, maybe 1970s, basically as soon as we got high quality what's called X ray crystal structures, so 3D structures of proteins, which machine learning made a big breakthrough recently. And CASP organizers, CASP 14 organizers, said that AlphaFold 2 solved the problem. Right. So there's huge progress in AI in just getting the shape of the protein. But let me talk about. People have been experimentally were able to get structures since the 1970s, and people basically tried to use computers as soon as they could to work on those. And so the first generation was the physicists, you know, and the physicists came in, they said, what's the problem? Right. Like, I can compute Van der Waals dispersion, I can compute coulombic charges. The cows are spherical. How is this even a problem? Right. Like in. In the way.

Sounds like a typical physics approach.

Exactly.

Whatever you're doing is applied physics, so I don't see why this is a problem.

Exactly, exactly. I think, you know, in the show notes, maybe you should put that XKCD about the physicists. Right. So my dad's a physicist, so I like bragging a little bit on the physicists, so. Exactly. Right. And it turns out actually that if you do full quantum mechanical simulation, you get the right answer, which. Which frankly, I find comforting. That physics got it right. That's nice. But full quantum mechanical. Quantum mechanical simulation doesn't scale. Right. It's incredibly computationally taxing. And so you get the right answer for like lithium hydride, you know, very, very simple inorganic molecules, and doesn't scale to the Thousands of electrons which might be in. In a biological system.

Yeah, this was the whole thing with my PhD. I studied density functional theory, which if you. You probably run across that. And, yeah, for anyone that's interested in that, you can Google that and we won't talk about it here. But, yeah, I'm totally eating up everything you're saying.

Okay, great. So you know better than me. I mean, like, you know better than me.

Both the.

Both the strengths and the challenges in the approach. So then, roughly speaking, here's what I'd say is kind of the next generation came through, and it wasn't physicists, it was chemists. And the chemist said, okay, I don't just know physics. I also know a little bit of chemistry. I know that there are features that, in my experience, correlate with binding. Right. So hydrogen bonds. Right. Like, I believe hydrogen bonding is important for a medicine to hit a protein and inhibit it. So actually, it occurs to me maybe I should take a step back for the folks who explain how medicine works for a second.

No, that's a great idea. Give the context.

Okay, so here's the context. Here's the 92nd crash course in biology, which is.

That's all I do.

That's a big promise right there.

So think about the proteins in your body as machines on an assembly line, in that every machine takes in a very specific input, transforms it in a specific way, hands it off to the next machine on down the line. Out of the coffee you're drinking or the food you're eating, your body breaks that down and then builds more. You. One of the ways that disease happens is when those machines break or when they go haywire. Imagine, for example, the machine that governs cell growth, cell division switches on, and it never switches off. Okay. So that means that that cell will keep growing and dividing, growing and dividing. That's a tumor. That's cancer.

Yeah.

If you were wandering, you know, a factory floor and you saw a machine that was going haywire, you might throw in a monkey wrench so that the machine, instead of doing whatever it's normally doing, is just busy chomping on that monkey wrench. And you've essentially turned off that runaway machine. Okay. Just by physically blocking it up.

Right.

Turns out, actually, that that's how most of our medicine works today is you make a molecule and it just physically slots into a protein and it shuts down that protein. And so you arrest the disease process.

Or it connects to the receptor. Right. So that the. Whatever else cannot connect.

That's right. So it connects to the receptor. And so you disrupt signaling, or it blocks up an enzyme, so the enzyme doesn't catalyze the reaction that it should catalyze. Exactly right. And so I know that this sounds pretty abstract, but if you google something like the Philadelphia chromosome, you will see that there is just a mutation which switches on cell growth and cell division and does switch off. And it's this very clear link in cancer, exactly what we talked about. And so this means that people were able to design the first cancer specific drug to block exactly that mutated protein. So what does a drug need to do? Now imagine you've got a monkey wrench, right? You want it basically to do two things. You want it to stick really well to the disease protein, right? You want it to bind to the disease protein, to shut it down as completely as possible. You also want it to bounce off the proteins in your liver and your kidneys and your heart and your brain that you want to keep functioning. Right? Because you don't want to turn off 100 different proteins on the factory and cause all the nerve side effects. And so basically, you can phrase this as it's got to stick to what you want it to stick to. It's got to not stick. It's got to bounce off where you don't want it to stick to. Right? And that's talking about both efficacy and safety. Okay? Okay. And so. So that's a core piece of doing drug design. Now, there are other things in there, right? Like you want to make sure that it's soluble, right, in water, so that, like, when you drink it, it actually gets into your bloodstream, that it doesn't get metabolized right away by your liver or your kidneys. And so it hangs out long enough to reach the protein. I mean, there's other factors in there, but basically, like a core piece of what you want is, does it stick? Does it not stick? And today that is answered by setting up an experiment physically, which, as you can imagine, is difficult, laborious, finicky, expensive, time consuming, and all those problems. Instead, we phrase that as a binary classification problem. And so we're the first team to use convolutional neural networks, where you set up and run that as a prediction problem. And so an Image is a 2D grid of pixels, and every grid has red and green and blue color channels. Well, proteins are 3D, so we set up a 3D grid. Instead of red and green and blue color channels, we have oxygen, sulfur, nitrogen, carbon color channels. As soon as you do that encoding, then you can Essentially move all of the, adapt all the algorithms that people have used for image recognition, change them to the 3D biochemistry domain and get them to predict binding. Does that make sense? Very cool.

Yeah, it does, it does. So how early in your process did you move to this? Because I mean, certainly as Hinton was setting, you know, we weren't quite to the mature convolutional neural networks that we have today. You know, how did that progression look as you started to turn to the technology and then you kind of arrived at this utilization of convolutional in its present? You know, how did that look along the way? How did you make the steps and what did that reflect on what you were able to produce?

Absolutely. So Alexnet, if you remember.

Yes.

In the dark proto history of modern machine learning, that all the way back in 2012, that was published I think in December 2012 and we had our first publishing neural network running in January 2013.

Oh wow.

But that's because we had been talking to folks, right. Like before things were published. That's the question about being, being at U of T. Being on that same hallway. Yeah, yeah, it was pretty early on and then we published and these have become popular, popular tools in the drug discovery, the chem informatics, drug discovery, AI realm. You know, that's, that's the beginning of the story.

Right?

Like there's many things you got to do to make these things practical and successful.

Yeah.

I'm curious. So part of it is figuring out how to encode the information that you want in a way that you can utilize it in say a convolutional neural net. So that's the encoding piece, but I'm also thinking about the sort of data labeling piece like you're saying you're trying to predict binding or not. How clearly is that defined in terms of actually getting a data set that will tell you about all of these different molecules and whether they do or don't bind. What does it look like to put together that data set? I guess is what I'm asking.

Absolutely. You struck on something that's absolutely critical, which is the quality of the data. In some sense, if you're an academic machine learning researcher, you get to just care about MNIST and imagenet and cifar and in some sense you don't even care if those are labeled correctly or incorrectly. Like you now have ground truth data and you can ignore whether they were accurate or inaccurate. And I think people, people have, like, there are still mislabeled things in imagenet and totally ambiguous things and mnist, but who cares, right? Like it's, it's really how, how well do you overfit to those three standard now it's the standard. Exactly. And there's value in having the standard. But you know, our bug, the bug we're trying to close is have we ever helped a patient, right? Like have we ever cured a disease? It doesn't matter whether you're doing well on a benchmark. It matters whether you're having practical, pragmatic outcomes. Right? So anyway, if any of your listeners, you know, don't want to improve click through rates on ads, right? Like, or don't want to improve performance on a benchmark, but hold themselves to that standard of whether they're, they're helping humanity. Boy, like we've got more than enough machine learning problems to work on. So to get to your question about data quality, I want to talk to two pieces of it. Input data and output data. How do you tell if you can trust the input data? How do you tell whether your system is predicting anything worth paying attention to on the input data? There's a huge amount of data out there. Huge amount of data. So the National Institutes of The Health, the NIH, has a database called PubChem and last time I checked there was something like 240 million labeled data points of the kind that we use in there about proteins, small molecule binding, 98% of which fails our quality control filters. Okay? So there's a huge amount of data and there's a huge amount of noise. Let me give you one example. There's lots of examples, but let me give you one. You'll see in these databases, you know, a protein and a molecule and a measurement of binding, 3.14159 nanomolar, which is a measure, measure of binding. And then you'll see the same protein and the same molecule and the same 3.14159, but now it's millimolar. Okay? And so the only part here which is important is the nano versus Milla. Your people following along at home can see that you have exactly the same number, but it's off by a factor of a million times from each other. And you look at this and you say this can't possibly be two different assays what happened here, right? And if you dig into this, what you find is that somebody was citing their earlier work and they copied an entry out of their previous paper. And the letters N and the letter M are next to each other on the keyboard. And so they fat fingered. There's a Typo and, and you ended up with off by a factor of a million. I can't tell you in which direction.

But just a million, just a million.

And it's only when the poor schmuck of the medicinal chemist is trying to use your prediction, your prediction what a neural network was going to learn, or any machine learning, it's going to learn the average of these two. And so you're off by half a million, which you only discover when the portion of the medicinal canvas is sitting there trying to do the physical experiment. And so you have to do huge amounts of data cleaning. And it's not good enough just to do random cross validation, right. You know, five fold cross validation because you have to really appreciate where these error sources come from, how you got the data. And that's even before you talk about things like there are molecules which interfere with the assay, which aggregate up, which fall out of solution, which if I put sulfuric acid in a test tube, boy, it'll look like it's a great, great drug, but it's not for the reasons you care about, you know, and so you have to be able to clean, clean, clean a lot of that data. And so one of the things we've had to do is we've had to put the machine learning practitioner in the same room as the medicinal chemist, in the same room as the structural biologist, in the same room as the software engineer. Right. Because these things have to run at massive scale with incredible accuracy. And so we put a huge amount of effort into the data cleaning. And that's just on the input data. Right. There's the output data side as well. How do you tell whether any of this stuff is working? How do you catch the wrong, you know, in half a million in either direction problem?

Yeah. And is that part of just like monitoring and you know, trying to gain some intuition about what you're looking for or, you know, how do you go about that?

So that's the core question, right? Is how do you convince yourself even before you convince anybody else? Right. But how do you convince yourself that you're making progress and that this is working? Actually, if you look up my name, the last paper that I wrote was a paper basically looking at every, at every benchmark that we could get our hands on. We looked at every one of the standard benchmarks in our field. We looked and basically we found that there was this problem of data redundancy. How to explain, here's the kind of conversation I was having with people. People would come to me and they'd Say, hey, I've got a machine learning system. I can predict yesterday's stock price to within a dollar. Give me your life savings, we're gonna be rich. And you'd hear that, you say, okay, well it's good that you can predict yesterday's stock price. That's, that's something. Have you ever predicted tomorrow's stock price? And they say, okay, tough customer. And they go in, they come back and they'd say, okay, now we can predict yesterday's stock price to within a dime. Give me your life savings. Let's, you know, we're going to be rich. You can say, okay, well that, that doesn't really answer my question. Have you predicted tomorrows? And they'd say, I can tell you're an expert, right? And they go in and come back and they say, now I can predict yesterday's stock price to within a penny. What more can you want? And basically you look at that and you say, I have less confidence now than I did at the beginning because now you've convinced me that you're overfitting to the data, right? Like you're modeling the noise, you're overfitting to the data. And you've never done the killer test, right, like about whether you can do a prospective test. And so sort of similarly, similarly in our space, you see a ton of papers basically that the history is, there's a ton of papers where someone has a shiny new whiz bang system and then they report results on a protein. And you never know if that means does it work on that one protein or does it work on every protein or does it work on that one class of protein? Like what, what does it mean? Right? Like, or you show two or three results and it's, it's super, super challenging to really understand what the limits are or what it means. And so people set up benchmarks, kind of like MNIST and CIFAR and imagenet to try to answer this. But basically, my co founder and I, the last paper that we wrote, we looked at every benchmark and we found that there were these redundancies where basically you were teaching to the test, we were memorizing yesterday's stock price and we showed that the more that we came up with a mathematical definition for data redundancy between the training and the test set. And we basically showed that the more redundancy there was, the better machine learning algorithms look like they do. And this was true for every benchmark we looked at, for every machine learning algorithm we looked at, for every feature set we looked at for every training test split we looked at, it's just an incredibly robust set of results. And so my personal conclusion is that most of the history of computational chemistry in this sort of corner of it has been rewarding overfitting and teaching to the test instead of rewarding real prospective can you predict tomorrow rather than can you predict yesterday? Sort of depressing result. And so what we said to ourselves was, okay, we're just going to have to show that this works on 100 different proteins. It works, you know, on different diseases, it works in different labs hands, just so it's a, it's a robust result, you know, like, you know people are hiring, right? Like you can get a job selling ads if you're a machine learning person. Like so does this matter, you know, are we making progress? And so what we did was we launched, you know, we're not expert in 100 different proteins. You know, I'm a computer scientist, I'm not an expert in any protein. And so we decided we were going to have to partner with people who work and nobody's an expert in 100 different proteins. So we launched a wide set of collaborations with academics and, and what we did was imagine you're a professor and you believe protein xyz. If you block protein xyz, that would cure cancer or, or you know, a certain type of cancer or Alzheimer's or Covid. You tell us you want molecules for protein xyz. We go screen commercially available molecules. We buy the best molecules out there, we get them formatted, plated, you know, ready to go into your assay. We ship you physical molecules, you run the experiment and you tell us whether we were right or not.

Foreign.

We deserve a better Internet. And the Brave team has the recipe for bringing it to us. Start with Google Chrome. Keep the extensions, the dev tools and the rendering engine that make Chrome great. Rip out the Google bits, we don't need them. Mix in add and tracker blocking by default, quick access to the Tor network for true private browsing and an opt in reward system so you can get paid to view privacy respecting ads. Then turn around and use those rewards to support your favorite web creators like us. Download Brave today using the link in the show notes and give tipping a try on changelog.com.

So now that you've kind of taken a step through how you got there and how all this stuff is kind of how you think about it and how it works, could you pick kind of a use case of a, maybe a specific disease that you guys have worked on and kind of like, how do you apply that? Maybe tell us about a story where you're doing this and you got some level of the success of doing the process as well as some of the challenges that you hit along the way just to make the whole thing real and give something very tangible.

Sure, I'd be happy to. Sure. So in this setup where you're a professor and you're sending us compounds, let me give you the sort of the stats on the program and then I'll give you an example.

Sounds great.

So we set up this set of applications, this program, we set up this program called the Artificial intelligence Molecular screen or AIMS program. And it's been hugely successful, actually. We've had over 1600 applications from more than 250 universities and research hospitals in 50 countries. We've accepted more than 775 programs, projects covering 600 unique proteins. And that, I mean for context, like a big pharma company might have like 60 small molecule projects. And so we have 600 unique protein targets here. And so this is really operating on a massive scale, covering every major therapeutic area, cancer and neurodegeneration and infectious disease in there. And so we have data back now for more than 150 of these projects. And so we can really give very statistically sound results about how well we're doing. When I present this at chemistry conferences, I usually stop and I ask the audience like, okay, you've seen the setup, what success rate? Think about your favorite screening technology in your mind, and maybe that's a computational approach, or maybe that's a physical approach, or maybe it's just throwing darts at a catalog. Whatever you think is the best way, what do you think the success rate is? And the answer I most frequently get back is 10% success rate that they think out of 100 such projects, if we found anything in 10 of them, that's what they expect. And if we we were able to get 20, then they would be really impressed and delighted. And so we have about a 75% success rate on these projects. So this is importantly better. Yeah, significantly, significantly better. But actually, let me walk you through an example here. Here's the important part, because it's not just a question of cost, it's not just a question of speed. It's actually a question about making the impossible possible. There's something that not a lot of people appreciate is in the human genome, we've only ever had medicine like FDA approved medicines for 4% of human genes. There's another 16% of human genes that have it good evidence implicating them in human disease that we would want to be able to target. We've never been able to target. So this is like four times. The entire pharma industry is waiting for us to figure out a way to get medicine for those. Right? So there's huge opportunity. But to do what's been impossible, you need technology that have never existed. That's why we're out there inventing these. Right? Like, so that's what we're working on. So let me give you an example here. And so this is joint work with Professor Ron Viola at the University of Toledo. And there's a disease called Canavan disease. This is an ultra rare neurodegenerative disorder. If you're pregnant, it's one of the things that. And you're doing a genetic screen. It's like one of the things that they test for in genetic screens, basically. I won't get deeply into the biology, but you've got a molecule in your brain called naa. N acetyl aspartate. You have a system that makes it in your brain. You have a system that clears it out in your brain, and these kids lose the ability to clear it out. This NA builds up in the brain and, and basically the sheath around your neuron, the myelin sheath, starts to degrade and these kids stop hitting developmental milestones. It can be fatal. There's really no cure for it. So it's pretty tragic disease. But there was some mouse data that showed that having lost the ability to clear this naa, if you slow down the synthesis, you could bring the system back into balance. And the mice live full lifespan. And so that gave us the idea that we could develop a drug for the synthesis side. But that the N acetyl aspartate synthetase, that synthesis side is a classic undruggable target. And I'll tell you why. It's something called. It's inside the neuron, so it's in the central nervous system, it's inside the neuron. Your brain's protected by something called the blood brain barrier. So it's like an armor around your brain, so it's very hard to get to. That protein itself is membrane associated, so it's stuck in the membrane inside the cell. And that makes it very finicky to work with, very hard to work with, difficult to express, difficult to purify. Just getting enough of it to run large experiments, which is, as we discussed, the state of the art in pharma today. You couldn't get enough of the protein, you couldn't get a crystal structure for the protein. And so I don't know what alpha tool can do, but experimentally we had no way of designing molecules based on the shape of that protein, designing that monkey wrench which would block up the protein. And so basically all of the standard doors for, for drug discovery were closed. You couldn't run a big physical screen, you couldn't design based on the structure. There were no molecules, drug like molecules, which were known for this protein. So you couldn't build a machine learning model specific for that protein. Human medicinal chemists didn't have anywhere to get going from to start working on. Basically all those standard doors were closed until we worked on them. What we did was with these machine learning techniques, empirically they are more robust. And so you can use distant homology models. So the kind of thing that AlphaFold2 produces, we can use that rather than having to have an experimental X ray crystal structure, a 3D experimental 3D structure of the protein, we used a very, very distant protein, a bacterial protein. So separated by 3 billion years of evolution, only 20% of the Amino acids were identical. So very, very distantly related. But we use that as a homology model. We screened 7.2 million molecules and for context, a big pharma company might have like 3 to 5 million molecules in its corporate collection. So this is maybe twice the size of a big pharma corporate collection. Out of 7 million we pulled down to 60 that we thought we'd test and 5 of the 60, so quite a high hit rate. 5 of the 60 were actually accurate with the best one being more potent than what you would expect from a physical screen. And so that's not a drug yet. We have to continue working on it. But where all the doors were closed, now we've opened the door to doing drug discovery.

That's so cool.

It really is. Yeah.

That's quite amazing story because I know like, I've heard of these things like and you know, not being a medical person or a biology person, you hear about these diseases or other things where it's really not possible to develop a drug or they don't know where to go. And it's really cool to hear about, you know, some stories of people trying to push, push through that barrier and really commendable work. I'm curious, maybe a slightly weird question, but one of the things that people of course are so concerned about with kind of the rapid expansion of applications of AI into all spheres of life is various sources of bias in, in the data that we're using. And I think this has been particularly, you know, not in the case of AI, but another source of bias that people have been talking about recently is with vaccines. And of course, certain populations who maybe have a certain history with vaccines or other things are very concerned, for example, with the coronavirus vaccine of, you know, hey there, you know, does this work well for our population or other populations, Are we getting the, you know, the bad vaccine or something like that? And so I'm wondering, as you are specifically trying to apply AI in these cases, what is your thought process around sort of making sure that you're accounting for some of that bias in your methods and you're creating, you know, sort of drugs that, that are kind of applicable to general population that is diverse.

Absolutely. So I think it's absolutely the case that we need medicines for, for all people everywhere in the world. And I think one of the things I'm proud of is, is the fact that we're working, you know, that we open this program globally, that we're democratizing access to these kinds of technologies to researchers around the world and that they can decide what diseases, you know, they're concerned about, where they see the ability to make a breakthrough. Right. Like I said, we've had these applications from over 50 countries. If our priorities about which medicine we work on is just happening in Boston, Massachusetts, there's going to be a skew to the kinds of diseases which are familiar in Boston, Massachusetts, or keep people up in Boston, Massachusetts. But if you look at another place of the world, stomach cancer has a much higher prevalence in East Asia than it does in the US I don't know if it's because of environmental or death, dietary or genetic factors, or it could be a mix of all of those. South Asia has a high cardiovascular burden. There's different kinds of liver disease in Southeast Asia than other parts of the world. So absolutely right that what is high on the list and what is top of mind for people is going to be different in these different places. And so I think by dropping the cost of developing medicines with these new technologies and dropping the barriers and reducing the timelines and putting these technologies into the hands of people, then we can help democratize that decision making. So one example, we're working with an NGO based in Geneva called the Drugs for Neglected Disease Initiative. And there we're working kind of similar style story, but we're working on a disease called Chagas disease, which is an endemic disease of poverty in Latin America. By having new technologies and making success easier to reach and faster to reach that, you know, we can, we can go and we can tackle a much broader set of diseases. So that's, that's our perspective on a very important problem. I'm glad you raised it.

That's a really great perspective. I think it's a great point that this sort of diversity of diseases and also how it's diverse across geographies and populations is like an AI sized problem, right? Because we do only have so much capacity of like expert chemists, expert doctors, medical professionals, like, like we only have so much capacity there, but yet we have this sort of increasingly complex disease situation around the world. So yeah, I think it's a really good point.

So, you know, to that point, when you say AI sized problems, can you kind of tell us. I mean, this sounds like it's truly in the process, in the early stages of revolutionizing an entire industry because of the scale and the accuracy. You've changed the whole thing. And so where do you see this going? You know, no one can tell the future, but if you'll put your, your wizard hat on for a minute and pull out the crystal ball and speculate on what it looks like, 5, 10, even beyond where you think, you know, when you lay in bed at night and you're thinking about where you're going with this and where you want to go with this, what does that look like? And how does the world change as a result of this, you know, within that industry, which affects all of us, where are we going?

I think you're right to note that it affects all of us. Right? Like this is, this is one of the fundamental universal truths about being human, right? Is that we get sick. You know, our parents get sick, our kids get sick, you know, the people we care about get sick. And so this is something that all of us face. And I think, you know, in some sense no one comes down on the other side. No one, no one says, yeah, chemotherapy is, those side effects are fine and the success rates are fine. Alzheimer's, we don't have any real treatment. Well, I guess that's it. We're just going to have to live with it, or not, as the case may be, or diseases of poverty around the world that we haven't been able to afford to direct large discovery. We'll just, what can we do? I don't think anybody is actually advocating for any of those approaches. We want better treatments for all of it, let alone diseases that we had fixed. But we're losing ground, right? Like the WHO is talking about, you know, a post Antibiotic apocalypse, right? Like, and when you see serious governmental organizations use words like that, right? Like, it's. It's serious and that's the growth of antibiotic resistance, right? Like, we need fundamentally new approaches to antibiotics just to maintain the kind of lifestyle that we're. That we've been happy with, right? Like one.

One of the.

The stories there is Calvin Coolidge's son got a blister playing tennis on the White House lawn, got sepsis and died. This is most powerful man in the world. In the era before penicillin, you had no protection. Nobody had protection against it. And so we need technologies that keep up with the evolutionary arms. So I think you're absolutely right that this is critical. I also, this is a long, maybe long and maybe rambling answer to your question.

No, long and rambling is good when you're predicting the future. Okay?

I also want to give credit. I want to take the opportunity to give credit to the chemists. Real transformation happens at the intersection of multiple different pieces. If you think about AI, there's data and there's algorithms that come together, but those wouldn't be able to be run without the huge success by DevOps and cloud computing and GPUs, breakthroughs in the hardware, which are driving much of that. What is it? I was just looking at this because I'm a nerd. If we didn't already establish this, the ASCII White, the most powerful supercomputer in the world, and therefore in the history of our species up to 2001, ASCII white, right, clocked in at over $100 million and 100 tons. That machine peak flops is an Xbox today, right? Like, that's an incredible transformation, right? And so if you think about, like, why are we able to do these things with AI today? It's because of the massive success by hardware engineers, right? It's driving a huge part of it. So I want to give a shout out to the chemists. There's been this equal exponential change on the side of the chemists, which is why AI, why we should care about AI. So here's what they did 15, 20 years ago, big pharma, like, if you and I wanted to order compounds out of a catalog, right, These commercially available compounds, there may be a million molecules that you and I could buy, okay? And big pharma like Pfizer, Novartis and GlaxoSmithKline and Bristol Myers, they had maybe 3 to 5 million molecules in their warehouses. And so in that world, it was better to be Pfizer and Novartis. Right? Because you had a better shot about finding something in your catalog. And then with an army of chemists, you could iterate your way to a drug from that initial something. Remember in Kanyman, that's. That's what we lacked was that initial something. That's. And that's what we were able.

They were already scaled up enough to. To have a good chance starting.

Exactly. Well, they had a better chance than you and me. Right.

Good or not.

That's. That's an empirical question. Right. Like, after you ran the screen, you could tell, but they had a better chance than you and me. Okay, but here's what happened in sort of the 15 years afterwards is we've adopted something called. The industry has adopted something called synthesis on demand, and here's where it works. You may remember Dell computers. Okay. Of course. So you remember, okay, Michael Dell had what was principally a business innovation, which was, I'll get to work after your check clears. Right. Like that's principally a business innovation. But there was a corollary to that change, which is that the range of different computers that Michael Dell could sell you was way more than what anybody else could sell you, because it was every potential combination of printer and memory and monitor that you could choose to put together. Sure, right. He waited until you said what you want. And so basically the same thing happened in chemical vendors is they store these days building blocks, and they say, I know how to put them together. And so what they sell you is a catalog of 16 billion different compounds that they know how to make, but that they haven't made yet. Right, exactly. Like Dell computer knows how to make. That computer hasn't made it yet, and they're adding about a billion molecules a month. Okay, so we're talking here already like maybe 5,000 times the size of a big pharmacork question that you and I have access to, and we can get it shipped in four to six weeks. Okay? And so this means that, like, basically 99.9% of all molecules ever available to medicinal chemists today are accessible only through computational approaches, because you can't test them physically. They don't exist physically to be tested. You have to run the experiment first, and then you can run, you know, you can. You can purchase the molecule to run the physical. You have to run the computational experiment first. Every chemist today is a computational chemist. If they're being, you know, if they're really looking at it this way, every. Every medicinal chemist is a computational chemist. Okay? So that's a fundamental Shift. And that was driven by the chemists. Right. The fact that they've been so successful in these syntheses and coming up with these syntheses, and if you draw the trend line of that growth by 2024, if the. If the trend, you know, if they stay on trend, those libraries are going to be about a trillion molecules big. And so this is why we need AI. It's because at that scale, it's not enough to be 99% accurate. Right. 99% accurate means 1% inaccurate. And the point where you're running a trillion molecules, 1% accurate means 10 billion false positives. You need way better than 99% accurate. You need 99.999. Whatever. And it just turns out that our best technology are machine learning technologies. Yeah.

You know, for most people listening to this, you just put their problems in perspective of being not so bad. Most of US aren't having 10 billion false positives. I'm just saying.

Right, exactly. Yeah. And so there's huge promise out there. Right. And these are deeply meaningful problems. Right. Like the potential, if you can crack, this is worth it. But boy, you got to be willing to take on a 10 billion false positive problem. Right. Like Facebook doesn't have a 10 billion false positive problem. Right. Because there's only 7 billion people. Right. Like, you're capped how badly things could go.

Yeah. Well, I definitely, for one, am super glad that you and your team are willing to be one of those taking on this problem. It's so inspiring and cool to hear about some of the things you've already done and, you know, some of the success that you've had. And it does really sound like there's some wonderful things in the future. So, yeah, definitely look forward to having you back on this show to give an update on how things are going. But we appreciate you taking time to talk with us. I know I've learned a lot. It's been really great.

So that's a fantastic conversation.

Thank you.

Yeah, I had a lot of fun. There's a ton of open problems like, well worth working on. And at their core, it turns out that medicine is an AI problem.

Thank you for listening to Practical AI. If this is your first time, make sure you subscribe so you don't miss a thing. Head to PracticalAI FM to subscribe or find us in Apple Podcasts, Spotify or wherever you listen to podcasts. And if you get value from the show, please do share it with a friend or a colleague. We appreciate you spreading the word. Practical AI is hosted by Daniel Whiteneck and Chris Benson. It's produced by Jared Santo and our music is provided by Breakmaster Cylinder. We are brought to you by some awesome sponsors. Shout out to Fastly, Linode and LaunchDarkly. That is our show. We hope you enjoyed it and we'll talk to you again next week.