Founder Stories: Daphne Koller, AI-Driven Drug Discovery Using Digital Biology

Daphne Kohler is CEO and founder of Insitro, a machine learning-enabled drug discovery company. She is co-founder of Engage.ly and was a computer science professor at Stanford, co-founder of Coursera, and chief computing officer of Calico. Daphne was one of Time magazine's 100 most influential people and is a MacArthur Fellow a member of the National Academy of Engineering, and a fellow of the American Association for Artificial Intelligence, the American Academy of Arts and Sciences, and the International Society of Computational Biology. Please welcome to the stage Daphne Koller.

Good afternoon, everybody. It's a real pleasure to be here and have an opportunity to tell you about some of the work that we're doing at insitro on bringing together biology and technology in order to make better medicines for patients. So the, the tale of drug discovery is a tale of the glass half full and the glass half empty. The glass half full is exemplified by this slide where demonstrates some of the advancements that have happened over the last 50 years that have entirely transformed the lives of patients that used to have diseases that were a sentence of death or to a life of pain that are now manageable chronic diseases. Or sometimes even cures. And cure is not a word to be used lightly. I'm not going to go through every example on this slide, but there are many patients that have been helped by the advances in this area. The glass half empty is what's come to be called Eroom's Law. Eroom, for those of you who may have noticed, is the inverse of Moore's Law, which I'm sure you're all familiar with. So Moore's Law is an exponential increase in productivity in technology. Eroom's Law is the exponential decrease in productivity of pharmaceutical R&D, where if you look at the y-axis, which which is on a logarithmic scale, it shows that the number of approved medicines per billion US dollars has gone down exponentially year on year for about 50 years. Now, why is it that those costs are increasing so rapidly? It is because fundamentally, this is a game of failure. The success rate of a program that goes into clinical development of actually being approved as a medicine on the other side is about 5%. Varies a little bit by therapeutic area, but that's has been holding pretty steady, which means that the amortized cost per approved drug is $2.5 billion and rising because each successful drug has to carry on its back the cost of all of the many failures that didn't make it through. So there are many reasons why one can— that can be attributed to this. But the one that we think is really important is that we fundamentally do not understand human biology and human disease. So when we think about how we characterize disease today, the taxonomy that we typically use is one that is derived from very subjective coarse-grained symptomology that often dates back decades, that is filtered through the subjective lens of both a patient and a clinician, and ends up lumping into a single mushy bucket multiple things that arise from very fundamentally different biologies. So the goal of what we try and do at Insitro is to actually objectively, quantitatively, and with high content measure the underlying biological state of health and disease so that we can then create a taxonomy of disease that is much more driven by those underlying biologies, provides us with actionable interventions, and hopefully allows us to make a real difference to patients. So in order to do that, you need to collect large amounts of data. And we collect data in two forms. We collect data from human patients, which we aggregate in partnership with others. You can collect data from modalities like MRIs and CAT scans, from histopathology data, from transcriptional profiles, which measure the expression of every gene in the genome simultaneously, other forms of blood biomarkers, and more and more modalities that are emerging. In parallel, in order to also create an intervenable experimental system where we can actually do because you can't do experiments in humans until the very end, you— we create in our own labs a rich set of in vitro data, which means in the test tube, which again uses modalities such as transcriptional profiles, imaging, proteomics, and many more. Now when you create these large amounts of data, these are way too much for people to interpret. And so this is where we use machine learning to interrogate those data and come up with an association to clinical clinical data, clinical outcomes, and association to genetics, which gives us that very hard to find but super important notion of causality because by and large genes are causal of clinical outcomes. And then this serves as a bridge between the genetics and the clinical outcomes, allowing us to find these intervenable nodes in human disease. So the platform that we've been building over the last 4 years or so is really something that collects large amounts of data from humans, generates in our own labs large amounts of data from cells, uses the genetics as the bridge between the cells and the humans, and allows us to create this manifold of disease state that we can then use to interrogate human disease. So in order to do that, we have built both a network of incredible partners on the human data side and some pretty remarkable capabilities in our lab that range from automated cell culturing systems that use machine learning to figure out when you want to culture the cells and what you want to do with them, ways to screen hundreds of millions of cells in parallel with different genetic backgrounds, and a really cool microscope that is also machine learning enabled that I'll tell you about in a few minutes. So the focus of the first part of the talk, or actually a thread that will proceed throughout the talk, is learning meaningful representations of of clinical state. So where we measure human biology with very high fidelity, high content data, we run that through a self-supervised machine learning model. I'll tell you a little bit about that. And then use that in order to understand what we're seeing in the data. And the focus of much of the talk today will be on histopathology data because we've done a lot of work there, and there is just so much richness in that signal. So the first case study that I'll tell you about is work by these wonderful two scientists, Mikhail Barakat and Paolo Casale, and it works, and the work is on understanding phenotypes of NASH. NASH stands for nonalcoholic steatohepatitis, which is a liver disease. It emanates from what's called NAFLD, nonalcoholic fatty liver disease, which is something whose prevalence, unfortunately, is growing very rapidly. It currently covers about a quarter of the world's population because of the growth in obesity and type 2 diabetes, which lie upstream of NASH. NAFLD progresses. So NAFLD is the uptake of lipid droplets into liver cells called hepatocytes. When there's too much lipid in the hepatocytes, they become inflamed. They start to die. That in turn causes inflammation more broadly in the liver, leading to fibrosis and scarring and ultimately hepatocellular carcinoma, to the point that today NASH is one of the most common reasons for liver transplantation. And cancer in the liver worldwide. So in order to interrogate this disease, we had access to a pretty incredible data set via partnership with a pharmaceutical company where we had histopathology data from a liver biopsy that was taken at the beginning and end of a clinical trial, RNA sequencing data, which is the transcriptional profiles were taken from the same samples. We also had biomarkers, blood biomarkers, and importantly, human genetics, which again speak to causality. So each of the biopsy samples was graded by a single central pathologist, so very high-quality consistent data. And they graded each of those samples using 4 ordinal scores that measure things like fibrosis, inflammation, steatosis, which is those lipid droplets, and ballooning. And we tried to do various things with these ordinal scores. And unfortunately, we're not able to see a lot of signal in those scores. Scores, not because the scores weren't good, but just because there's not a lot of variation. These are patients with fairly advanced disease, so they sit on a very small subset of the scale. Usually they have fibrosis like 4 to 5, and so there's just not a lot of signal to associate with other types of data. So the first thing we did is we applied self-supervised learning to the histopathology samples. I didn't include the result data. Any details on the machine learning model, but it uses an attention model, so it doesn't rely on any pathologist annotation whatsoever. It is driven entirely by those images. And then with that self-supervised embedding, we overlaid a single linear layer to predict the pathologist scores, as well as to predict other important clinical covariates that are potentially quite informative. So what you see here on the left is the association with with a pathologist score. So on each point on the x-axis is the actual pathologist score, and you can see on the y-axis the machine-learned score. Very strong correlation, but with a fair amount of variation. And so the question you might ask yourself is, is that variation just noise, the machine learning doing poorly, or is that biologically meaningful? And what you see here on the right is that one way to get at that, because of course we don't have ground truth here, is to ask whether a patient who would if a patient progressed over the course of the trial or not. And what we see is that patients who progressed actually have— actually, this is regression. Patients who regressed were lower down in the scale of the machine-learned score than patients that did not regress, suggesting that the biological variation that we see in the score is actually biologically relevant or clinically relevant. Now, that's nice, but we don't aim to replace pathologists. That's not the goal of the work. The goal of the work is to really understand the biology of the disease and how it relates to other important biological covariates. So what you see— so what we did here is to associate those scores that we got from machine learning with various important things. So first of all, we tried to see if we can predict liver gene expression. Remember, the gene expression was extracted from each of those patients. And what you see here on the left side of the chart is the association of liver gene expression. Each of those points is a gene. With the pathologist scores on the x-axis with the machine learning score on the y-axis. Anything that's above the diagonal is the machine learning having a stronger association with the gene expression than the pathologist score. So you can see pretty much everything is above the diagonal. And the orange dot that you see is what is demonstrated in the right-hand panel where you can see the correlation with the pathologist score across patients being marginally significant. And the machine learning embedding giving a much stronger correlation with the gene expression. So even more interesting than gene expression associations is associations with genetics, because as I said, that speaks to causality. And this is where we found an interesting phenomenon, because we looked for associations with genetics. And we found a good number with the machine-learned score because they are quantitative and hence better powered. To associate with a genetic variation in genome. What was surprising to us is that the best characterized genetic factor that drives NASH was not discovered, not only via association to the very coarse-grained pathologist scores, but not even to the machine-learned quantitative scores that were trained towards the pathologist scores, which is that middle bar on the left graph. However, When we said, you know what, forget the pathologist scores. Let's just take the machine-learned embedding and see which genetic variant is most correlated with that unsupervised, self-supervised embedding. And you can see that very strong bar on the right-hand side being a very strong association there. The lesson from this for us was that it's not the case that the reason we don't see these genetic associations, this this particular genetic association is because we were not powered to see it, but because the pathologist doesn't know how to look for it. And therefore, if you train a machine learning model towards what a pathologist knows to look for, it learns to ignore what is a biologically relevant signal. What you see on the right, by the way, is that association with a beautiful dose response. The x-axis is the number of disease alleles going from 0 to 2. So in order To teach the pathologist what to look for, we employed two forms of explainability. On the top half is fairly straightforward, which is just looking using our attention model for the tiles that were most predictive of the disease variant and the healthy variant. And you can see that they look different. But it's hard even for a trained pathologist to look at two completely different tiles and say exactly pinpointing the difference between them.. And so in order to create a more interpretable model, we used a generative adversarial network to basically hallucinate tiles along a disease axis that goes from the GG to the CC, from the disease allele to the healthy allele. So you can really highlight to a pathologist, as you move along this axis in embedding space, what is it that makes the disease versus healthy. And that turned out to be a really valuable way of visualizing visualizing things to a biologist. So that's one vignette. I'm now going to talk about a different vignette. This is very recent— sorry, this is still on the NASH front, but this is how to tie that back into the in vitro or lab data that I talked about. And this is work by Hervé-Marina Lee and Zach Phillips. So as we said, we are looking to align clinical data with in vitro data. And so in order to create in vitro data, we basically built in our lab— and this, by the way, is only one of the forms of data that we collect in our lab, but it's particularly, I think, interesting in the context of this audience. So Herve and Zach built this machine learning-enabled microscope. So what do you see on the top half panel of that left panel? You see a set of rotating LED lights, and what And the idea here is that as light shines through the cells, which are semi-translucent, depending on what the light hits, it refracts at different angles. So if you're able to capture not only the intensity of the light but also the angle of the light, you can try and infer what is it that the light has hit. Is it a lipid? Is it chromatin? Is it cell membrane? And so what you see here is a machine on the bottom side a machine-learned reconstruction of what is effectively a gray blur that provides a high-resolution, high-contrast reconstruction of cellular morphology, including its internal structure. Then we had another machine learning layer overlaid on top of that that basically marks particular cellular compartments with fluorescent stains. This is trained using a method by which we take the cell We fix them. We stain them. And then that stained output is provided to the machine learning model. So it learns to map between the gray input to the colored input— to the colored output. Now, with that machine learning model in hand, you can actually start to apply that to live cells. And so basically, you can now get all of the information content of this— what's called fluorescent stained microscopy, which requires requires that you fix and stain the cells without actually stopping and killing them, which is really important because first of all, killing them changes them, and it stops the experiment in the middle so you don't get longitudinal phenotypes in the way that you see on the right-hand side. So with that in mind, we basically were able to— you can see this is a reconstruction between— up on the top is the real staining with the experimental staining. On the bottom is the imputed or hallucinated stains. You can see they're visually indistinguishable. And now you can start to look at various cellular phenotypes, including in live cell imaging. So this is how we created, for example, a NASH disease model by taking hepatocytes, introducing a cocktail that induces a NASH-like state. And you can see that we were able to actually, again, take all those cells, self-supervised learning on top on top of those cellular images, and then with a single linear layer being able to impute which, sorry, to predict which of this, how much NASH cocktail each cell got with a single linear layer. So very clear morphological phenotype to this NASH cocktail. And now you can take, and I'm not sure the video will work here unfortunately. Oh, maybe it will, or maybe not. Where you can see that you could actually see in live cell imaging the cells reverting when you're introducing a drug that reverts the disease phenotype. So you can see that on the top. Okay, so the next vignette that I'll share is very recent work by Chris Probert and Zach McCaw, and we took took the success that we had on histopathology in the context of NASH and applied it to a completely different domain, in this case in oncology. So here we took a fairly large cohort of multiple different cancers. It's a pretty old cohort. We're actually working now with a much newer one via partnership with Genomics England. But here we started with publicly available data, which has over 20,000 tumor and normal samples from 33 cancer types. And there is a fair bit of metadata of additional covariates that are associated with each of those— with some of those samples. Not every sample has all of these, and that's important. We'll get to that. So first of all, we look to see whether you could actually predict, for example, mortality risk using, again, a self-supervised embedding on the histopathology data. And you can see that this is, by the way, a pan-cancer model. We have similar models for individual cancers that also create that nice separation. And you can see that again, single linear model, in this case trained using Cox regression, for those of you who are familiar with it for survival analysis, you can see a beautiful separation between high-risk and low-risk profiles that was learned with that single linear layer on top of the histopath embedding. Embedding alone. But again, I think the more interesting aspect is to try and associate the learned embedding with unmeasured covariates to get increased biological insight around the pathophysiology of the disease. So here is an example that aligns with what I showed you for NASH, which is trying to predict or associate genetics with genetic drivers, in this case somatic driver mutations in cancer, those mutations that happen in cancer cells that give rise to cancer. And what you can see here is that from the histopathology data alone, you can actually predict with pretty high accuracy the genetic variant that was the tumor driver for this cancer. This is within breast cancer, which is actually pretty impressive because a person can't look at the image of a breast cancer and say this is a G53 mutation. Going one step beyond that, we looked at a fairly new type of biological measurements called ATAC-seq. This was something that was first described less than a decade ago in 2013. And what it basically measures intuitively, for those of you who are not biologists, is the extent to which the chromosomes are open or closed. If they're open, that means it's an active region of the genome. Which allows stuff to happen. And if it's closed, it means it's packed up tightly and you can't— it's not accessible. So chromatin accessibility allows us to go considerably beyond pure gene expression because it also allows us to interrogate things like non-coding driver mutations, which are actually quite common, as well as other mechanisms of disease that have to do not with necessarily the expression of genes, but just with something called epigenetic, and those are often an underlying cause of therapy resistance. Now, ATAC-seq is a very finicky assay. It is not used as a standard of care. It's very sensitive to sample quality. So there are a few small cohorts where these have been measured very carefully in a research lab, but they are not available in general for normal tumors of patients. So here what we tried to do was to ask ourselves, is there enough signal in the histopathology data in order to predict these ATAC-seq signals? And so again, you take the histopathology, self-supervised learning, and then single linear layer, seeing which ATAC-seq signals can be predicted. Not all of them are predictable because not all of them actually associate with histopathology, but there's 5,000 that were actually predictable from the histopathology alone. And you can see that the predictions are actually pretty darn good. So from the embeddings, you could actually predict with pretty high accuracy 5,000 of those histopathology— of these, sorry, ATAC-seq peaks. Now, this is a really interesting slide because it then asks the question, Does understanding this attack-seek signal give us additional information? So what you see here is the same kind of survival curve that I showed before. High risk at the bottom, low risk at the top. On the left-hand side, this is in colon adenocarcinoma, so it's cancer-specific mortality risk. On the left-hand side is predicting it from the embedding alone. This is commensurate to what I showed you a couple slides ago. On the right The right-hand side is predicting it from the attack seek alone. So basically you take the embeddings, you predict attack seek peaks using the embeddings. Now you have 5,000 variables from which you predict outcomes. It's not quite as good as the embedding alone, but it's still pretty darn good. What's really interesting is that when you combine the two, you actually get a better separation between the high risk and the low risk. Now this is a little bit surprising because the ATAC-seq is actually imputed directly from the embeddings on the histopathology. So there is not actually additional information that's provided to the algorithm, but simply by training it in this way, it gives the algorithm an additional focus that allows us to make better predictions about outcomes than either of these two representations alone. Now, without getting too much into the biology for this audience, I can tell you that there is some really interesting biological interpretability that came out of those imputed ATAC peaks. So basically, for example, one of the most important predictors of outcome to immune oncology treatments, those immune checkpoint inhibitors that I'm sure you've heard about, is something called tumor mutational burden, which basically says how much variability does this tumor have you have in terms of the number of mutations, which is a surrogate for genomic instability. Surprisingly, the more of those mutations you have, the better in general your response to immunooncology because the immune system is better at recognizing those aberrant cells. And so the question is, what is the driver of this tumor mutational burden? Because maybe that is a therapeutic target. And so we associated the imputed ATAC peaks with tumor mutational burden and were able to identify a locus on chromosome 11 that actually, that the genes that are within that locus, all 3 of them are implicated in tumorigenesis and indicate poor disease progression. And so this is actually a really interesting sort of connection between the genetics of the disease, or in this case, actually the epigenetics of the disease, and outcome. Now what's important is we would never have been able to make this association using the very small cohort of 400 samples on which ATAC-seq was measured. There's just not enough statistical power. But with the imputed ATAC-seq over a much larger cohort, you're able to make this association with tumor mutational burden. So in some sense, you can think of this as allowing us to provide— to use embeddings to provide a bridge between research data, which is very high quality but often on very limited cohorts, and the very large real-world standard-of-care datasets where very limited modalities are measured, often with uneven quality. But if you can tie them to each other using the kinds of embeddings that we've talked about, you're actually able to get the best of both worlds. And what's important is that it allows us not only to make better predictions for patients like the example that I gave you, but also to do better biological analysis by having these research-grade covariates that give us insight about the underlying mechanisms of disease. So effectively, you can think of this language of histopathology, which is what we're we were able to demonstrate here, and we could do this for other modalities as well, providing us with sort of a unified multimodal representation of patient state that allows us to do all sorts of things like make better patient outcome predictions, potentially make predictions about response to treatment, as well as uncover novel biological insight that might give us give us the path forward towards better therapeutic interventions. So I'm going to end this talk with a philosophical note that is really the answer to a question that I often get asked, which is, of all the different things that a machine learning person can do, why are you doing what you're doing right now? And so, I mean, there's the obvious answer, which is you want to make a benefit to patients. But I think there's also another answer which is maybe broader and more philosophical, which is when I think about the history of science, there are times in history where a particular discipline has made a tremendous amount of progress in a relatively short amount of time due to a different way of looking at the world or a different way of measuring it. So in the late 1800s, that discipline was chemistry, where we for the first time understood the periodic table and that you couldn't turn lead into gold. And that really created a very different understanding of chemistry. Early 1900s, that discipline was physics, where we understood the connection between energy and matter and between space and time, completely transforming how we look at the universe. In the 1950s, that discipline was computing. Where we created artifacts, machines that are able to perform calculations that up until that point only a human, and soon thereafter not even a human, had been able to do, which created not only really useful things, but also the whole notion of the theory of computability, for example. And then in the late, mid to late '90s, two disciplines suddenly took off in parallel. One is the field of data science, which emerged from computing, but also from statistics and neuroscience and operations research, and it's what gave us modern-day machine learning. The other is what I call quantitative biology, where we measured for the first time biological systems not as a one-off small experiment, one gene or one organism at a time, but really at unprecedented fidelity and scale. Using tools like sequencing and microscopy and multiple other ways to interrogate biology in the world around us. These two streams, data and biology, largely moved in parallel with relatively little interaction between them. And the field that I think is emerging right around now is really the synthesis of those two disciplines. It's what I call digital biology. It is the ability to measure biology at unprecedented fidelity and scale, to interpret what the human can never interpret in terms of the amount of data using the tools of data science and machine learning, and then to take some of the incredible tools that bioengineers have developed and actually cause biology to do something that it would not normally otherwise do. And that has huge implications across a whole set of set of realms. I've talked today extensively about human health, but I think there's equally important repercussions in agriculture, in the environment, in biomaterials, in energy, all of which lend themselves to the engineering of biological systems. So I think that is really the field of the next 20 or 30 years. I think we'll see a tremendous amount of progress over those decades. And ones that will make a huge impact to all of us in the world that we live in. So that's it, and thank you very much.