Machine Learning in Clinical Genomics

Well, it looks like we're right at the hour, so we'll go ahead and get started with session number 4 on machine learning and clinical genomics and give a couple of brief introductions. As a co-moderator, my name is Casey Overby Taylor. I'm an assistant professor of medicine and biomedical engineering at Johns Hopkins University.

Good afternoon, everyone. My name is Eric Boerwinkle, and I'm the Dean of the School of Public Health at the University of Texas Health Science Center in Houston. On behalf of Casey and I, we thank you all for being here and welcome. There's no other business from the organizers. Why don't we just dive in and introduce our first speaker? Who is Su-In Lee from the University of Washington, and the title of the talk is Explainable AI for Cancer Precision Medicine.

Hi, I'm Su-In Lee in Computer Science and Engineering at the University of Washington.

My lab develops explainable AI techniques for a wide range of problems in biology and medicine, including clinical genomics, which I'm going to mainly talk about today. So when we have a set of patient features and a well-trained machine learning model, we can predict various kinds of clinical outcomes or clinical phenotypes. However, in many cases, we cannot answer a very natural question of why a certain prediction was made. And similar problems occur when we use a complex black box machine learning model to understand the relationships between molecular profiles and phenotypes, for example, gene expression levels in brain tissues and neuropathological phenotypes. The model may predict the phenotype accurately and then describe the data accurately, but it does not give us mechanistic explanations of how gene expression levels influence the phenotype or the other way. Machine learning is basically a black box to many of us. So recently my group focused on addressing the black box nature of machine learning by asking largely 3 questions. First, how to learn or select features that are interpretable. Second, how to explain a certain prediction by estimating the importance of each feature and their interactions. And how to biologically or clinically interpret a complex black box model such as a deep neural network. So to address these questions, we develop various kinds of explainable AI techniques for decision support systems for hospital to better understand cancer and then precision medicine and Alzheimer's disease for which there aren't any drugs at all. So we need to understand the molecular basis for the disease first. So that we can discover therapeutic targets. In this short talk, I'm going to focus on cancer precision medicine and general AI techniques that are important to these problems. Say that there is a patient X who has AML, an aggressive blood cancer. Like other cancers, there are many anti-cancer drugs this patient can be treated with. However, Standard therapy is not personalized, and our long-term goal is to build an AI system that can take molecular profiles of a patient and then predict the response to a large number of the anti-cancer drugs that can be used. And then in our prior project studies, we showed that showed the effectiveness of the method for jointly training the prediction model with various kinds of prior knowledge on genes' driver potential so that we can identify explainable gene expression markers for each drug. And they showed that it improves the model's performance, and then also the ability to identify the molecular markers that are actually likely to be biologically meaningful. Then in our ongoing project, which I'm going to focus on in this talk, we are interested in improving the process of selecting a combination of drugs. The reason is that single drugs are often not effective, and then in this case, this makes our problem harder. Because if there are 100 FDA-approved drugs, suddenly there are over 10,000 drug pairs. So when we have too many choices, it would be extremely useful to know why or why not certain drug pairs are a good option. So for that, we are collaborating with Camilla Nexorova at Harvard Medical School. We are in the process of developing the eXpress method. For each patient and a particular combination of drugs A and B, our method, eXpress, takes as input the patient's gene expression data and then drug combinations features and then outputs the predicted drug synergy between A and B for this particular patient tumor. And then not only that, it also explains that for each prediction, by giving the gene-level score, how much each gene expression is important to predict synergy or not synergy. And then by using that, we can infer the— we can gain pathway-level insights into what processes are important for a certain synergy prediction. So we trained this express model based on the BEAT-AML data from Oregon Health and Science University, which contains contains tens of thousands of samples and then from nearly 300 patients, and it contains 133 combinations of drugs. And then to perform the cross-validation test, we created many multiple settings in terms of how training and test samples are divided from the easiest setting where you simply randomly split to the more difficult setting where we made sure that we only see novel combos in the test set. That's number 2, and the novel patient or novel drugs in the test set. For various such cross-validation settings, we compared between our method with alternative methods by learning the prediction model using the training data and then testing using the held-out test data. The performance is measured by 1 minus MSE, so higher the better. And then the y-axis here is the XGBoost model we ended up using, deciding to use, and then we compared that with 3 alternative approaches because we had to choose the best performing model in terms of the accuracy. So as you can see in this scatter plot, this XGBoost is a fast implementation of gradient Boosting trees, its performance is better. Its y-axis is better than that of the other methods, which is x-axis, linear model, neural network, and random forest in terms of this cross-validation test, the prediction accuracy. And then especially the predictive performance of this linear model was, you can see that it is substantially worse than the complex models. So these pink-colored dots are more deviated from diagonal than other methods. And this is a very common situation in machine learning, the trade-off between accuracy and interpretability. What that means is that with large and modern datasets, the best accuracy is often achieved by complex models, which even experts struggle to interpret. So, and then, but linear models are easier to interpret, but they often perform worse than the complex model. So to resolve this challenge, we have developed a series of machine learning, general machine learning techniques to measure each feature's importance for a particular prediction made on a particular patient. This first method is named SHAP, and I'm going to show you, give you a very basic idea of how it works. It's a general machine learning framework that can be applied to any model. It doesn't matter whether you use a tree ensemble like we used this XGBoost or deep neural networks. And then it has 3 desirable theoretical properties that are important, which many other methods in this domain do not. And to us, its ability to provide simple explanations of predictions from arbitrary complex models, that eliminates the typical trade-off between accuracy and interpretability. So how exactly today do we estimate the feature's importance? And then to give you a, you know, very high-level idea, I'm gonna use a very intuitive example. So let's say that this is, you know, there— this is John, and then he is a typical bank customer like many, you know, customers today. He, when he applies for a loan, his information is sent through a predictive model. And then this model is designed to calculate the risk that John will have a repayment problem, which unfortunately for John, it's 55%, and that the bank declines his loan application. And then again, the natural question is, why? Okay, so to explain his denial, it is important to start with a base rate of loan repayment problem. What is the percentage? What is the chance that any loan is declined? It's denoted here by the expected value of the model's output. And then to explain John's risk, which is 55%, we need to then explain how we got from the base rate to his rate, and that's going to give us explanations. So to do that, we start at the expected value of the model output, and then let's say we condition on a single feature of John. Which is age. And then since this increased John's risk by 15%, we attribute that increase to John's age. That's how much age is important. And then next, we condition on John's occupation, which is not the— may not be the most stable, and then again attribute the change in the expected model output to his occupation as a day trader. So if you Repeat this for all the other features, say open account and the capital gain, and then you end up conditioning on all features, and so you arrive at 55% of the model, which is John's predicted risk. So basically, these lengths of arrows, how much you attribute, just attribute to each of the features, may indicate their importance. Here, it's actually important that in this process the order matters. For example, the impact of John being a day trader, it could be particularly bad if you're a young day trader. It means that the impact of day trader could be large if you already know that he's young. So which means that if you reverse the order, it could be that age is more important, day trader shrinks. So this SHAP values, our method, it averages, it tries to average, consider all possible orderings and then average these attributions over all n factorial possible orderings. And we showed in our paper that it's the only solution which maintains important three properties. And then these are— these properties make any comparison among features in terms of their importance values meaningful. So, um, then, but of course, you know, this is how— certainly not how you want to actually compute the value, the shape, these values. Because you have to consider all possible orderings, it can be computationally very intensive. Further research needs to be done. We are doing some of the follow-up research to develop efficient ML algorithms to compute these SHAP values. Just to tell you, in our prior study where we used the SHAP method, the preliminary version was used to help us the prescience method to predict the near-term risk of hypoxemia during surgery and explain the factors that led to that risk. And then we showed that the explainable prediction of hypoxemia improved the clinician's ability to predict the event. And again, for this project, we used earlier version of the SHAP method. And then we recently developed ML methods to efficiently compute exact SHAP values for particular model types. We have done for tree ensemble, now we are working on deep neural networks. And then first of all, why trees? Because it's extremely popular models, especially many of the Kaggle competition winners used XGBoost. And then, you know, the result of the Kaggle data survey done by a couple of data scientists to show that Tree models are most widely used model in industries these days. And then we designed tree algorithms that reduces the exact computation of SHAP values from exponential to polynomial time. And then I will show you soon that how this method was effective in identifying the explanations underlying drug sensitivity. So, and in this, tree SHAP paper, we applied it to various clinical and prediction problems in medicine, such as CKD or mortality or surgery duration. It's currently used by a lot of companies. And then in the link at the top, you can see where you can find this code, and then so that you can use it for your own research. And then as a part of this work, we developed a rich visualization, visual representation of a global importance of a feature. So we call it CHAP summary plot. So say that you used NHANES data to predict mortality of nearly 10,000 people who were interviewed and tested in 1970s, followed up for 20 years. And then we extracted nearly 60 common measurement and demographic values as features for Cox proportional hazards model in XGBoost. And then for that way, for each individual, we can compute not only predicting the mortality, we can also compute the SHAP values. And then for each feature, we can visualize its distribution. So here, x-axis is the SHAP values, and then these dots represent the SHAP values of all individuals, and they are plotted horizontally and stacked vertically when they run out of space to show the density. So each dot here is colored by the value of that feature. For example, for age, it's young— blue means young, red means old. So, and you can see, you know, surprisingly, that age is the most important factor. And then overall, you know, some of the features show rare large effects, and overall, many features show long tails reaching to the right but not to the left. For example, systolic blood pressure only has the large impact for a minority of people with high blood pressure. So this general trend means that the extreme values of these measurements can significantly raise your risk of death, but cannot significantly lower your risk. In other words, there are many ways to die younger, but there are not many ways to be out of range and live longer. Then now let's return to this eXPRS, explainable prediction of anti-cancer drug synergy. To understand what makes a drug combination have a synergy, we computed the SHAP values for each sample. Again, each sample means a patient, and a combination of drugs. And then we create the SHAP value matrix for gene expression features, so where each row is a sample and then each column corresponds to a gene. And then, as we saw before in the SHAP summary plot, if we focus on each feature here, gene expression level, then we can generate the SHAP summary plot. That shows the idea of which gene expression features are important for drug synergy. So then here is the SHAP summary plot for genes with a positive trend, meaning high expression usually indicates synergy, and then the negative trend. And some of the genes are very well known to be relevant in AML biology, known drivers such as MYC1 and the TLR3. And some of the other genes in the top, you know, N genes. And then this, you know, gene list enables us to perform a pathway enrichment analysis to identify the pathway gene sets that are— that could be potentially mechanistic explanations of a drug synergy.

So now we—

so we did that pathway analysis, and then the most important genes for our model turned out to be enriched for metabolism or hematopoietic lineage. So tight junctions and then the JAK/STAT signaling and so on. And you're looking at the result of this KEGG pathway, curated pathway, the enrichment analysis result. And in addition to that, we also look for enrichment in C2, GC's C2 gene expression signature. So more, you know, general gene expression signatures outside of the canonical pathways, we observed that many of the top genes fall in the pathways that are related to hematopoietic stem cell-like expression signatures, so-called stemness. And then these results show that this, you know, hematopoietic stem-like expression signature is relevant. It's important to drug synergy. And then We performed drug or combo-specific analysis to reveal similarities and differences across these drugs. Again, there are 133 drug combos in the training data from nearly 50 drugs. And then one thing— one of the results indicate that when we embed the SHAP values for each drug combination, we see that certain drugs tend to be clustered together, like venetoclax and then penobenostat, and then the combinations containing drugs between the two clusters tend to be embedded between these two clusters. And then when we examine combinations containing other sets of drugs, we notice that they are not clustered as well. So we have many, many sets of results on this important scores, the shared values for the drug combinations in our BioRxiv preprint. Okay, so to summarize my talk today, so in this short talk I presented our new, you know, the new and old explainable AI techniques, SHAP and then MORE, and then, and then, you know, I focused on this cancer precision medicine, the project, and then the high-level conclusion was that These complex models are useful. They are more accurate. They can also express complex relationships between biological variables. But we need new methods, machine learning methods, to make biological or clinical sense. So I'd like to thank the students in my lab, including 11 PhD students, 1 undergrad, and then 3 MSTP students. This MD-PhD student who chose computer science as their major. And then I'd like to— we work with a large number of collaborators at UW Medicine and Harvard Medical School and then many other institutions. And then the project we are doing is largely funded by NIH, NSF, and the American Cancer Society. Thank you.

I guess now we can start the Q&A. There are a few questions that came in. We can— so one of the questions that I had was, in your discussion of future importance to help make models more explainable, it was really clear that there are several demographics that are important. And I'm just wondering if you you could comment also on the importance of modifiable factors like behaviors since that has implications for discussions between clinicians and patients.

Yeah, right.

That's an excellent question. So when you have these features that got the high importance score, your first question is, is there any way to mitigate the, you know, the modify the outcome, clinical adverse outcome we usually are interested in predicting. So the features that are modifiable versus not. So demographics, it's obviously not modifiable, but features such as, say, blood pressure or some other features, there are things depending on the clinical context, there are features that are modifiable. So those features would be important, but one more thing that's important is causality. If some feature is modifiable and has a causal influence on the outcome you want to predict, then those features are the most important in clinical context or even biological understanding. You can now then know which gene you want to overexpress or knock out to see the effect of its impact. So yeah, I mean, you know, so these, the, this kind of important score that's given to any feature is important to understand, you know, interpret the results, the results, and then the machine learning model. But also, you know, this especially is modifiable features and combined with those that have a causal impact are the ones that, that are the most important in many, many applications.

So first, Sue Ann, great work. So someone's asked a question actually that was on my mind. I was going to ask if there was a lull, but I'll— so if you compare your importance, so the result of SHAP from the AI, how does that compare to, or how is it different from what you would just get from a linear regression type analysis where you would then look at the beta coefficients and are they correlated or is it just apples and oranges, they can't be compared?

That's also an excellent question. So it depends on the data. If the data implies that many important features and then the outcome have linear relationships, I think that then in terms of both of the prediction accuracy and then also the genes that are important, I mean, for the features that are important, I don't think they will be very different. But all the datasets we have been working on, which include both gene expression data and also various kinds of EHR data, they were very different to each other and we often found a lot of interaction effect. Say, if you, you know, plot the impact of the feature value, the relationship between, you know, feature values in the x-axis, in the y-axis if you plot this shape, then you will have a lot of, you know, vertical divergence, which means, you know, individuals who have exactly the same feature values for a particular feature have different values in terms of that feature's importance. It depends on some other features. So, and then we, you know, we found in many cases where, you know, the conclusion from this, you know, beta coefficients and this SHAP values are very different to each other. And then one more, you know, one very obvious difference is that this kind of SHAP values or other, you know, feature attribution methods, they will give you the personalized importance. So for each sample, yeah, for each instance, you will get this is set of importance values given to all features, which like personalized marker. Whereas in the linear model, the beta coefficient that's applied to a population of samples.

Thank you.

Thank you. Excellent talk. It looks like it's time to transition to our next presentation. So next up we have Dr. Sankara Raman, and he'll be talking about machine learning for large-scale genomics.

Hi, my name is Sriram Shankar Raman. I'm a faculty in computer science, human genetics, and computational medicine at UCLA, and I'd like to thank the organizers for this opportunity to present. So my lab works in machine learning applied to genomic and clinical data. And today I'm going to focus on a problem that comes out of the field of complex trait genetics, specifically trying to understand the genetic architecture of complex traits, which are a function of genotype and environment. And a major progress in being able to understand genetic architecture over the past 15 years has arisen from a study design called the genome-wide association study, or GWAS. And one of the big outcomes of these genome-wide association studies is the realization that complex traits are often controlled by hundreds, often thousands, of genetic mutations or SNPs. Despite the progress coming out of GWAS, we still have a lot to learn about genetic architecture. And this has been spurred by the growth of what we call biobanks. So these are datasets that collect genomic variation and trait variation across hundreds and thousands of traits, often in the context of a healthcare setting. A very prominent such example is the UK Biobank, which is this dataset that contains data from about half a million individuals. This includes genetic data as well as trait data across thousands of diverse phenotypes. So this leads us to the question about how we can learn about genetic architecture from datasets that contain hundreds of thousands and imminently millions of genomes and thousands of traits. So what are the machine learning problems and challenges that arise from biobank-scale data? Turns out that there are several such challenges. On the one hand, there are statistical issues. For example, how do we build machine learning models that accurately capture the aspects of genetic architecture that we care about? There are computational issues which focus on how we can scale inference in these models to deal with these millions of genomes. There are issues of interpretability which deals with how we make sense of the inferences that arise from these models. And finally, there are issues that focus on data privacy and data sharing. In today's talk, I will focus on primarily the statistical and the computational issues. And so to ground this discussion on genetic architecture, we'll focus on one aspect of genetic architecture, which is this quantity called heritability. Heritability refers to the proportion of variation in the phenotype that can be explained by a linear model of the genotype. Typically this is referred to as narrow sense heritability. And a powerful way of estimating heritability uses a class of statistical models called variance components models. So these models model the phenotype as a linear function of the genotype plus some environmental noise. And because of the high dimensionality of the genotype data, typically we have millions of SNPs that are measured, we need to put some assumptions on the effect sizes of these SNPs. So the assumption is that the effect size of each SNP comes from a distribution whose variance parameter is related to something called the genetic variance component. The environment is also assumed to come from an underlying distribution whose variance parameter is termed the environmental variance component.. And given the genetic and the environmental variance component, we can compute the SNP heritability. Now heritability is a single parameter. However, there have been a number of studies which have gone beyond a single parameter that describes genetic architecture. So there have been a number of studies that have looked at how heritability varies across across specific genomic loci, across functional annotations, as well as across traits. And each of these analyses have revealed interesting, important insights into genetic architecture. All of these analyses have been enabled by extensions of this variance component model. And so broadly speaking, the variance component model models the phenotype as a linear function of genotypes, where each SNP has been assigned to one of K possible components. And so in each of these components, the effect sizes of these SNPs come from the same underlying distribution. And our goal is to be able to estimate the variance components associated with the K genetic variance components. And the environmental variance component. The typical way of estimating these parameters is by using classical approaches like maximum likelihood, so writing down the likelihood of the model and searching for values of these parameters that maximize the likelihood. The challenge is one of computation. So typical maximum likelihood estimation techniques scale cubically in the sample size. And so this makes it challenging to apply to biobank-scale data, which has sample sizes in the order of hundreds of thousands, if not millions. So to tackle this, we have been developing an alternate class of estimation approaches. So these estimation approaches are based on an alternate approach to estimation called method of moments estimation. So these are estimates that try to match the theoretical moments underlying the model with the sample moments. It turns out method of moments estimators by themselves are still not scalable enough, and so we combine the method of moments estimator with the notion of randomization. So the key insight is instead of working with the genotype matrix, which is this large matrix, We work with a sketch of the genotype which is obtained by multiplying this genotype matrix with some number of random vectors. So the resulting sketch is a smaller matrix, so it is much more efficient computationally to work with. However, what is interesting is this sketch often preserves the statistical properties that are needed for the problem.. And so this sketch is accurate even for a small number of these random vectors. So the resulting algorithm has a computational complexity that is essentially linear in the number of SNPs, the number of individuals, and the number of random vectors b that we use to form this sketch matrix. Turns out these kinds of randomized estimators are accurate for b as small as. So this resulting algorithm is something that we term randomized HE regression. HE regression is a classical method of moments estimator that has been used in quantitative genetics. So we performed extensive benchmarking of this randomized HE regression estimator. The first thing that we find is across a wide variety of genetic architectures that were simulated, estimates from RHE tend to be quite accurate compared to other scalable approaches which are based on summary statistics of genotype data. And the reason we hypothesize is because the RHE is preserving more of the statistical properties of the original data to begin with. Further, RHE turns out to be much more scalable compared to likelihood-based approaches, and as a result, it is something that can be practically run on datasets with hundreds of thousands and millions of samples. We then applied RHE to a number of traits from the UK Biobank. In our first application, we computed the heritability of traits across different classes of SNPs So focusing on SNPs that are common and present in array data, looking at common SNPs on imputed data, and then looking at low frequency SNPs on the imputed data. And we find that the heritability for many of these traits increases as we include these lower frequency SNPs. We are also able to apply this method not just to estimate the total SNP heritability, but also to understand how SNP heritability is distributed across the genome. For example, in this analysis, we partitioned heritability across different functional genomic annotations and asked whether there was an enrichment of heritability in specific annotations. And we find that across traits, there is an enrichment of heritability in conserved regions of the genome. This has been documented in previous studies. However, we also identify trait-specific heritability in annotations such as the enhancers. We are also able to partition heritability across population genomic annotations like the allele frequency, the LD patterns of a SNP. And what we find is that the per allele heritability tends to increase as you have SNPs that are lower in allele frequency, as well as SNPs with lower levels of LD, consistent with the effect of negative selection acting on these traits. So we talked up here about heritability and an efficient method for estimating heritability, but what is interesting is that these general insights can be applied to a number of analyses that allow us to go beyond heritability and ask questions about the contribution of nonlinear effects, the contributions of environmental interactions, as well as how genetic effects are shared across traits. All of these are unique aspects of biobank-scale data. So more recently, we've extended this basic model to allow us to estimate nonlinear effects, specifically effects that arise from dominance deviation. So now we have a model which fits an additive variance component as well as a dominance variance component, and now we are able to estimate for— the number of traits, what proportion of the variance comes from additive versus dominance effects. And what we find is across about 50 quantitative traits, dominance heritability is substantially smaller, less than 1% of additive heritability. And across all of these traits, we find no statistically significant evidence for dominance heritability. We are also able to interrogate the contribution of gene-environment interactions. And the reason this is interesting is for many of these datasets, we not only have a trait of interest, we also have a large number of ancillary measurements in terms of environmental variables. So now we have a method that can also estimate these kinds of gene-environment interactions on large-scale data. And in this application where we looked at the environment, these be smoking, and try to quantify gene-environment interactions across a number of traits, we are able to document a substantial contribution of smoking in terms of its interaction to the genetic variability underlying the trait. Finally, we are also able to quantify how genetic effects are shared across traits. So this is a quantity that is termed genetic correlation. Because this underlying model is able to quantify directly the genetic correlation, it turns out we have greater power to estimate genetic correlation compared to existing summary statistic-based methods. So in this example, the application of this method to estimate genetic correlation identifies a number of novel statistically significant pairs of traits that are correlated in terms of their genetic effects. And because in this example application to the UK Biobank, we also have access to blood biomarkers, identifying genetic correlation across blood biomarkers and different diseases reveals a novel genetic correlation of coronary artery disease with serum liver enzymes, And that is particularly exciting because it's pointing to connection between liver and heart biology. So just to summarize, what I've discussed so far are a class of methods that scale machine learning models to biobank-scale data. And the key takeaway from this is the effectiveness of randomization in a principled manner. There are of course a number of other approaches that have been proposed in the field of scalable machine learning to scale inference to large-scale data. For example, there is approximate inference, which allows you to give up on guarantees of exact estimates for more computationally tractable estimators. There's distributed inference, which allows you to perform inference when the data is distributed, for example, in the cloud. So one of the things that would be of great interest is to explore further how techniques from scalable machine learning can empower and enable biobank-scale analyses. So in the remaining few minutes, I'm going to talk about some promises and challenges that arise from biobank-scale θ. So our big picture goal is to connect genetic variation to different health outcomes. What is extremely interesting in the context of biobanks is the availability of not just data on the health outcome, but data from multiple modalities. So this includes imaging, gene expression data or other molecular data, biomarkers, as well as sensor data. So a key challenge is to be able to devise and develop models that allow us to integrate different modalities while still being scalable to these large datasets. A second major challenge is to get at causality. So ultimately, we are often interested in being able to say whether some kind of an exposure, for example, a biomarker or a drug, is causal for an outcome. The gold standard way to do this is to conduct a randomized clinical trial. However, such trials are often expensive, sometimes unethical. So this leads us to the question of whether causal statements can be made when we have observational data. So in the presence of genetic data that has been collected, these genetic variation data often serve as instrumental variables that allow us to make these causal statements. So this is a technique that has become extremely powerful and has revealed novel insights in the form of Mendelian randomization. So Mendelian randomization has really been influential with the availability of these biobank-scale datasets. However, Mendelian randomization does require assumptions about the relationship between genetic variant exposure and outcome. And so violations of these assumptions can often violate the causal inference that stems out of Mendelian randomization. So for example, the kinds of assumptions involve the fact that there is no underlying population structure which serves as a confounder, there is no horizontal pleiotropy, which leads to another violation of the assumptions of Mendelian randomization. So this leads us to the question of how we can design Mendelian randomization techniques that are robust to these kinds of confounders? And how do we actually apply these techniques at the scale of biobank datasets? Another important consideration is one of generalizability. So what has become clear in the last few years with the ability to predict complex traits from genetic data is that the prediction accuracy depends critically on the match between training and test data set. For example, when you have these genetic predictors that are trained on primarily population of European ancestry, these predictors tend to have decreased accuracy when they are tested on populations of non-European ancestry. Further, this lack of generalizability goes beyond ancestry. For example, a predictor that is trained on primarily women tends to do worse when it is tested on a dataset of men. So this leads us to the question of how do we design techniques that actually generalize well across diverse settings? How do we design evaluation strategies that test the algorithms in the right kinds of settings? Especially in the context of these kinds of genetic predictors being increasingly adopted in a clinical setting, a good way of handling generalizability becomes critical. Finally, although there is growing amounts of data in these biobanks, there's always going to be the need to be able to pool and analyse data. And so this is where the constraint from privacy kicks in, where often there are restrictions on how this data can be shared. So this is where techniques from distributed and federated machine learning, which allow you to train and deploy models without having all of the data sitting in one location is going to get increasingly important. With that, I'd like to thank my group that has been driving a lot of this work, and I'd like to thank my funding sources, and I'd like to thank you for your attention. Great work.

Thank you.

Thank you for an excellent talk. So I'll kick off the questions just to get us going, and I'll maybe remind all the participants to please use the Q&A feature at the bottom to ask questions because Casey and I then can see them. So my question has to do with very simple in the beginning. So if you run the random HE regression to help save time, but for let's say height, let's just pick height, and then you just do the usual analysis, you know, you basically take, you know, you spend the resources to do the usual analysis. How do those two answers differ? I mean, I think you showed that graph, and I don't know if that was simulated data or real data, but I just wanted to pick one example where you've run a phenotype. I just arbitrarily picked height. You run it both ways. I assume the answers are very close to one another that got us to this point, but I guess I wanted to hear it just for one trait.

Yeah, so that is mostly the case. So what we have not been able to do is run it all the way to the size of half a million individuals. Instead, the experiment that we are able to do is subsample where the alternate approaches are feasible to compute, and then we can compare our results. And our broad observation is estimates from this randomized HE correlate very well with likelihood-based methods, which we can think of as kind of the gold standard, typically there's a slight increase in the standard error of these randomized methods, maybe a 5% increase, 5 to 10% increase in their standard error. So you do pay a little bit of a price, but often in the large sample setting, that is not the dominant error modality that we care about. We want our biases to be as low as possible.

And I see that you're answering some of the questions online. Please continue to do that. It looks like we're— it's time to move on to the next presentation, and we can come back to questions during the Q&A.

So our next speaker, it's a pleasure to introduce an old friend, Russ Altman from Stanford University. And Russ is going to be speaking deep learning to predict the impact of rare variation in drug metabolism genes.

Thanks very much. This is Russ Altman from Stanford, and I'm happy to be talking today about deep learning to predict the impact of rare variation, particularly in drugs that are involved in the metabolism of— in genes that are involved in the metabolism of drugs. So to make sure I thank the people who did this work, I want to particularly thank Greg and Adam in the upper left, who are the graduate students who worked on this project. Rachel and Erica in the upper right at University of Montana at the time collected the experimental validation data. And then, of course, the entire research group in pharmacogenomics at Stanford. So pharmacogenetics is the variation in drug response due to genetic differences. So the idea and the vision for pharmacogenetics is every patient will be sequenced and we'll know what variations they have in important genes so that we can figure out if the drug will work as expected, or we should change the dose of the drug, or we need to be on the lookout for increased toxicity, or maybe just use a different drug. And this is rapidly becoming a reality. Pharmacogenomics is in the clinic. In fact, I have a clinic at Stanford where I see patients, we genotype them or sequence them, and then we figure out what that might mean for their drug response. So PharmGKB.org, you can see it's cut off here, but PharmGKB.org is the resource supported by NHGRI and others, and it's about 21 years old, and it's filled with information about drugs that interact with genes. So, for example, codeine. One of our favorite pain relievers, is significantly metabolized by CYP2D6. This is an example of the data in PharmGKB, and in fact, on the left, you see it's Level 1A, which means we have very good evidence for this interaction. And this is going to be the example gene, CYP2D6, that I talk about in today's talk, and a little bit in the context of codeine. So there is a spectacular 3-dimensional structure of CYP2D6. I should say that's the cytochrome P450 Family 2, subfamily D, sub-subfamily 6. A beautiful protein that metabolizes— that's in the liver and metabolizes xenobiotics, foreign substances, basically to make them more water soluble so that they can be excreted. So codeine is shown on the left, and codeine is a substrate of CYP2D6, and it is transformed into morphine. And morphine, of course, is a very powerful pain reliever. In fact, codeine has very little impact on pain, and it must be metabolized to morphine in order to relieve pain. And the whole chemical trans— action is just changing the methyl group in the upper left, you can barely see it, in the codeine, to a hydrogen. That demethylation winds up creating this powerful medication. The important point for our discussion, however, is that as many as 23% of people in the U.S. have a compromised ability to metabolize codeine into morphine. And it goes in both ways. Some people are poor metabolizers, and codeine does not work because they never turn it into morphine. So they take codeine and it's like a placebo. Other people are what we call ultra-metabolizers. They turn the codeine into morphine too quickly, and therefore, have an unusual response where they get this high from the morphine for a few minutes, but then they don't get pain relief for the next 3 or 4 hours. So this is an archetypal example of pharmacogenomics and one where if we're going to practice precision medicine, we're going to need to know for every patient what is their CYP2D6 status and what kind of adjustments should we make in their medications. Now, CYP2D6 has a number of alleles. In fact, 161 haplotypes have been reported in the literature. And they have this funny nomenclature that some of you may be familiar with called the star nomenclature. Star 1 is typically a wild type. Star 2 is some combination of SNP alleles that lead to a variant. Star 3 is another set of combinations, et cetera, for the 161. And we're actually still discovering them as the sequencing projects do more and more. So, really, for pharmacogenomics, the challenge is to not just understand the role of CYP2D6, but for all 161 or 200 or 300 haplotypes and their pairs as they're observed in humans. We all have two pairs, obviously. To understand what the variant that a particular patient has means for their drug response. And this is not an insignificant question because, as many of you may know, CYP2D6 is responsible for the metabolism of hundreds of drugs. A small subset are shown here, and then some of my favorites are highlighted in red. So you can see some antidepressants, you can see tamoxifen, a very important breast cancer drug, ondansetron, which at Stanford is the number one prescribed CYP2D6 inhibitor 2D6 substrate, and then codeine, which we've been just talking about. And there are literally hundreds of others. So just knowing the activity of CYP2D6 for a patient could have a profound effect on your ability to give them the right dose of the right medicine. And in fact, we and others are involved in writing these CPIC guidelines, Clinical Pharmacogenetics Implementation Consortium, where we tell practitioners or advise practitioners how they can use genetic information to optimize their use of, in this case, codeine. But there's the problem. The problem is that these guidelines in general are addressing the common variations of CYP2D6. And we know that throughout the population on Earth are many, many haplotypes that are not in these guidelines. And those folks, people with those haplotypes cannot benefit from pharmacogenomics. In fact, in one study that we published recently in Rare Variants in the UK Biobank, we looked at 8 key genes, including 2D6. We saw almost 500 predicted variants that would be deleterious across these 8 genes. Most of the— half of those were not found in NOMAD, one of the resources for population variation. So these were somewhat novel. And in fact, 6 of the individuals in the UK Biobank carry at least one novel deleterious or predicted deleterious variant. And in fact, overall, each individual has an average of 12 drugs for which they might be expected to have an unusual response because of their variants that are seen in their genome. And of course, I must mention that the UK Biobank is not the most diverse resource, but we did see in the non-European populations contained in the Biobank that novel variants were enriched. So not only is it a question of bringing pharmacogenomics to the population at large, but it's also an issue of justice to be able to understand what these novel variants that might be more common in non-European populations, what they mean for those patients. So here is just a graphical depiction of the 8 genes that we looked at. Their allele frequencies, and on the far right, you can see a huge number that are simply— haven't been seen before, and therefore, and as pointed out at the bottom of this slide, we have no idea what they mean for the CYP2D6 function, and therefore, in the setting of codeine, we would have no idea if we should use this drug or not based on the pharmacogenomic background. So that's what I wanted to talk about today. Which is how can we predict the function of these rare or novel haplotypes that are observed in population surveys so that we can bring pharmacogenetics to all patients. So we use deep learning.

You've—

many of you know a lot about it. I'll just remind you that it's a new machine learning paradigm kind of based on an analogy to neural processing. For those of you who benefited from an education in biology, you will remember how the retina works, that we have light in this case coming from the bottom, but it hits the pigment epithelium at the top, the little square cells, those have the rods and the cones that detect light. Those cells are interconnected with a next level of cells, they're labeled here as the horizontal cells, which integrate signals from the rods and the cones in order to under— begin to see if we're looking at shadows, edges, corners. Those horizontal cells then feed into the next level, the amacrine and the ganglion cells, which further integrate and combine the raw data from the rods and cones to start to see image features like edges and lines. This then goes to the visual cortex where further processing happens, and then our brains recognize, you know, our mother, our car, our refrigerator. So that analogy is what inspired, in a loose way, deep learning. In this case, it's been very well— it is performed very well on image analysis. The pixels go in on the far left, on those gray nodes that you're seeing in the far left, each one indicating whether it's a— what the color is in the pixels of the— of an image. And then, on the analogy of the amacrine and horizontal and ganglion cells, We have all these levels where there's high integration between the neurons at one level and the neurons at the next level in order to start to see features. And you start to first see edges and corners, and in this, because they're looking at faces, they start to recognize— in the middle levels you can see facial features like eyes and ears and nose, and eventually you see faces and you can say, is that Joe Biden or not? And these things have become very good at this, and we want to use this for analyzing the variations in the DNA. We want to put the DNA on the far left, and we want to get the function of that CYP2D6 out at the far right, but we don't have a ton of data. We're going to use something called transfer learning. In transfer learning, you train on a different task for which you do have data, and then apply a small amount of data to make a special purpose classifier for the problem that you really care about. So let's say that I went to the internet and I found a ton of cats. I have plenty of cats to build a deep learner to recognize what type of cat is in a picture—Tabby, Siamese, etc. I can take that and I— and many of the image processing steps that it's doing are very relevant to dogs as well. So we're going to lock that middle section and then we're going to use a relatively small amount of dog data just to turn it from a cat recognizer to a dog recognizer. And this has been shown to be a very effective strategy. So in CYP2D6, our cat is going to be something called an activity score. This is a very heuristic score used currently to estimate what the function of CYP2D6 might be. It's based on looking at known functional haplotypes and saying, are they nonfunctional, a little functional, or perfectly functional, and then counting up points. And it doesn't work great, but it works pretty well. And importantly, we know how to calculate that score. And so what we're going to do is we're going to use that as our CAT We're going to train a neural network to predict the activity score. Then we're going to give it real data to predict the actual activities of CYP2D6. So here's the overall strategy, and you can see the reference to the paper, which just came out relatively recently. We generate 50,000 sequences on a natural background of NOMAD sequences of CYP2D6. Then we spike in known variations that is to say variations which we know what their activity score would be. Then we compute the activity score and we train a model simply to learn how to go from those sequences to the activity score. It's just— it's really just reverse engineering addition of these activity scores. But in the fourth point, you can see this forces our neural network— we're calling it a CNN— to learn key features of sequence that might affect activity scores. And other features that might not affect the activity scores. Then we're going to bring in our very valuable but sparse experimental data plus some database information to refine and create a dog classifier, if you will. And then we're going to predict. So graphically, we are going to take simulated data at the bottom here, put it through all those layers, and come out with activity score predictions. Then we're going to use the real data just to refine the green segment here. And then we're going to do the database information from PharmGKB and other resources to make a final predictor. So the green is the dog. I think you get the idea. So the data we used is from Erica and Rachel. They took 360 liver samples, sequenced the CYP2D6 with colleagues, and made 2 activity measurements per sample. They found 161 variant sites, and that is our very valuable activity data. And then, of course, we have some databases of star alleles where we have several that have been annotated normal, decreased, or no function. And then there's 71 for which we have no function. They cannot be used in the training because— oops— because we don't know the function. We're going to have to use those to evaluate the classifier. So just to be clear, what we give at the very beginning, the equivalent of the pixels of an image, are the A's, C's, T's, and G's in the sequence, plus some annotations, which I'll talk about in a second. We take them through all of our layers of connection, and we wind up coming up with two scores: the score for normality and the score for no function. That's just how we did it. You could have done it as one, but we did it as two. It worked better. And then I'll talk about that in a second. The annotations that we included are these. Is it in a coding region? Is it a rare allele? Is it deleterious per some of these algorithms that look at sequence information? Is it an indel? Some methylation or epigenomic markers, transcription factor binding sites, or a known active site? Of the protein. So that plus the sequence is what we give it. We do our little trick that I've already described. Then we take our normal score and our no function score and we make a little decision tree that tells us for each haplotype whether we're predicting that it's normal in green, decreased in yellow, or no function in red. That's how we use the output of the neural network. We take that blue and orange output, we put it through this decision tree in order to make a haplotype prediction. So here's the results. We were lucky because a group in Japan, shown in the lower left, published a relatively recent paper of 49 CYP2D6 alleles where they measured the activity. This was not available to us in training, so for us it was perfectly great validation data. We had never seen these before. What you see on the far right on the upper right is the different star alleles that they had measured. That should add up to something like 49, I hope. Maybe a little bit less, but it's 49 or so star alleles from this Japanese group. We have the predicted— we have the actual activity that they measured on the, on the x-axis, and then we've colored what our prediction was. Green is normal, Yellow is decreased. Red is non-functional. So I misspoke. Some of these data were in our training set and they're the triangles, but the ones that are circles we had never seen before. So we were very happy because as you can see, the green ones are right— are to the far right in general, with a couple of exceptions. The yellow ones take up the middle region and the red ones, the ones for which they measured Well, the red ones are the ones for which we measured no function, predicted no function, and they generally are either very far to the left or pretty far to the left. And in general, 71% of the variance in these measurements was explained by our labels. And so we took this as extremely reassuring and encouraging that these kinds of transfer learning approaches with small amounts of data do enable us to build these models. We looked at what each feature brought to the dance. So the full model is shown on the far left, and then what we did is we removed each of the annotations to see what the impact was on the performance. So you can say we get a big drop in performance when we provide no annotations. We also get a big drop if we don't do the transfer learning. We get an even bigger drop if we do neither. And then this shows that The fact that something is a rare variant was very important. The fact that it was predicted to be deleterious is important. And then the other ones all add a little bit of information as well. We also found— we went back to our uncurated star alleles. Some of those were curated during the time that we were writing the paper, so we reran the model. Again, none of this data— you can see they're all circles here, no triangles. This is all data that we've never seen before. And once again, we were very encouraged to see that the greens were for the most part to the, to the right. The yellows were in the middle with a little bleeding into the green, and the reds were predicted very nicely. In these, in these just two cases, we got them— we basically nailed that they were non-functional. The final thing I want to say is that we want to provide some explanation to an interested pharmacogeneticist or biologist about why we make these predictions. So what we do is for each star allele that we made a prediction on, that's in the y-axis, and the red, yellow, and green shows what we predicted. And then we have these balls that show which variants that we observed drove the prediction. So just to take one, star allele 15, which was red— that, whoops, The dominant— sorry about that. The dominant feature that drove that red prediction is this ball, this big black ball here, which is a frameshift at position 47. So at least we can tell our biologists something about what's driving these predictions, which is, of course, quite important where deep learning can be somewhat inscrutable. So let me end there. Pharmacogenomics is in the clinic. The UK Biobank and other resources tell us that there's a lot of people who cannot benefit from pharmacogenetics because we don't know the function of their rare or novel variants. These deep learning methods, such as I showed you today, hold a lot of promise for predicting which ones will be useful— hold a lot of promise for producing clinically useful predictions that will be useful in the clinic, even when they're not perfect. And that's something else we can talk about later.

Thank you very much. Great. Thank you, Russ. And so I'll kick it off. And I am surprised at one thing that I would— I'm not doubting it, just surprised is— but I'd like your insight. You know, there's this sort of rule of thumb that, you know, common variants with small effects, rare variants with large effects, you know, and there are exceptions. And so I'm a little surprised then, unless I misunderstood, which is also possible, that by training on common variants, you trained a model on common variants, the cats, you can then apply the model to rare variants, the dogs. Or is it that the training was done on both common and rare variants, I believe, because of the sample of livers that were used, if I understood correctly. But I turn it over to you to get your insight.

Yeah, it's a great question. Thank you, Eric. So first of all, and I don't mean to be glib, but variants are variants from the— with respect to their impact on the biology, the common and rareness of something has to do with evolutionary history and everything. So that's the first thing to say, which you know, of course. The second thing to say that I think is quite important is that these cytochromes are under a very weird kind of pressure. You know, they don't cause disease, so to speak, and they're very much— they've evolved in response to what our ancestors ate while they were migrating around the world. And so they are quite free, they're quite free to mutate. And unless the mutation is in some way really deleterious to some basic biological function, you're going to be okay. And in fact, folks who have non-functional CYP2D6s, they can do fine in the world because it's not critical for life. So, so a lot of what is common and what is rare is purely a, um, a historical accident and not because of very strong pressures. So it's drift and not selection for the most part. And I'm speaking a little bit generally. So that's the— those are the backup comments to say. And then I can— to answer your question, there would have been, yes, some rare variation in our training set, although it would have been dominated. But the machine learner, we didn't tell it about frequencies, and we only gave it one example of each variation. So we were trying to get it to learn biology, so to speak. We were thrilled that it learned that frame shifts, especially early in the sequence, are very bad. That was good news because now we can give it a frameshift that it's never seen, but it has figured out— and I don't want to be too anthropomorphic about it— but it knows that frameshifts, especially early in the sequence, are bad. And it also, of course, figured out to some extent that big changes in the amino acids, especially in certain areas like the binding sites for the small molecule substrates, are also very very important. So I think that we were heartened that it was able to learn these things, but our performance, as you all saw very clearly, especially for those yellow ones that can be all over the place, is not perfect yet. And so the next generation of these models has to look explicitly at 3D structure and has to look a little bit more carefully at the nature of the sequential changes.

So I'll stop there.

It's a great question. So maybe I'll ask one other general question, then turn it over to Kaci. There's a question in the chat that I think the whole group will be interested in. In pharmacogenetics, as you point out, things have moved into the clinic. And now, talking to a large group of machine learning kind of nerds, as we produce models, results that are predictive enough to move into the clinic, what do you think, how do you, what and how do you think the FDA is going to look at the results and will they start to be, will machine learning start to be treated as a quote, you know, sort of medical device that we'll need to think about how it can be validated in that context. And my question is not a question of your talk specifically, but is a translational question of how you think we will be interacting with regulatory agencies moving forward when our tool now is more stochastic in nature.

It's a great question. I know that the FDA is very interested, very aware of all of these developments, and is considering what its appropriate role is. They, you know, they have a very difficult balancing act between allowing the community to figure out ways to do this effectively, but to make sure there are not charlatans or even well-meaning folks who are just not adding anything other than noise. You can imagine trials, and I think it would be very reasonable to do, especially pragmatic trials where you compare. In my clinic, I have all these patients. If I get their exome data, I will see these rare variants. You could imagine a trial where we would use this not perfect but pretty good 71% machine learning to try to guide CYP2D6-based recommendations. Versus not. And what not would mean would be use the common variants only and ignore the rare variants. And then it's an empirical question, the degree to which we get better outcomes. Those could be difficult trials to do in a randomized controlled way. But I think that the danger level is not that high for many of these drugs. And so therefore, a pragmatic trial or a non-randomized trial might be cheaper and still justifiable. You'd have to go through that logic, obviously. And then the FDA will be left with deciding, do we insert ourselves to approve or not approve these machine learnings?

Or do we—

and this is what they've done historically— do we say this is like the doctor reading the literature or reading a book? And as long as a doctor is in between the machine learner and the patient, We're going to allow this to be part of the general practice of medicine that we're not going to regulate. I could imagine arguments on both sides. And so I don't know how it's going to go. And I can see the benefits of each way. But it wouldn't be surprising if they took a close look at this and said, you know what, we're going to have some guidelines about what we'd like to see before you start giving doctors advice. Because as a machine learning algorithm, it's going to be somewhat inscrutable. It could cause some problems. Problems. And I think you probably want to have at least light supervision of it so that we don't do the whole field injustice by having a few bad actors who are generating random advice bring the whole thing down.

Thank you.

I think you might be excellent. I'm on now. Thank you for a great talk. I think we'll transition into our discussion section. And, uh, so I guess we can bring all of the panelists back in the spotlight. Um, just as a reminder, this session is about machine learning and clinical genomics. And one thing that seems common across all the talks, we're considering genetics as well as other factors that can influence what we see in the patient and how machine learning and AI algorithms are useful in this context, and with reference to the usefulness in clinical genomics applications like drug synergy, disease presence, and drug response. The question I have for the panel to start it off is, what are some of the biggest challenges to translating these, the approaches that you guys were described clinically? And if you're advising NHGRI, what might be some things that they could do to help with research in this area of machine learning and clinical genomics?

I know I've been talking a lot, but let me just say that the one thing that I mentioned in my talk about the relative underrepresentation of certain ethnic groups is it must be addressed immediately. So I can do in my clinic a pretty good job with Northern European descent people in giving them the best for pharmacogenomic advice I have, but my confidence in that advice goes down very quickly as I deal with other populations, and I don't like doing that as a physician, and it doesn't seem fair or just. It's not. So I think that the hard work of characterizing variations in all populations and understanding, especially for the common ones, what their impact is, I think has to be an A number number one target.

I'd like to second Russ's point. Yeah, I think that's one of the critical challenges for us. And maybe another thing to be wary of is this principle of do no harm, where there is the possibility that we should be acutely aware of these models and algorithms kind of exacerbating biases. So that's something we need to be always aware of about how there's an interaction between the data that's collected and how algorithms can amplify some of the biases in the data.

So one more thing I want to add is that the term that most frequently appeared in many of the slides of the three talks and then many other talks was interpretability. The trustworthiness of machine learning models. Well, feature attribution is one of the many ways, and then also identifying causal factors. And then even what Russ just talked about is a domain shift. When you use machine learning trained based on certain population and tested in another set of population, you are experiencing domain shift, and then machine learning the model can capture something that did not appear in the training data set. Even that kind of behavior can also be captured by interpretability, the right interpretability methods. So I think the development of this, the machine learning methods for improving the interpretability or explainability can really important.

Thank you all. And one thing that came up in Dr. Altman's presentation was the value of using the knowledge resources and transfer learning. And one thing I'm curious about for all of your approaches is like, as evidence and knowledge changes, how does that change the performance of your approaches, or is it robust to those changes over time?

Incredibly important issue. We have to make the assumption that anything that is deployed clinically will be regularly evaluated with respect to the data it was based on. There's a very thorny problem, though, that once you deploy an AI system, it will affect the data that is generated subsequently. And therefore, there's a real risk that we become impoverished with truly independent new data from the clinic once the first generation of AI systems is deployed. And I haven't heard great solutions to how do we make sure that we don't basically taint all of the data because the AI algorithm is driving us in certain directions, which means other directions are unexplored. And potentially, by the homogenization of the advice, we actually lose the chance for serendipitous discoveries that we're doing things wrong or that things could get better. Um, I think this is increasingly going to be a big issue throughout all of AI, because once you deploy it— self-driving cars, medication selection, disease diagnosis, once you employ it, you really can't go back to the pre-AI days of kind of in-the-wild data.

Thanks. And that's great. Thanks. So there's another question about— that came from the audience that's, I think, relevant to both Dr. Li and Dr. Kankaramana. So it was about the performance of linear versus AI models. And it seems like there's consideration— well, probably for all approaches, there's considerations for performance in terms of feature importance or for heritability. And I was just I'm wondering if you all could comment on that.

I'd like to understand your question better.

So you mentioned the linear model versus complex AI model. Yes. So sometimes you can start with the simplest model and get really good performance, and you're OK. But since we're in this— it's part of this workshop, there are circumstances where machine learning algorithms may work better. And I was just wondering if you guys have had experiences with comparing the two and understanding scenarios where one kind of model works better than the other.

And that's an excellent question. So I have one story.

So linear models work really well in many cases.

And then sometimes when you have a larger sample size and then complex model works better, but sometimes the difference in terms of accuracy or test-to-test error, test-to-set MSE, it can be just incrementally better.

So we did some analysis. It's now published in Nature Machine Intelligence 2020 as a part of our TreeSHAP paper.

Synthetically generated data increase the amount of nonlinearity. So the powerful aspect of this complex AI model is the ability to capture complex relationships.

So when you have a set of X and then you have a Y, and then naturally in a lot of biomedical problems, it's complex.

We call many phenotypes complex phenotypes, right?

So which means that, you know, it's not really linear.

In most cases, And we see a lot of in clinical journals, U-shaped curve. There is always a good range and then it's by itself, it's not linear. So in our experiment, we showed that when we increase the nonlinearity, the complex machine learning model, for instance, the tree ensemble model or deep neural network, they performed better than linear model in terms of accuracy. But at some point, linear model did still well when there's some amount of nonlinearity between X and Y, but still picked up wrong features. So prediction accuracy or tested log likelihood or MSE, these are sometimes not the right measure to see whether linear model does well. And then we did the analysis using synthetic data 'cause that's the only way you know what the true features are. And then, you know, still the ability to model or capture the complex relationship, it's something that's really important in many of the biomedical problems we are solving these days. So I would still encourage to use, when you have, as long as you have enough sample size, using, you know, complex nonlinear model. And then you can now this area of explainable AI or interpretable machine learning, it's growing rapidly. Then there are so many methods to help users to understand the inner workings of black box models. I would encourage, keep using this model and then use this interpretability method in a post-hoc manner.

I just want to acknowledge your question, Casey. I'm sorry, Sriram, but So many students want to use deep learning because they know that there is a good job market for deep learning. And so it needs to be on their resume. But so it should be a rule in academia that you must try a linear model first because there's so much pressure to do something overly fancy for kind of non-scientific reasons that I do understand. But really, Su-In's comments just now were right on. You need to try the linear models and only move it when there's evidence of important nonlinearities in the solution space.

Yeah, so to follow up on what Suwen and Russ just mentioned, I think the linear models are a baseline, and that's kind of what you need to start with and show that these nonlinear models are doing better. And whether they actually do better is going to depend on the ratio of the number of features to the sample size and signal-to-noise. So in the example of complex trait genetics that we're dealing with, we are still in this high-dimensional regime where we have hundreds of thousands of samples, but the number of SNPs is in the millions and the tens of millions. And so what happens there, and this is an open research question, is we don't have good nonlinear representations of this million-long, tens of million-long genotype vector yet. I'm sure we'll get there, but we don't have that. So which is why, as a community, we have been dealing mainly with linear models and modifications to these linear models to try to introduce nonlinearities in a specific biologically plausible manner. But one thing I want to add there is even these admittedly simpler models can be really challenging to work with at scale. So there are interesting issues even with these linear models that need to be tackled at the scale that the data is being generated.

Thank you. So I'd like to have Sue in, talk a little, talk to us a little more about this concept of interpretability. So for those of us that are new to SHAP, How can we, how do we, or how should we interpret interpretability? Sorry for that. And is it wise or a big mistake to have sort of causality and mechanism in our mind or the back of our mind when we're thinking about interpretability?

That's a really good question. You know, in this feature attribution methods such as SHAP I presented, what it really does is to measure the influence, measure the impact of each individual feature on a certain prediction. It doesn't have any implication in terms of the causality. So the main question there is how this prediction was driven. So one of the big mistakes from this SHAP users or people who use this feature attribution method is that it has somehow causal impact. So downstream analysis needs to be done to nail down to the question of whether something is causal or not. And we all know that when we have observational data, so you are seeing most of the time in biomedical problems, we see this cross-section of just one time slice that we are measuring gene expression or epigenetic, and then we can never answer the question in terms of the causality. We don't have any interventional data. So in that case, there needs to be some other methods to actually answer, for instance, using prior knowledge or clinical knowledge. The things that are already known in terms of X influence Y, or gene— some gene X, you know, transcriptionally regulates gene Y. And then, you know, there needs to be some methods to, you know, explain these explanations. So we have some several, you know, follow-up papers on that. For instance, you can model your SHAP value based on various genes. Let's say each feature is a gene expression. You can model each SHAP value of a gene as a function of the gene's properties. Say that you want to predict cancer drug sensitivity, and then it can be gene's driver potential, how much it's mutated, epigenetically modified, and so on, to come with some certain explanations. There needs to be some further analysis to answer that question.

If I can just agree with Su-In, you know, one of the big problems with machine learning— and I know that the organizers of the session said that we should wind up giving some advice to NHGRI, and as Eric knows very well, I'm happy to give advice to people. One of the big things is deep learning has been very greedy for lots of data, and the fact is, in biology, with a few exceptions. Yes, we have genomes, and we have a lot of them, but in many, many important cases, we just are not dealing with the amounts of data. And so what Su-In just said about the importance of having background knowledge integrated into these machine learning systems is absolutely critical. There was a question in the Q&A, and I don't know how to use it, so I deleted it instead of answering it, but it was about what they call zero-shot or one-shot learning. In machine learning, this means you get it right, even though you've never seen an example before, which sounds like magic, but humans do this all the time. I have a 2.5-year-old grandson who has seen one cat, and when he looks at a new cat, he knows it's a cat, even though he's never seen it before. And so I think another area that I think we have to— that where biology has even more of an investment than other application areas is nonlinear machine learning in data-poor situations where background models and background data and priors, prior probabilities, are in a very strong way integrated. And of course, the community works on those, but I think that we have a special interest in this problem because of what's at stake and because even our biggest data is often not very impressive to Google or Facebook or Twitter. So that would be a research program in machine learning that I think we could take a unique unique lead-in.

So I'll draw you out a little more, I think, on that concept because you have so much practical experience in the EHR space where there is a lot of sparseness. And, you know, bringing together all of these talks and translation and thinking about interpretability and sparse datasets is— what do you see as the future of machine learning, again, in translational genomics for data in particular, moving away from what, you know, in sort of very carefully controlled, you know, or designed studies where there's very little missing data, you know, sort of longitudinal epidemiologic studies, there's just very little missing data. We've very carefully controlled environments for measuring blood pressure, let's say. And now you move over to the EHR space where there's a lot of missing data and often the measurements are not as carefully controlled, the measurement methods.

I think that is a great example because why doesn't that mess up clinicians? And it's because they have this background knowledge So in fact, Sue In and her colleagues just recruited an excellent new faculty member named Sheng Wang, who was in my lab. And what Sheng showed is the ability to do this zero-shot learning, where he didn't do it for patient medical records, although there are students now in the lab trying to do this. But he showed that when you have a structure of a field, for example, an ontology, that when you look at data, you can say, Oh, I've never seen one of these, but I know it exists and it's a pretty good match. I'm going to call this, you know, a liver cell, even though I've never seen a liver cell. And so I do think that not all missing data is equivalent. Some of it I think humans impute almost implicitly. So docs, when they're looking at the chart, they don't see certain measurements. They are very good at guessing what those measurements might have been. And so I think that part of it, deep machine learning should be able to do. The problem is When the missing data is missing and critical and very difficult to predict from common sense background knowledge, that's the harder case. But I am bullish on these methods eventually being able to fill in a lot of that background missing data. I'd be very interested in my colleagues' thoughts on that.

Yeah, I think Ross gave a very good answer. And then just to add, We've been talking about supervised learning methods mainly today. There are a lot of unsupervised methods to understand the structure of the data. I see many papers on embedding, learning low-dimensional embedding from EHR data. What that can do is that when you have very well-controlled study data, then you can use that data as a way to impute this missing missing values in the datasets that are relatively not dense. So a lot of this method for learning the structure of the data, unsupervised or semi-supervised learning methods, should also be developed in this space.

Good. Yeah, I think there's a lot of value in methods which can essentially take small numbers of carefully collected data and extrapolate that to settings where there is noisy data that's collected. So I think that the kinds of methods that Sooin alluded to in terms of learning common embeddings, those seem to show a lot of promise in being able to do those kinds of extrapolations.

There are a couple of questions in the chat that I'll kind of summarize, but the bottom line is the ability to incorporate longitudinal data into machine learning methods. Could one of you speak to sort of the methodological task or challenge of incorporating longitudinal data?

I'll just say that you're getting a lot of silence because in my career I have tried— whenever faced with longitudinal data, the first thing you try to do is make it unlongitudinal. But that's the wrong answer, but it just simplifies the analysis. But let's just acknowledge that this is absolutely critical because disease trajectory transcriptomic trajectories, developmental trajectories. They're all absolutely critical. They take your big dataset and all of a sudden make it look very small. So part of the answer is going back to the previous discussion about prior models, because gigabytes of data, when you then map it out over the timeline, becomes megabytes. And then people say, oh, I can't learn anything from this. So it's, I think our silence is only, I'll speak for myself, embarrassment that the field has not been able to really make a lot, a ton of progress there. Now, there are people on Wall Street who are very interested in AI and following the temporal trajectories of stocks. So this is an area where we should probably look to Wall Street to see if they come up with anything good. But so far it's very short-term and it's not very causal, which gets everybody very nervous.

So yeah, I have a similar thought. I actually think that longitudinal data is not a challenge, it's actually an opportunity. So for instance, think about causal inference problem. In longitudinal data, you have a little more hint, right? I mean, doctors do something and then after future time point, you see the effect of it. So you have a little more hint about causal relationships than just one slice, time slice, the observational data. So I think the main challenge is that in the medical domain, that kind of data are less available because of patient privacy issue. I mean, this is what we are experiencing in general, an EHR data field where we analyze EHR data. So I think that I'm viewing that kind of data set as an opportunity. And there should be more development of the method. For instance, causal inference and also how to do explanation better. Because neighboring time points, they must have similar explanations. So I think there is a lot of opportunity there.

We should credit NHGRI for the All of Us, the whole NIH, the All of Us effort with a million people followed over time, plus the other biobanks internationally. We really won't be able to ignore this problem, and hopefully there'll be some good new methods for watching these patients over time.

It also depends on the density of the time. So there are some kinds of time series data which is let's say you're getting blood pressure monitoring, you have very dense time series data, whereas if it is getting somebody's CT scan, then the density, it's much sparse. So you need to be careful about how you integrate across different time scales.

Yeah. So, Kaci, I'll give you the last word there, and then we'll turn it over to the organizers for the wrap-up.

Yes, I see that we're in our last 2 minutes. I was— this last discussion is very interesting, and I'm realizing that there's like opportunities for collaborations across clinical and genetics, genetic research, just because you see a lot of analyses that are either genomics with minimal phenotypes or very phenotype heavy with little genomics. But that's a kind of an opportunity I'm seeing to cross those two areas more deeply. Thank you all very much. Wonderful presentations. And great discussion. So we can turn it back around, back over to the organizers.

Thanks, Casey.

Wait till my co-chair Mark gets back up here. Yeah, so what a fantastic workshop it's been. I'm sitting here reflecting, making notes, and The quality of the talks has been fantastic. The quality of the discussions has been at least as fantastic after the talks. It's also been nice to see all the appreciative feedback we've been getting over a variety of media channels, including email. At first, I was sad not to have this workshop in person due to COVID. Now I'm not so sure. If anything, I think what we've seen is the power of Zoom to reach very large audiences. People might have noticed that at any given point, there were between 500 and well in excess of 1,000 people listening. And that was just the ones we can see who were directly logged into Zoom. And the attendance, I think, easily spans from card-carrying machine learning biologists to the general public. What Mark and I want to do in the last few minutes here, we have about 15 minutes left here, just to summarize some of the brief themes we saw today, kind of like we did yesterday at the end of sessions 1 and 2. Let me try to do that now for summarizing briefly the, the, what we saw today in sessions 3 and 4. So, so the first observation I had was we saw lots of examples of how large NIH-sponsored datasets have contributed to machine learning science. I think starting with Alexis's talk at the beginning of session 3, she really highlighted very nicely GTEx as a major data resource. That came up actually at several points in the discussion today about arguments for and against investments in those large resources. I think we also heard similar discussions about Roadmap and some others. So it's not just GTEx, of course. So that was point one, is these large datasets really do help stimulate the field. The other thing I think Session 3 did was to start to outline some of the central problems people are thinking about in genomics with machine learning and problems they should be thinking about. Again, Alexis nicely highlighted this variant function problem that has long been a central challenge of genomics. And now we're seeing deep learning and other associated technologies really making inroads in variant-to-function, also in integrating other data types like expression and the awesome power that she showed of doing that. We also had a nice discussion in various talks today about some desirable characteristics, Mark and I have decided to call them, of how machine learning studies should expect data to be structured and organized. So what are some characteristics of the data that make machine learning easier and more facile? So one, of course— and some of these may sound obvious, but they're worth stating anyway— so one, of course, is the data have to be fair and, most important, readily available to researchers to access those data and to to build analytical approaches against those data. The second point we wanted to make and that we heard was the high value, not just in the raw data, but in the processed datasets. And so deep sea was one example that was mentioned several times where that had been done to great effect for the research community. Also associated with this is good metadata. So again, not just the raw data, not just the process data, but annotations on those data. One never knows how they could be used in future projects. I thought Anshul Kundaje in his talk had a really good point about being transparent about the limits of machine learning and data, the blind spots, the biases, the pitfalls. And so as applied to datasets, it's always nice when dataset providers are transparent with respect to what they think the weaknesses are. So with that, I'll let Mark— I'm about halfway through, Mark, so I'll let Mark take over and finish the summary.

Thanks, Trey. So another theme that we noted that was discussed a number of times today, I think, was the emphasis on using machine learning not only to make predictions but to try to gain novel insights into biology. And some of the implications of this we saw in the talks were an emphasis on interpretable machine learning, We saw a number of talks that brought up using causal inference methods, since of course very often what we care about in biomedical applications is understanding mechanistic relationships between inputs and outputs. And then Anshul made a very insightful point about, in this context where we're focusing on gaining insights, that the usual performance metrics may not really capture what what we care about, and there's probably a whole different set of performance metrics that need to be considered and applied in that context. Another theme that we saw a lot today and tied to the last one, and this really carried over from yesterday, was a lot of emphasis on interpretability. So today we saw both Anshul and Russ Altman applying DeepLift. We saw Alexis Battle and Greg Cooper talking about graphical models, causal graphical models in the case of Greg Cooper, and Su-In Lee talking about CHAP. So a variety of methods for peering inside the black boxes of these learned models to try to understand why and how they're making their predictions. And then again, related to this, we saw the interplay between machine learning and causal inference and the complementarity there. So Greg Cooper talked about the TCI algorithm for causal inference. In one of the later talks by Dr. Sankaraman, we heard about Mendelian randomization. And then Greg Cooper made this very important point that that we want to keep in mind that most machine learning methods are learning associational relationships and not necessarily causal relationships. And of course, a key criterion, a key driver in many of these studies is to get at causality, as we've talked about. So I think that the talks, the discussions have given all of us a lot to think about. And a lot for our colleagues at the NHGRI to think about in going forward in their sponsored activities. So I would just second what Trey said. I think it's been an outstanding workshop, and we would like to thank the speakers whose talks were uniformly excellent over both days and all sessions. We'd like to thank our fellow members of the NHGRI Genomic Data Science Working Group, many of whom were session moderators throughout the workshop. Thanks to the audience members for your participation, for offering lots of very excellent questions. We want to thank the people behind the scenes who kept Zoom and the other technology aspects of the conference running very smoothly, and also the organizing committee at the NHGRI. Which includes Carolyn Hutter, Valentina Di Francesco, Shurjo Sen, Chris Wetterstrand, Natalie Kutscher, and Sean Garron. And thanks again everyone for your participation in the workshop and have a great day.