Scientist Stories: Dana Pe'er, Single-Cell Gene Regulation

Hello everyone and welcome to the distinguished seminar series of the Cambridge Center for AI in Medicine. Before I'm starting to introduce our distinguished speaker today, since our center is relatively new, I would like to say a few words about the center. So I'm Mihela Vanderschaar and I'm the director of the Cambridge Center for AI in Medicine. Our center aims at developing pioneering AI and machine learning technologies to transform biomedical science, medicine, and healthcare. And the unique sweet spot of our center resides in the interdisciplinary cutting-edge research that we are doing, where we believe that medicine drives innovations in machine learning and artificial intelligence, and conversely, that machine learning and artificial intelligence drive a revolution in medicine and bioscience. We also are joined in this endeavor by two industrial partners, AstraZeneca and GSK, and we believe that through these industrial collaborations, we can achieve a true impact in this area. The main four research pillars of the center are biomedical discovery, omics analytics using machine learning, delivering precision medicine using machine learning and artificial intelligence, and machine learning-enabled next-generation clinical trials. Our center has had its inaugural event not too long ago. On the 22nd of January, we have had the inaugural event. If you want to hear more about the center and the activities that we have started, and engage with us, please take a look at the inaugural event and get in touch with us. Today, it is my great pleasure to introduce our distinguished speaker, Professor Dana Pe'er, who is a chair and professor in computational and systems biology program at the Sloan Kettering Institute. And it is truly my pleasure to, to introduce Dana today. She's a leading researcher in computational systems biology, and she has a fantastic track record and a large number of awards, inclusively a Packard Fellowship, the Overton Prize, NSF Career Awards, NIH Director's Innovator Award, and many others. What is unique about Dana is that she started her, um, her studies in mathematics and then she moved to computer science. She has done her PhD in computer science and graduated from Hebrew University in Jerusalem, but since then she has completely made the transformation and she is not only making cutting-edge machine learning methods, but through innovations in computational biology. I've had the pleasure to listen to Dana give a fantastic NeurIPS 2019 keynote talk, and I was very inspired by her work at the intersection of these two areas. So I'm delighted to have the opportunity for all of us to hear her, and I'm sure we can learn a lot in our center from her research and her journey. Into the unique and interdisciplinary area. Thank you so much, Dana, for joining us.

Thank you for inviting me. Calling me someone that really innovates in machine learning is a little bit of an exaggeration. Learning all the biology that I have has come at a little bit of a cost. But my true belief is certainly if you want to get the stuff to work for medicine, that you really— it's all about your modeling. You can't— biology is so complex and so noisy that you can't take a machine learning nuclear hammer and swing it wildly, thoughtlessly. And the juice is how you interconnect biology, medicine. That little interface of the Venn diagram is where the juice is. And what I hope to do today is to show you some— my thoughts on how I approach modeling and some key things you should be thinking about when you're modeling biomedical data with some very visual examples. And because it's AI, I will have deep learning, but it will be at the very end. So right now, with all the technologies, particularly the single-cell technologies, we can finally ask this age-old question that was even posed by Leonardo da Vinci. How do we develop from a single cell into 40 trillion cells that form the human body? What are these cells? What do they do? And, you know, a lot of disease goes awry because they hijack these developmental processes, certainly in cancer, or because they didn't work right in the first place. So, you know, that's sort of a big goal that really drives a lot of the way I approach and question biology. And, you know, when I first jumped into biology, I was extremely naive. I was trained as a computer scientist, and, you know, in fact, there was a lot of things I would have never tried if I had known how complex biology is. And at the beginning, my naivete, I think, you know, powered me forward because I, you know, did things which, knowing the complexity, I would have thought otherwise impossible. And I always thought of computers, cells, as tiny little computers, and instead of transistors and Wires, they have molecules and signaling proteins and genes and transcription factors. And that drives the decisions. And they, you know, feel their environment and make choices about proliferating and differentiating and choosing a cell type and choosing a response to a stress or to a trigger. And basically cells are like little computers and we want to learn what they're doing. And now that we have single-cell measurements, if we think that this, that this computation is happening at the molecular scale, and we have many examples of single cells where we can measure the molecules in these single cells, then we can assume that if we look at the distribution of the molecules as a multivariate distribution, we can say, OK, if A regulates B, molecular influences should create statistical dependencies. And even vanilla Bayesian networks managed to take 2 decades of biochemistry and learn it from an experiment that took an afternoon and a computer, an old computer from 2004, less than a minute to compute. And it really recapitulated all of the known 2 decades' worth of biochemistry of what protein, what influences what protein. And we even managed to learn new connections that hadn't been discovered by biochemistry. Which validated. And the really wonderful things about single cells is that, you know, if you collect enough of them, we can actually learn patient-specific networks. We can take samples directly out of a patient, measure enough cells in that patient, and using the single-cell technologies, it's a CyTOF or single-cell RNA-seq, and try and learn patient-specific networks. So from this very simple example of like T cells in a dish, we're going to jump all the way to the T cells in the clinic. This is joint work with Cathy Wu at Dana-Farber, led by Elham, who was my postdoc, and she's now independent faculty at Columbia. And what we wanted to understand is why some patients respond and some patients don't respond. Immunotherapy is the big rave and the big rage, and it really helps and saves the lives of many patients. But then there are many other patients where, you know, we just can't do anything for them and it doesn't work. And we simply don't know why. Why does immunotherapy work for some and not for others? We don't even know what it's doing. And one of the first forms of immunotherapy was actually donor lymphocyte infusion. When, you know, after a stem cell transplantation, the leukemia returns and you reinfuse the T cells from the original donor. And the immune system managed to get reactivated and fight the cancer and remove the cancer in some cases, but not others. So we wanted to understand what is the impact of this therapy and what differs between these responders and these non-responders. And the reason we went with this one is because we had, you know, well-annotated controlled longitudinal studies, which is very rare to get in the clinical setting. So one thing is when you have data from the clinic, and this is really important, the data is really messy. You can't control it as well. You know, the, the surgeon is focused on, on the patient and not on getting you the perfect data with the perfect, uh, protocol, uh, that's perfectly replicated. And there's just a lot that goes on. So we work a lot with clinical data, and it's some of the messiest that you have because each sample cannot be equivalently processed. And so when you first process the data, you have massive batch effects that are formed not by the technology itself but by the sample handling. And actually, each dot here on this t-SNE— what you're looking at is a t-SNE image where each dot is an individual cell colored by the patient that it came from. And you'll see a lot of these images where each dot represents a single cell. And you can see that there's these patient-specific regions, and they even have biological interpretation. There's different biological processes. The yellow cells have a lot of interferon going on. And one could even give a patient-specific interpretation and misinterpret the data. And one of the things I like to see, particularly in immune cells, is good mixing. So if you look at this data back at the high dimension and say, what are the most similar cells to a given cell, do they all come from one patient? I call that low entropy. Or are they mixed well among the patients? And that's what you want, certainly, in immune cells. And batch correction is not yet solved. Every single batch correction algorithm makes very strong assumptions. They all have trade-offs, and there's simply no free lunch. But one of the favorite assumptions I think is most biologically motivated is the one that's made by BISCUT, which was also developed by Elham and Sandhya. And Biscuit basically says, OK, a gene-gene relationship— when we see this sort of covariation between a pair of genes, that represents important biology which we hope is preserved across these different batches. Batch effects are often independent of these critical biological relationships. And so if we assume that each cluster, each cell type should be represented by these important gene-gene relationships, and we're trying to basically learn the gene-gene covariate structure of each biological cluster or cell type, and then correct the data so that all the different samples and batches align and share this gene-gene covariate relationship. And one of the reasons I like this approach is because it's using a biological anchor to try and match across these samples. And indeed, when you run Biscuit, it really improves the clustering. You get beautiful patient mixing. Now you have clusters that really capture the full gamut of immune cell types that you're supposed to find in the tumor immune microenvironment. And on the bottom, you see here that the entropy is high. Every dot has a lot of samples from a lot of cells. The downside of BISCUT is that it's computationally extremely heavy. And so I only use it in these sort of more patient scenarios. Another important thing to note when you're taking data out of tumors and when you're actually working with clinical samples is that the immune cells should mix. And if the immune cells aren't mixing, the tumor immune cells, then you have a severe batch effect. But actually, the tumors are supposed to be very separate from one another. Each tumor is its own little beast, and you don't want to correct them. So we often split apart the immune cells and only correct them. The really sad part is that every cell type has individual batch effects, so it's sometimes hard to learn batch effects in one cell type on another. I did this little, you know, oh, and finally, you know, this is so strong, Biscuit, that even if we take two sets of patients and measure them with two different technologies two years apart, measure a different part of the gene, the 3' end and the 5' end, processing the samples very differently, this covariate relationship stays, and we can find the same immune T-cell clusters mapping across these two datasets on almost one-to-one basis. And so we use this and we could look at our CML samples and identify all sorts of very specific immune subpopulations. We found 43 different immune populations, which ones are diagnostic and are predictive of response pre-treatment. We could look at the dynamics after therapy in all these populations and see which ones expand the most, most consistently, and the responders after treatment. And we could learn a lot about this cohort once we could normalize the data. But we wanted to understand why. So we had also collected ATAC-seq. This was before the single-cell ATAC-seq kits were ready. And ATAC-seq, now, single-cell RNA-seq measures the RNA, what the cells are doing. And the ATAC-seq measures the open regions of the regulatory regions and really measures what's regulating them. And we said, okay, what's driving these differences? What is the underlying regulation that drives these differences? So here, this is work with Elham and Cassandra, and a symphony is really a generative model that says, okay, I'm going to look at a ATAC-seq, and remember, this was bulk, so we had to deconvolve the bulk into each of its cell types. So there's cell type-specific regulatory peaks, which aggregate together to these observed bulk replicates. But of course, you could also just shove in the single-cell ATAC-seq data and skip this deconvolution. We can run it in both modes. These peaks are prior for the possible regulatory pathways of how transcription factors are regulating targets. They're only priors. There's only suggestion. The other thing that's sort of driving the regulatory pathways is that this gene regulation should explain these observed covariate relations in the data, which I just showed you is very strong. And this covariate relationship is exactly what we can measure in our single-cell RNA-seq data. We can cluster it and look at the covariate structure between pairs of genes and beyond in each and every cluster. So we have the bottom as the observed thing, and we're trying to learn a generative process of the deconvolution of the peaks the regulatory functions they suggest, and how these drive the covariate structure observed in the data. And remember, we don't need a motif in the binding site for TF to regulate a target. It's just a prior on that. So we put this all together into a nice plate model. There was a lot of details in how to build the priors and how to estimate good initial conditions for these priors from the data. And basically, we have a model that explains the generative process in a way that mimics regulation. And again, I'm stressing— and this is often— ATAC-seq is used too strongly to drive these patterns. And here, it's only a prior. And we applied it to our data. And we managed to pick up all the key known regulators of these processes and a lot of new regulators. I might not believe every single edge that's predicted by such a model from such noisy data. But if a transcription factor is repeatedly predicted as a strong regulator of important immune genes, I do believe that signal. And again, we picked up some of the master poster child regulators of these subsets. In fact, TOX was unknown when we first ran this, and suddenly it was all over the place with Science and Nature papers because it is a key master regulator. And the reason I'm raising TOX this way is because TOX doesn't have a known regulatory motif. So if we would have been too strong on the prior, we wouldn't have been able to learn any regulatory function for TOX, which is a critical transcription factor. And of course, these do break down into these regulatory pathways that are very different between cell type to cell type, between responder to non-responders. We can even look at the covariation patterns in the data that's driving these inferences as well as the motif binding that's driving these inferences. And our immunology friends that are looking at these networks are very excited. And the hope is that there's some leads here for some new targets that might help boost immunotherapy. And while they're heavy and while they're old school, what I really like about Bayesian graphical models is that first of all, I can encode a lot of my prior understanding of the biology. I can model that into these models, and they lead to very interpretable models that I can later interpret and understand biology. In order for these to work, of course, if I'm putting in model assumptions, these modeling assumptions, you know, they better be right. So I'm gonna— this is actually old work that I I haven't spoken of in a while, but I just thought it's a great example for here, especially since we're talking about these covariate relationships and what you assume about the data and what the data might look like. So you assume— I've been telling you if A influences B, we should see this nice relationship. We should see an influence of A over B as a correlation in the data. I mean, even if not a linear correlation, we should see a strong relationship. Relationship, right? That's the assumption. You all agree with me. And this was done by Smita, who is now at Yale with Gary and Stanford. Here's real data. We're still working with T cells. And CD3 zeta is a major strong activator of SLIP76. The x-axis is a very strong and potent activator of the y-axis. Let's look at the kernel density information. Hmm. It doesn't look like a very strong function. The xy relationship is obscured because almost all the data density is on one value of— or a small range of value of CD3z. So what happens to this data? This is the same data, but instead of looking at the joint distribution, I'm going to show you the conditional distribution as I condition it on x. So now all we're looking at is what does y look like conditioned on x? That's the very definition of a function. —and this captures the behavior across the full dynamic range. Now we see this beautiful sigmoid. Now we see this beautiful response that was missed when we were looking at the joint distribution. And of course, if we want to quantify this, if we want to automatically pick up relationships like this from data, if we want to compare one condition to the next, then we have to have our sort of mutual information-like score capture this range and resample the distribution across the conditional density instead of the joint density so that we can really pick up this mutual information and see the amount of information from x to y conditioned on x. So we really want to model the conditional rather than the joint distribution. And just to show you how this really captures the edge strength much stronger, now I'm looking at two scatter plots before stimulus and after stimulus. Remember, this is like we hit the T cells on the head, we activate this pathway, we should see a big difference. The scatter plot, to my mind, looks the same. And really, correlation and mutual information and all these standard metrics also agree with my eyes. In the kernel density estimate, we see a shift of the entire population, but no big shift in the information or in the correlation between these two. But when we do DREAMy, when we condition on the x, we suddenly see a much clearer signal. And the signal isn't just, you know, SLIP76 became higher because, you know, CD3z became higher. The influence of X on Y became much stronger under activation. Given the same level of CD3z, we have a much higher activation of SLIP76 when this is activated. And so this is, I think, an important point. And if you take home one message from this talk and forget everything else I said, zonk out at this point. It's this: in biology, often the rare rather than the average cell matters. This right here, where all the action is happening, that's 2% of the cells. Many methods would call this the outliers. But when you measure enough cells, these outliers are 2,000 cells. And when I see a relationship like this in 2,000 cells, I believe it. I take that seriously. And so I really wanted to jump in a little bit on gene-gene relationships before going into cells, because in the past, we're all about cells. And we're all about these sort of t-SNE maps and UMAPs, where we plot each dot as a single cell. And the cells have now become our entity. But it's actually these gene-gene covariate relationships that are driving this structure in the data. So the data, the cells reside on this low-dimensional manifold, which is being driven by the cell phenotypes. And these densities are not uniform. The density can change 5 orders of magnitude across different cell types, and many methods out there don't take that into account. And so a little recap, really molecular influences are creating these statistical densities. The gene-gene covariation, that's what's driving the structure in our data. We can learn gene regulatory networks from it, and never forget rare populations are important. The thing I really love about biology, the thing that, you know, made me, you know, once the single-cell field started, I'm not willing to touch anything else, is the asynchronous nature of tissues. In this data, we don't have only different cell types, but we have continuous. We can get dynamics out of a single sample. If we take a single sample of bone marrow out of a person, We have the full gamut of hematopoiesis. We have all the earliest hematopoietic stem cells and all these different, more mature cell types, and we can get the continuum and every step in the middle. We can look at a process of epithelial to mesenchymal transition, and we not only get the two endpoints, but we can get every intermediate transitional state. We can order it along this transition and actually learn the regulatory dynamics from one single timestamp. And that's my favorite thing about single-cell data in biology. And here's the same data before, and we see B cells developing in this bone marrow sample. And one of the things I wanted you to notice is how the progression of development twists and turns. That's why we use nonlinear rather than linear methods in biology, because markers in development rise and then fall. So this yellow edge looks really close to each other, but it's after marker A had risen and fell before marker B goes up. So if we really want to capture these transitions between cell types, these developmental trajectories, we really have to capture them in the lower-dimensional manifold as these sort of long, nonlinear twists and turns through the data manifold. So the first time we saw this, this idea of pseudo-time, that we can get time out of a single time point, this was with Alain David-Hamir, with Gary Nolan. And single cells, for those of you who don't know, are modeled as a graph. So each dot is a single cell. And we can connect most similar cells. And now, instead of going in the high-dimensional Euclidean space where I don't trust long distances, I can take short distances and take paths between cell types where I trust these short distances. And I sort of almost view this graph as a mesh on the data. And one of the wonderful things of modeling this as a graph is it becomes dimensionless. And you don't have all these problems, statistical problems of high-dimensional geometry. And so here, we managed to basically order the cells along this pseudotime and find the timing and order of key events along differentiation. What was critical here was to collect a lot of cells so that we could again focus on the rare cell types and have an accurate algorithm. This signal is so strong that even PCA sort of works, and there's many pseudotime algorithms out there. I mean, you could do something really dumb and get something that looks sort of right, but it's a little bit more work to get— you know, there's a little bit devil in the details to get something that's very accurate. In this case, we got something that was so accurate that it managed to find this population right here, this early progenitor population that's only 7 in 10,000 cells. And what we found out that's happening in this population, which hadn't been previously identified in humans— we couldn't find it in humans— is that this is where the B cell receptor shuffles. In these cells, you actually have the DNA break and reshuffle in creating a novel combination of the B cell receptor. And after that occurs, there's a checkpoint to make sure that happens okay. And there's a signaling, you know, JAK/STAT checkpoint, IL-7 JAK/STAT, that happens only in this tiny population. It doesn't happen in the population right before it. It doesn't happen in the population right after. There's a unique signaling checkpoint right in here, right in this tiny 7 out of 10,000 cell population. Just to give you a feeling on how rare it is, because that's again a point I want to make, this is our data. This tiny little barely seen red slides is the progenitor B cells. Opening up this pie, this tiny pop, 3 cells, that's the population that we identified. That's the population that has this unique signaling. And, um, this breaks in pediatric leukemia. So this is the exact signaling pathway that goes wrong and children get pediatric leukemia. And by understanding normal development, by identifying these rare cell populations and signal pathway that should be happening there, we can understand what goes wrong in disease and pinpoint the cell type that went wrong in pediatric leukemia. Again, another example of this really rare population being the one that really matters, the one that leads to pediatric leukemia in these poor kids. And so taking this idea forward, this was old work, is really, OK, now how can we really use this and really understand development? And this is work by Manu Sethi, who's now at the Fred Hutch. And here, I like to model self-weight probabilities as a mark of change. Basically, you have the undirected graph, that neighbor graph that I just showed you moments ago. And you could use pseudotime to build some type of orientation of this graph. And then you have a directed graph. And the moment you have a directed graph, there's all the beautiful theory and math and power tools of Markov chains. I know they're not deep BI, but they're pretty cool. But here, the details matter. How you build this graph, what cell-cell similarity kernel do you make, and waypoint sampling. And again, I'm going to show waypoint sampling as an example. Because this really points to the idea that the cell manifold has very different densities. And if you just do random sampling, you get these sort of very dense regions, but you don't get the transitional states. You don't get where the action is happening. And if you do this sort of min-max sampling here, then you actually get the waypoints. And a shout out to Milot that uses the same min-max sampling as a wonderful way to look at differential expression of these cell types on the manifold. And I was really happy to see this algorithm developed in Cambridge using the same min-max sampling. And I think it's the best differential expression algorithm out there right now. So I wanted to give it a shout out since it's local to you guys. Just to sort of show you on the manifold, typical methods really go after the density. And you really get all these rare and important cell types. And I hope I've convinced you that all the good stuff, all the important stuff, often happens in these rare populations. So you really need to capture the entire manifold when you're modeling it. Once you do that, once you build a good chain, then you have the terminal states as absorbing states. And you could just use an absorbing Markov chain. All you have is a closed-form solution. Linear algebra is at your disposal. You power your matrix. And I love watching algorithms on data. I think it's really informative. So this is the original graph. And as you power the matrix and figure out what is the probability to go more distant and distant on the graph, you build up these probabilities of each cell traversing its phenotypic landscape more and more and more steps, and where it might go, and where it might end up in a terminal state. So for each cell, I have not only where it's a long pseudotime, What is its probability to reach each of the possible terminal states? And how much plasticity does it have? What is its differential potential of that cell? And this is, again, a very— it seems simple. It's just a Markov chain. But it's a very flexible framework. But it all depends on how you model it. What features are you going to put into the similarity metric? How are you going to build a similarity metric? You can, you know, plug it with biologically meaningful genes. You could incorporate epigenetic marks and regulatory function in there. You can even put imaging features, and I'll show you an example of that. We can change the way the edges are oriented. Here we did pseudotime, but I'll show you an example with velocity and ATAC-Seq, or even an accumulation of genetic mutations in cancer. So it's incredibly flexible. So just to really show you— and this is my favorite work. This is with Kat, and she's also British. We applied it to hematopoiesis, and we could figure out, we could look at cells making their very, very, very earliest cell fate decisions, whether they're going to be embryonic or extraembryonic. We could accurately order the cells and see that they decide to become extraembryonic when there is a combinatorial expression of two receptors, and only when you knock both of them down do you mess that up. We could figure out something surprising. So here at this arrow, we see this is that first cell fate choice, and then they lose their entropy. But then there's this other surprising entropy right here. And we discovered a novel transdifferentiation of a cell that had already decided to become an epiblast, to become part of the cell body, change its mind and become extraembryonic. It really surprised our biological collaborators, but we made a computational prediction. Convinced them to validate it, and they validated it, and our computation was right. There is this novel transdifferentiation of the epiblast to a visceral endoderm cell, which just, you know, rewrote the biology textbook. And using Palantir, we actually did manage to rewrite some early biology textbook stuff. And as I said, you know, you could change the features. So this is work by Prisca Liberale at FMI. And here she just used the organoids as features. So this is an imaging dataset. And here, rather than genes, she's looking at the morphology of these organoids, at the symmetry breaking of these organoids, at how many and where and how symmetry breaks. She has all these features that measures these organoids that she extracts from imaging. But also she measures some specific proteins in her images and quantifies that. She does Palantir Vanilla, the exact same algorithm, The only thing we're doing is changing the similarity metric. All the mechanisms of the algorithm is the same. And now we could connect the app one— well, not we, Prisca did this. She just used my algorithm. She could connect a particular signaling event with a functional morphological change of these organoids. So she could connect signaling to function by modeling them both together and connecting what signaling— what signaling? How YAP1 is a critical factor that drives some of this symmetry breaking that she could observe in her images. As I said, we can also change the way that we calculate directionality. Pseudokine makes very strong assumptions. It makes the assumption that cells go from less differentiated to more differentiated. It makes the assumption that cells can't go back, and we know that cells do go back in many biological processes. So here velocity was a big, um, splash because it sort of uses some information of, of direction, of some causality. Basically, as an RNA is being built, first it gets built as a mature RNA, an unspliced variant, and then it becomes spliced. So there's a directionality here. So people use this directionality to try and orient where things are going. But this is a very noisy measure. So when you sort of zoom out of these smooth, beautiful things— and never believe smooth, beautiful things that you see in 2 dimensions— you could see how these things don't agree. They point out of the manifold. They point in all sorts of different directions across very similar cells. It's very noisy. So here with Fabian Theiss and led by Marius Lang, we decided to combine these. And basically, we said, let's take Palantir and the cell manifold and basically orient things using RNA velocity so that we say that cells are more likely to go in directions that are similar based on the cosine similarity to the RNA velocity. And this is just the same algorithm, but we changed the way that we orient it. We changed the way that we learned the directionality on the manifold. And we have sort of the best of both worlds, using velocity to get the directionality, but still we constrain it all to the phenotypic manifold and gain some global consistency using these longer Markov works. There's lots of savvy math tricks that went into this, and, you know, one of them is really coarse-graining the dynamics of these very fine-grained transition matrices to sort of metastable states using pure-on-cluster analysis. All the details of everything that I'm describing is in all these bioarchives. I tend to write methods in gory details and explanations so people can follow, but I'm trying to convince you, you know, that these are good things and why I did that. And here we actually had a ground truth, and a ground truth is something very rare in biology. One of the things that that makes what we do hard in machine learning is we build these algorithms, we build these models, and certainly in the single cell field, we're waving our hands like crazy. There's almost no ground truth situation. That's why we're also having a hard time building normalization algorithms. Ground truth is very rare. But in the case of cell tagging, where we could take cells and we can tag them in such a way that we can say this cell and all its progeny are one tag, and you can connect cells and its progeny with these sort of tags, this you can do in vitro, we could sort of know where a cell's progeny is going to go in the future. We can connect one time point to another with the cell tagging. And we looked at this sort of reprogramming experiment, and CellRank said this is the initial state. So remember, CellRank can tell us what the initial states are and what the end states are. And it automatically found, OK, here's success and here's a dead end. And you can see that you can't really see on these velocities that there's any path from the initial state to the success state. And actually, it falsely seems that there's a path from the success to the dead end, which is also not true. But we actually— CellRank managed to find these paths, managed to overcome the fact that the two-dimensional short-term velocities seem misleading. And actually correct with incredible accuracy where a cell was going to go, whether it was going to succeed or enter the dead end. And we could see that the closer that we got in our reprogramming, the better we did. We could also pick up a novel dedifferentiation during regeneration. Regeneration is exactly a condition where the pseudo-time assumption of less differentiated to more differentiated is broken. And we could actually predict the dedifferentiation from goblet cells to basal cells, which we could then see in imaging, and we predicted novel intermediate cells. And this is entirely new. Not only that the goblet can go back to a basal, but the forward loop is basal, club, goblet. Even a direct path in any direction between basal and goblet wasn't known. But again, Cell Grant could look in the data and find all these novel paths of differentiation and cell fate change. But velocity isn't the magic bullet always. When you're using these modeling assumptions, you always need to figure out when they work, when they don't work. Velocity, you need to be in the right dynamics. That's about 8 hours. If you're looking at dynamics that go beyond 8 hours, good luck to you. You're making a mistake. There's entire systems where it doesn't work, where you don't have enough splice variants and introns going on to even get the data. And so hematopoiesis is an example where velocity miserably fails. So here we said, okay, let's use regulation, that same regulation we were using at the beginning of our talk, to orient. And this is work by Cassandra again, and she said, okay, I'm, you know, a gene is supposed to be some function of its transcription factors. Let's Let's make it linear. It's more robust. It's easier to learn. We can do it closed form. Yes, biology isn't linear, but there is power to making things simpler and more robust when you have very dirty data. And, you know, here's the single gene model, the genome-wide model, and this has an analytical solution. And now we use, you know, the regulation itself to predict what your future state will be like. Of course, we have to learn the regulatory velocity from the data. We have to learn which TFs regulate which targets and how. And we have some unknowns here. We don't know the slope. We don't know this regulatory matrix. And here comes EM to our rescue. Of course, there's a lot of dirty details. EM works when you have a good starting point. EM works when you constrain it well. But when you fix all these little dirty details— and it's the dirty details in these things that take the time. This is really simple first-year machine learning, but it only works when you take care of all these fine details. And we can get the velocity right. All these green arrows are going in the right direction. And the nice thing is, because we modeled these velocities as regulation, we can also ask about the regulation. And the individual genes are going the right way across as cells are choosing from one lineage to the next. Like PLEC goes up in megakaryocytes but not erythrocytes. And we can even ask, what are the underlying regulation of these single— regulators? What are the activators and the repressors of these single gene patterns? And the real fun is we can now do an in silico ablation experiment and say, let's knock out all the transcription factors, see which ones change the velocity, which ones create the biggest change in future cell fates, and say these are the important regulators of the system. And, you know, we got RUNX1 for hematopoietic stem cells, and it's a well-known key regulator. But if you were just to look at expression, it's actually not that highly expressed. But because we could pick up its impact, its regulatory impact, we can figure out that it was important. Now, finally, we're applying some of these methods in the clinical setting. This is a rapid autopsy, which we have at MSKCC. Where we're taking from a patient multiple samples from its primary tumor. This is pancreas cancer, as well as multiple mets. This is a deceased patient, so we're taking it out of autopsy, but recently deceased so we can get the data fresh. We're getting 3 different liver mets here. It's the closest thing to biological replicate, and we're asking how these cancers spread. Now, this is still work in progress, and I can't tell you a biological story, But all this sort of developmental mumbo-jumbo that I showed you, you see in the clinic in deceased patients that you have these transitions between primary cells to metastatic cells, between different paths of adaptation, of transformation that make these cells go and become very aggressive liver mets. And if we can go and understand these paths and what regulates these paths, and what drives these paths forward, then we can have new therapeutic approaches to perhaps block metastatic spread of these tumors. So that's where I'm going with it, but this is still work in progress, so you'll have to wait, you know, invite me again to hear the results. So really, the synchrony is powerful, and you can learn the dynamics of cell fate transitions, of what's driving different cell fates, and if we can learn regulators we can disrupt these regulators and maybe get some of the outcomes that we want. But I just want to give you a few final examples. Sometimes, you know, computation works best when it's simple. I mean, as a computational biologist, as someone that was originally trained in computation, I like swinging big hammers. But here's a case of leptomeningeal metastasis. These are the worst type of metastasis. They're painful. They're, you know, patients who suffer them have very, very, very strong effects. Imbalance, pain, pressure, and also a very poor prognosis. We have nothing to do with them, they have a 3.5-month survival time. So we use single-cell RNA-seq to study, you know, what's going on here. And very simple computation, very simple clustering and differential expression analysis found a huge surprise, that all the patients including breast cancers that metastasize to the leptomeningeal niche and lung cancers that metastasize to the leptomeningeal niche, multiple patients, all of them express in the cancer cells these two iron transporter genes in a thousandfold increase. Now, when we looked at the bulk, we thought it was the immune cells. These genes are only expressed in the myeloid immune cells. But now we could see that they were expressed in the cancer cells. At 1,000-fold higher than they're supposed to be expressed in the immune cells. And this happened in all the different patients, the power of evolution. And this gave us a new way to treat it. Iron chelation inhibits the growth in addition in mouse. This is now a clinical trial. And this is a case where we got from data to a therapeutic approach in under 6 months. This is actually going to go in patients at MSKCC. But the lesson here is that cells have important consequences. The cells, the cancer cells, learned how to take the iron from the environment so that they can survive. They took it so efficiently, they took all the iron out of the environment, that there wasn't enough left for the immune cells, which had much less transporter. The immune cells were inhibited and incapacitated and couldn't attack the cancer. So this really teaches you that there's these a lot of complex relationships. And the next thing to do is not only learn what an individual cell does, but how all these cells work in the environment. And for this, you need imaging. So this is work with Sasha Rudensky, and we're trying to really understand imaging. Here we're looking at the lymph node, because I really believe that when I'm developing a new method, I like developing a new method where I have some known biology to grip into, and then I apply it on new data, just like we did hematopoiesis, and now we're applying it to a pancreas metastasis to the liver. So this creates beautiful data, and here you see an entire lymph node imaged at 30 colors. But creating the pretty image is not as easy as you think, and the problem is, is that these are misaligned. So these, these yellow loops and these, uh, blue loops, they're actually on the cell membrane and they're supposed to be aligned, but these multiple rounds of imaging really create the tissue, uh, deformation. And you see that even in the same imaging, in the same lymph node, it deforms in different ways and different degrees in different regions. Now people say, well, why not MIBI or something like that? Well, MIBI gives you the field of view of these green things. So, you know, if you want a big field of view, you have to do this imaging that creates this deformation. If you want to then interpret the data, you have to realign the images. So here, I promised a little bit of deep learning. Here is where deep learning helps, because we can actually learn how to realign the images and get perfectly aligned images. And with perfectly aligned images, we get this perfect high-resolution, high-accuracy picture of the lymph node. And now we can apply other algorithms to find different environments with different types of cells in them in different proportions. And this is a multi-resolution thing. We can look at and identify B follicle and germinal centers and T cell zones, and these have very different cell distributions. But even within the germinal center, this is where all the B cells are actually learning how to make antibodies to prepare your immune system and to have them respond to your COVID-19 vaccine, you see very big differences. So, we can actually find little differences in their microenvironment. Here we find a histotissue, that's very active. We see here a lot of B cells. The B cells are blue and they're proliferating. They're, they're, they're expanding. This is a B cell that's good. It's creating an antibody that we like. Proliferating is yellow. So we see this explosion of proliferating B cells in this region right here. In this region, the B cells aren't really proliferating. That's because we can see a regulatory T cell right here in red that's actually inhibiting them, and, and that's good. We want to be able to inhibit the immune system. Yes, we want the immune system to respond, but not too strongly. So we can see all these interactions of proliferation and inhibition of a proliferation and all these environments with this high-resolution imaging. And that's the next frontier— how these cells come together, influence each other, and create tissue-level function is where we need to go. And that's where a lot of the deep AI are going to become very powerful. So in summary, single-cell genomics is really rich. The data resides on this low-dimensional manifold that's driven by pseudotime, pseudospace, and the gene programs that create the covariate structure in the data. And knowing how these are generated, we can learn these and use these for therapeutic benefit. And spatial data is the next big frontier and gives us so much to do, including for all those who think that anything that's not deep convolutional neural nets is no longer kosher to do in machine learning. So a lot of people worked here. I mentioned their names along the way. A bunch of current lab members, particularly Cassandra and Doran. A lot of past lab members, Elham and Manu. Amazing colleagues, uh, both at MSKCC and at other institutions. And thank you so much for your attention.

Thank you very, very much, uh, Dana, for a fantastic presentation as always. We are all very, very excited. And I forgot to say to everyone, please add your questions in the chat box and we will unmute you one by one and you can ask your questions. But while the questions are coming, if you allow me, Dana, I want to ask a student, like on behalf of some of the students in our center that are just starting in this journey in the intersection, let's say, between machine learning and biology. If the students are coming like you from a machine learning background, what type of areas would you suggest that they should be looking at in biology? What areas are easier to understand to start the journey, where both the problems are easier to understand, there possibly are sufficiently many problems unsolved, and the data is available for them to start? What type of guidance would you have to them?

So biology is fundamentally different than machine learning and math. In machine learning and math, it's really built layer on layer. You need linear algebra. And I know that people— that you have all these tools. And I have high school kids saying, look, I used Pyro to build a neural network. I know I'm an expert in deep learning. But beyond using all these tools where you take off-the-shelf things and make yourself a 14-year-old expert in deep learning, to really do it well, you need to understand linear algebra, you need to understand probability theory, you need to understand statistics. It's layer on layer, theorem on theorem, and it's, it's a tower. It's a process. You, you can't start at the top, and I have a lot of biologists who try and start at the top and apply tools without understanding what they're doing and what the assumptions are and what the fundamentals are, and it's a disaster. The fun thing about biology is it doesn't work that way. It's spread. You don't need to know T-cell biology to understand early embryonic development. You don't need to understand immune response to understand the epithelial-to-mesenchymal transition, so I don't think there's any one area of biology that's easier or harder. The important thing is actually to find the good partner, to commit yourself to learn the biology and not just say, machine learning is so strong, I can blindly swing my neural network at this data. And it doesn't need to know anything because neural nets solve everything. And the whole point is that they don't under— you just plug the data in and out comes magical answer. And to find a good partner. So I think the best way to learn is to choose a partner that will work with you, a biological partner on a biological example. And I have a lot of students that come into my lab with machine learning and math and CS training. And they're paired with often a postdoc that's experimental. And they work side by side and learn from each other and make sure that the modeling is right and explain to each other, this is the data. This is the biology. This is the model. This is how to interpret it. And that's the best. So the best is not the biological problem, but the partner. If you want to develop a new biological method, as I said during my talk a few times, start with a system that's known, even if it's less sexy. You know, yes, start with hematopoiesis, and then after you're comfortable with your algorithms, apply it to the metastatic transition between PDAC to liver. So those are my advice. Basically, biology— thank you so much— playground. You don't need to build towers and foundations, just find a good partner that you enjoy working with and is patient to teach you.

Fantastic advice, thank you so much, Donna. The questions kept coming, so Pietro, I think you have the first question. Would you like to start? Okay, thanks for the lovely talk, it's really inspiring. So all these cell types, I'm seeing them as a sort of attractor states. I remember Stuart Kauffman, when I was in Santa Fe, was sort of proposing that about 20 to 250 attractor states. Do you think it's a good definition of a cell type?

I don't like attractor states that much because it suggests in dynamics that they attract, and cell types are often transitional. Cell types often trigger and switch. I like viewing them as densities in high-dimensional space. So I like viewing cell types as high-dimensional densities. And so they seem like attractors because they create a density in this high-dimensional space. But they're not attractors because they're dynamic with one another, and you have transitions, and you can switch between them. And you don't want a system that will— block you from modeling the cell state transitions, which is some of the most interesting biology. Thank you.

Thank you very much, Donna. We have another question from Tony Wood.

Tony? Hi, Lena, hi, Donna. Can you see and hear me? Thank you for inviting me along, Donna. That was wonderful. You've made my day. Really enjoyed it. A gazillion questions, actually. The one that I'm asking is quite specific, but I can explain why. I'm curious about what you can uncover about the fundamental biology from, for example, your gene-gene covariates. Do they tell you about gene regulation that you could discover using experimental techniques?

I will—

I'll leave my first question there, and then I've got one more, if I may. After that?

So again, that's why I said it all depends on how you model, uh, the data. The gene-gene covariate structure doesn't necessarily in itself imply regulation, because for example, if you're thinking of statistical dependencies in classical, you know, Bayesian network one-on-one, a joint, uh, parent, uh, joint regulator can create a dependency between its two correlated targets and you'll see two co-regulated targets, you'll see a dependency between them, and assume that they're dependent and their dependency is driven by a common regulator. Now, if you model things in such a way that says, okay, let's just allow edges from transcription factors to genes, which is what we did in both symphony and in the attack velocity or regulatory velocity, then at least you have a regulator pointing to a gene. If you incorporate some data, epigenetic data, of the motif of that transcription factor binding at the regulatory regions of that gene, then you have a better chance of getting real regulation. And I tried to give you a lot of examples to give you a flavor, and it was more like a vignette-style talk rather than going in anything deep. But, you know, both in ATAC velocity and in Symphony, when applied to, you know, PBMCs, when applied to regular blood, we got all the known master regulators of all the known cell types correctly.

Right, and let me just extend that because I'm curious about If you were to take a case like the one you described for IL-7, where there's a well-known genotypical relationship to an outcome, and you did— and the experiments you analyzed were a wild type and a genotype with a known dysfunction, what changes? Does the covariance change? Do the velocities change? Do any of those things potentially change? You've got me thinking very differently about what is the real origin of the functional consequences of gene variants.

So the IL-7 example was actually protein data, and it was really stark. I mean, IL-7 is a cytokine, and, you know, JAK-STAT are downstream signaling and transcription factors. This was protein data. And basically, you know, along pseudotime, if you took a window right before the cell type that I— the precursor type 3 cell type, there was absolutely no covariate relationship between those genes. If you looked right after, you know, basically random. And only in that little window did you see a very tight covariation between IL-7 and STAT. Right.

And then I'm wondering if you'd repeated that experiment in a system in which there was, let's say, a STAT mutation that we know predisposes to autoimmune disease. Do you see that difference only appear in that same window, or does it appear in a different population somewhere else? I'm a drug discovery and development—

We didn't do that experiment, but we did block JAK and saw that when we block JAK, the cells in a dish didn't progress further. In that case, that's what we did back then. This was a while ago, and B cells don't really like being manipulated in a dish. They're not friendly cells for in vitro work.

Yeah, no, and so, like, thank you, Michael. I've already asked one more question than I should have done.

Thank you, Yana, really enjoyed it. The questions are coming. So Shoko Hiroshue has a question. Would you like to unmute yourself?

Oh, yeah.

I would very, very— thank you very much for the great talk. I was very interested in the last regulatory velocity method because I have the exact same problem. I'm looking at a mouse model which develops tumor in a very long— over a very long time. So I felt the velocity method, traditional velocity method, didn't really apply. Did you implement it as a package, or can you direct me to the paper?

That's yet unpublished. So I usually like putting some unpublished stuff that I'm excited about because I'm actively working on. We're hoping to release that as a package, let's say, in the 2-month scope of time. It's towards the end. But I enjoy talking about things I'm actively thinking about. By the time they're done, they become boring for me. But of course, most of the work, it has its packages listed there or listed in the bioRxiv publication. And I do try and do everything I write, put it on GitHub and make it usable. This is just not quite ready for prime time, but it's wrapping up into a package. Thank you very much.

Can I ask you just a very, very quick question about this thing? I'll be very quick.

Yeah, so please go on, but very quick such that we have time for others. Thank you.

Thank you very much. So my mouse model develops tumor in like a month's time. In your experience, do you think it makes sense to apply the original traditional velocity package?

What kind of the time frame do you need to look at when No, we've seen that if you're not in a short time frame, it doesn't work. I mean, pseudotime works, you know, if you have a decent guess on a starting population that you can start from. We found vanilla, you know, pseudotime, something like Palantir or other pseudotime approaches to work the best. They're imperfect and they make very strong assumptions. So analyze your data and interpret their output with great care. But right now, they're better than other methods. They're just— because they make strong assumptions, particularly of the data going one direction, whereas in tumor, there's just increased plasticity and it goes all over the place. I just need to interpret it with care, but we've actually gotten good results and good insights using, you know, Palantir and other standard pseudotime methods with a little bit of grain of salt.

Thank you very much. I would like to now ask somebody else to unmute themselves and ask the question. So Boris, Boris van Breughel. Can you please unmute yourself and ask your question?

Yes, yeah, thank you very much for the wonderful presentation, very inspirational. I've got a question about discovery of regulatory mechanisms, like to what extent is this possible, especially considering the very high dimension?

So one of the, you know, when I started out as a graduate student and young postdoc, There was a— after I published the early Bayesian network papers, there's a whole flurry of papers that said, what I'm doing is impossible. Some people went all the way and decided, because I was young, to call me a quack and a charlatan. Great fun. The fun of being a female scientist. But the thing that's going for you, even though it's high-dimensional noisy data, is the biological signal is strong. I mean, you have this high-dimensional data, but it resides on these very low phenotypic manifolds. So yes, you have— if you look at single-cell data, I think, to show you how constrained biology is, you have 20,000 cells that can be explained almost perfectly with 7 diffusion components. So that tells you a lot about the— constraint on the signal. A lot of machine learning proofs are very adversarial. If you have a crazy distribution that you're trying to learn, yes, you need all these other things. But because biology is very low-dimensional, the relationships are very strong, we know a little bit about it, and we could put the prior in there. Then yeah, we have a chance in hell. And in the case of regulation, we know a little bit about it. We know transcription factors bind to regulatory regions. We know in many cases for many transcription factors sort of what these binding sites look like. We know when these regulatory regions are open and closed. And if we can incorporate that in some of our modeling, if we can incorporate both the epigenetics and the expression together and get enough cells, collect a big enough data set where we have statistical power, I'm very optimistic.

Thank you, Dana. So the final question, in view of time, from Sankit Misra. Would you like to unmute yourself and ask your question?

Can people hear me?

Yes, we can hear you.

Hi, so I'm an HPC researcher who's focused on computational biology. And in the past, we worked on a problem for gene regulatory networks where we modeled this as a Bayesian network. And since Bayesian networks are NP-complete, we use a heuristic algorithm and then scale it to Tianhe-2, which was the fastest supercomputer at that time. So we scaled it to a petascale. My question is that you mentioned Bayesian learning, using Bayesian learning, but you also mentioned that it's compute-intensive. If we could scale it to larger data, would it make a difference? And can we help with that?

So the regulatory network, I mean, yeah, I think both Biscuit and Symphony don't scale that well. They're built on things like Edward. And inference of graphical models is a pretty heavy thing. So for a given method, yes, some of them are limited by scale. But I find in many cases, particularly with the simpler algorithms that you can scale up, they're limited by statistical power. So one of the things I sort of think about is if you can run your algorithm on more cells, that's better because you have more training samples, and that's good. If you can run your algorithms on more genes, you better have the statistical power to do that.

Thank you very much. Thank you very much. Makes sense.

Thank you very much, Dana, for a wonderful talk. I think everybody got very excited from both sides, the computational side and the biology side, the true inspiration. So thank you very much for joining us. We will be seeing you in the private session. Thank you.

Okay.