Scientist Stories: Smita Krishnaswamy, Representation Learning in Biology

Welcome everyone. This is a new edition of the LS Live joint event with the NCT Data Science Seminar. And today it's my great pleasure to welcome Smita Krishnaswamy. And now it's my great pleasure to introduce also Wolfgang Huber who will introduce her.

So yeah, so welcome everybody. Welcome Smita in particular. I'm really excited about having you here. So Smita Krishnaswamy was trained as a computer scientist with a PhD from the University of Michigan's Department of Electrical Engineering and Computer Science. With a PhD that focused on algorithms for automated synthesis and verification of nanoscale logic circuits in which probabilistic effects are important. She published a lot of numerous first author papers on probabilistic network models and algorithms for designing very large-scale integrated circuits or VLSIs using computer-aided design. After the PhD, she went to do as a scientist to IBM's Thomas J. Watson Research Center, which again, where she focused on automated methods for formal error detection, automated error detection. One of her algorithms, DeltaSyn, was eventually utilized in IBM's P and Z series high-performance chips. In the mid-2010s, she switched, luckily for us, her research efforts to biology, and her postdoctoral training was at Columbia University in New York City, where she worked among others with Dana Pe'er and Gary Nolan on learning computational models of cellular signaling from single-cell mass cytometry. Since 2015, she works as a professor in Yale where she holds a joint appointment between computer science and genetics. Her lab works on the development of machine learning techniques to analyze high-dimensional, high-throughput biomedical data. And her focus is on unsupervised methods, as you will hear, specifically on manifold learning and other techniques for detecting structure and patterns. So thank you, Professor Krishnaswamy, for being here. I'm looking forward to your talk.

All right, thank you so much, Wolfgang. And yeah, I had forgotten about some of the old stuff that I used to work on, which maybe not a lot of people know. So thanks for the reminder. So today I'll be talking about some of the newer methods in my lab that have to do with both learning the geometry and topology of data, with, of course, emphasis on biomedical data. So the biomedical data emphasis actually really provides new problems for machine learning and AI, I feel, because there you're really trying to get data-driven insights from data sets that have a lot of different observations and underlying structure, but maybe not a lot is known about the state space. So there is a lot of emphasis on exploring and just learning things directly from the data, and this motivates a lot of the unsupervised emphasis in our lab. So, but having said that, we're fairly data agnostic in the sense that we feel there are general common challenges between many of these high-dimensional, high-throughput data types, whether it be single-cell RNA sequencing, molecular structural data, fMRI, or patient data. Some of the common goals involve things like being able to automatically denoise the data without necessarily knowing the noise model, Understanding representations where you can understand the structure, groupings, progressions in the data, understanding how to integrate and compare between multiple samples, and also trying to learn something about the dynamics of the underlying system from static snapshots. And these are types of problems where we've worked on and made contributions in. Underlying a lot of this are some some fundamental mathematical concepts. So one idea is that this high-dimensional data actually lies in a much lower-dimensional manifold. And the term manifold comes from the field of geometry in math, for example, Riemannian geometry. And the idea is actually that you have a locally Euclidean and kind of smooth space. That you can learn. And this gives us an idea that we can learn the underlying structure of the data. And once we learn the underlying structure of the data and perhaps some good representations of that, we can do a lot of the tasks that I just talked about downstream of that. So we use a few different methods for trying to get at this intrinsic structure of the data. One set of methods involves graph spectral theory and graph signal processing. Here the main idea is to turn your data into a graph. So you can imagine having a graph between where there are edges between all pairs of points weighted by something like the distance between their ambient measurement vectors. Usually in order to learn the data manifold though, we use a more localized graph by passing the distances through a variety of affinity kernels that we sometimes design, and that gives you something more close to a k-nearest neighbors graph, perhaps a softer version of it. And then we use the next trick quite a lot, invented by my colleague Rafi Koifman, which is to Markov normalize this type of graph so that the adjacency matrix effectively becomes a random walk operator. And you can take walks on this graph, and we show that these random walk matrices as well as their eigenspectrum is very useful for things like clustering visualization and, and denoising. So the idea is that if you have a random walk operator, the eigendecomposition is a construction called a diffusion map, and the diffusion eigenvectors actually form some kind of frequency harmonics on your graph. So the high eigenvalues eigenvalued eigenvectors, which would be similar to the low eigenvalued vectors of a graph Laplacian, encode slow-moving signals or slow-moving trends, and the later-on eigenvectors encode fast-moving trends. So we've utilized this a lot to, for example, low-pass filter a graph and remove noise from signals that are, that can be modeled as existing on a graph. Another method that— other methods that we use for representation learning include self-supervised neural networks. So these days, the self-supervised neural networks can be pretty complicated. They can be designed with many penalties. So if I have time, I'm gonna talk at the end about a neural network we use to encode molecular graphs. And then we can penalize the neural network to not only decode the structure back, but also to predict molecular properties. And that can give you a representation that's organized by a variety of different metrics of the molecule. So these are two main sort of classes of techniques that we use. So I'm just going to go a little bit in order. I'm going to talk a little bit about our manifold learning techniques. So some of our newer manifold learning techniques are still based on a dimensionality reduction method that we developed in 2018-2019 called PHATE. So I'm gonna briefly go over PHATE and then I'll talk about the newer extensions that we're making on it that can help this data-driven learning from biomedical systems, including things like neurodegeneration, cancer, as well as immunology. The idea of PHATE is just that we want a dimensionality reduction algorithm that might tell us what the overall structure of the data is in a relatively denoised fashion, as well as to tell us how different parts of the data are related to each other. For example, if these two branches are coming from a common origin, that can be important information if you're studying a differentiation system. Whereas some of the other previous methods have focused more on preserving, for example, neighborhood structure. You preserve neighborhood structure. You might scramble the overall ordering, but you might be able to understand what near neighbor groupings in the data could be. And so for this, actually, things like PCA do a pretty good job of keeping the global structure. But one of the disadvantages of PCA is that it can't denoise in nonlinear directions. So in order to really understand the intrinsic state space and being able to denoise in those those types of axes. We prefer nonlinear dimensionality reduction methods that are often based on this kind of graph spectra that I just talked about. So this is kind of our motivation for developing PHATE is that we want to understand the geometry of the data intuitively, visually. And so the steps of PHATE until here would be familiar to anybody who's worked on, for example, diffusion maps. So you have data, we convert this to a distance matrix, then we convert it to an affinity matrix. But one important detail here is that we use an adaptive kernel with a fast decay. And the adaptive kernel is pretty essential to decoupling the geometry from density variations in the data. This is something we come up against in biology a lot. The density or the sampling of the data is not always desirable, and you're trying to uncover the underlying geometry despite vast density differences. And then we Markov normalize this affinity matrix into this random walk diffusion operator. And at this point, the steps of fate are different. Diffusion maps at this point decouple the data into different branches or clusters, and that's wonderful for clustering but not so great for two-dimensional visualization. Instead, what we do is we log transform the probabilities point-wise and take Euclidean distances between the log-transform probabilities. This actually gives us a symmetric type of divergence that's called M-divergence. And this M-divergence we preserve, we call this the potential distance. And this new potential distance we can preserve in 2 dimensions or 3 dimensions or higher dimensions with metric MDS. So the main idea is that this is actually a fairly robust manifold distance. And we've derived it through these steps, and now we're just going to preserve it with MDS. Given this, you might wonder if you really need all those steps if you're just going to come up with a new distance. Can't you just measure geodesics in data like isomap or something? It turns out not really. The diffusion-based steps are actually very, very important. You can't just use distance matrices because they don't give you manifold intrinsic distances. It's very hard to actually just use the affinity matrix either because there are so many shortcut connections that geodesics in this space would not be really reflective of manifold distances. And it's actually the powered Markov matrix that gives you this clean connectivity that's both globally connected and goes through paths of high density. Once you have this operator, We use actually a trick that's often used in word vector embeddings. We represent each data point by its context basically. The context here is the T-step random walk probabilities to all other data points. And the divergence really gives you a way of comparing contexts. And some kind of similar trick is used in a word vector embedding algorithm called GloVe. They also use distances between log-transformed probabilities. To come up with word embeddings. So the— and then squeezing it using MDES is sort of a little bit of a no-brainer because that is actually the best way to preserve distances of any kind. And so once you have this embedding, you can look at sort of the different components and branches actually back with diffusion maps. So diffusion component 1 would be activated in this branch. And zero everywhere else. Diffusion branch 2 is activated just in this branch and zero everywhere else. So you can actually clearly see the interplay between these two where the diffusion map finds the components and PHATE kind of shows how they're all connected. And we show this in the paper about how we can find branches and analyze their progressions also. So PHATE, as I motivated before, could be useful for studying something like differentiation structure, but also I know some of you are at a cancer institute. We're using we use FAIT a lot to study progressions like mesenchymal to epithelial transition. So the idea is that the embedding of FAIT would give you potential paths of progress among your cells. So this is a 27-day time course experiment where cells were harvested every 3 days and measured in 5 lanes of a 10x machine. And then when you superimpose the time structure of when they were collected, which is not given to any of these dimensionality reduction methods, you see that some— you would expect cells in the same day to be more related to each other, and you also expect kind of a divergent progression where cells are differentiating into different types of cells so that they can become progenitors for cardiac, neural, and other organ systems. And you see that PCA actually does keep this structure, as I said. But it doesn't sort of denoise in these branch directions. t-SNE doesn't keep the global structure, so it's not useful for that. And diffusion maps on average will show you two trajectories, one trajectory per dimension. So it doesn't collect all that information in 2D. And that's basically the use of PHATE. Some people ask if it makes up trajectories. It actually doesn't. It keeps— it tries to keep this manifold distance notion. So if there— there is no connection between these, it won't put points there. So this is a retinal bipolar set published by the Aviv Regev lab, but you do see some structure here. So one of the disadvantages to a method that keeps distance is that it can plot points on top of each other. So they can go right on top of each other. This can be viewed as a disadvantage. So this is not a disadvantage that t-SNE has, for example, because t-SNE blows out everything so that even density, a.k.a., uniform manifold. And so when you do that, you see the number of points you have, but you lose a lot of the other structure. So it's kind of a trade-off. So either I can see how many points are in the cluster, or I can preserve the shape of the cluster. So we started to try to address this problem because we actually— if there's a really dense area, we kind of want to zoom in and see more structure. So now we've come up with a version of PHATE that can balance this trade-off and actually allows you to zoom in. So here you see you might have a lot of cells. You can zoom in and see additional substructure. So that's why I set up PHATE to get you to talk about that. But already in the original PHATE, you can tell a lot of things. For example, you can choose downstream whether you want to cluster your data, how many clusters. Where the branching points are based on intrinsic dimensionality offered by the diffusion operator and regions of analysis. And so basically PHATE, just like t-SNE, UMAP, etc., can carry these properties to almost any kind of data. For example, this is genomic or genetic data, and again PHATE keeps some kind of distance structure that perhaps the other methods don't keep. As easily so you can tell which populations are related to which other populations. So in the paper, we actually come up with sort of a way of measuring what fate or things like fate are preserving, which if it's desirable to you would be a benchmarking method. And the idea here is that if you have a noiseless manifold simulation and you measure geodesic distances versus if you added noise to that simulation, rotate it into high dimensions and put it through these dimensionality reduction methods, how close are the geodesic distances in the original noiseless manifold to the noisy distances that you produce from an embedding? And this is really the metric that we're trying to preserve. So it gives us something that we're trying to preserve. And it also gives us a way of benchmarking things. Sometimes people ask if you could just substitute diffusion operators into t-SNE. And actually, no, you actually can't, because the structure of the penalty is such that it still won't preserve distance. It'll still preserve near neighbors. And so we're hoping that this kind of benchmarking will take off. So we have an effort called Open Problems in Biology, which if you're interested, please contact us. I know Julio, I've talked to him about it. I see him there, about benchmarking other methods and metrics like this. But sort of back to the problem of this multi-scale structure and how can we see the multi-scale structure. For this we turn to topological data analysis and bring in ideas from this field to combine it with our data geometric analysis. And if you're not super familiar with topology, it's this idea that you can quanti— especially algebraic topology, it's this idea that you can quantify things about sort of objects a little bit invariant to distances, but basically based on shapes. Some of you might have heard of these Betti numbers, and Betti numbers actually characterize how many of different dimensional holes that you have in your object. So you can see that this object here actually has a 2-dimensional hole. It's kind of like the hole inside your soccer ball or something like that. And whereas this doesn't have any cycles or 1-dimensional holes. And here, this shape on the other hand, it looks kind of like a donut, also has these 1-dimensional holes that you see here. And also one here. And so these are characterizations and they've been adapted into the data setting is actually to give you multi-resolution analysis. So this is actually kind of an ideal candidate if you want multi-scale structure in your data. So the usual way computational homology works is the computation of this kind of persistence diagram. You have your data points, you actually know the distances between them, and you grow these epsilon balls and you see which components are created and which are destroyed. And this kind of gives you usually just a descriptor of your data. But in some algorithms, people have noted that this process also coarse-grains the data. So we actually started to use both of these along with PHATE. But one of the problems with this kind of method is that it's usually operating in ambient space. And we reasoned that this was one of the reasons it wasn't super successful in biomedical data based on previous attempts. So what we're trying to do is combine diffusion geometry with topological data analysis. And so we've developed a method that, as Fred told me, is related to mean shift actually, but it's quite different because it operates in a diffusion space. And the main idea is that we actually condense the data using low-pass filtering operations based on the eigenvectors of the diffusion operator. Repeatedly. And this repeated condensation, which basically involves application of this filter over and over again and recomputation of the diffusion operator, it condenses data so you get a tree that's sweeping through every level of granularity. And this was done in one step in our MAGIC method, which if you've read it, just denoises the data. And basically what you're doing is you're killing some high-frequency variation at each iteration. But here we kill the high-frequency variation and recompute the operator. So it's actually an inhomogeneous Markov chain. And this inhomogeneous Markov chain, what it does is it pulls data points closer and closer to the center of their diffusion neighbors. And if you have a high power t, it's able to condense right on the manifold. So it's not condensing across the manifold, but within— rather, within the manifold. And recently, we've proven that this actually does sweep all levels of granularity. It is guaranteed to converge to one point. And as you're sweeping through these levels of granularity, you might realize that sort of one of the salient levels of granularity is these 3 cluster centroids for the 3 clusters that were clearly used here to generate this artificial data. But this is the sort of information that we do not know in biology. We don't know what the salient level of grouping is, particularly if you're trying to find the driver of a disease or some kind of biomarker of response. And so this is really what motivated us to actually search through all of these levels. And we've done a couple of projects that I'll go over based on this premise. This is a dataset on retinal bipolar cells, and what we were trying to find is the population of cells that seems most indicative and perhaps a driver of a condition called age-related macular degeneration. This is actually a neurodegenerative condition that afflicts sort of older adults, and they begin to lose vision, and they can even become blind as a result of this disease. So we had measured healthy controls as well as AMD patients in early and late stage diseases. And we had previously seen also that it's generally useful for neuroscientific data. So this is data that was earlier published in Nature with our collaborators. And here, this is almost a little bit of a toy data set for this. Because it's only 200 neurons in the C. elegans worm. And we'd seen that we get coarser and coarser grain groupings of the data until you can tell how the entire nervous system comes together as a single circuit. And so what we had seen is that it's actually intermediate groupings of these circuits that can give you an idea of specific circuits that could have gone awry in some of these worms. So we sought to replicate this kind of analysis in this data, and for this, we actually used persistence. So this is a persistence barcode, and in the persistence barcode, we can tell which levels are most persistent. They have nicely separated structure, and then you can quantify the groupings. So the most persistent cellular clusters actually turn out to be these major compartments. But when we zoom in and look further, we get these sub-compartments that are sort of the next level of the most persistent that actually find the drivers of these diseases. So when we looked at these most persistent clusters and we saw which were differentially expressed and differentially enriched in the two types of patients, we immediately saw the immune cells pop out, the microglia and the astrocytes, rather than the neuronal cells. So we went into these components and zoomed in. When we zoom in on these components, we find other clusters inside them. The second-level clusters actually give us more granularity on what could be the disease signature. And what we saw was this cluster 1 that's in green is actually an activated signature that's very interesting. It recurred in all the AMD patients.. But to our surprise, when we went to public data and downloaded data from Alzheimer's patients and multiple sclerosis patients, we saw this same cluster 1 signature in early disease, always in early disease, but not in late disease. But if you take this whole compartment, this whole microglial compartment, you don't get this signature anymore. So it's kind of drowned out because you're not looking at the right granularity. And since then, we've looked at specific markers of these. Because, for example, they could be drug targets. Similarly, we can find these same kinds of targets, which could potentially be therapeutic for pan-neurodegenerative diseases, also in Alzheimer's disease and multiple sclerosis. So, we've right now validated this signature in the retinal tissue, as well as communications between similar clusters in astrocytes and microglia. So this analysis is fairly complete now, and we've suggested potential drug targets, and it's under review at Cell. So the second story that uses this kind of multigranular disease is— actually, this was just accepted at Nature Biotechnology. I should have updated my slide just a few days ago— is our proposal to use this together with fate. To alleviate some of the crowding problems and be able to see multigranular structure. So here is data from 10x. This is T cells, B cells, and myeloid cells, for example. Multi-scale FAIT is showing a high level of granularity here, and you see where there's a lot of cells. So for example, in this cluster that's far from the purple cluster, it has a high level of, a lot of density. And so you could, for example, zoom in. And then what you do is you just go down the levels of the condensation tree, and you get further substructure. And so this is kind of what multi-scale PHATE does. It allows you to zoom in and out of different areas of the plot to see more or less structure if you start to see either differential enrichment or you just want to see substructure in a particular area. And it also turns out that this method can be actually made very, very scalable because after you process the first level, the rest of the levels are pretty easy to process. So this gives us, together with landmarking tricks, a way of analyzing a huge number of cells. So, and we did have a huge number of cells here in this study. We teamed together with immunologists from Yale New Haven Hospital and the Yale Medical School. You might've heard of Akiko Iwasaki. She's pretty prominent in the field. She, together with the Yale IMPACT team, were collecting data from COVID-infected patients who had joined Yale New Haven Hospital for further treatment, and they measured 80 million cells from these patients in 4 different flow cytometry panels. So we started looking at these. This is cells of all the patients, And if you look at these cells, we've colored these cells by a different technique that was published in my lab, which gives you a likelihood score that's called MELD. So this likelihood score says this kind of cell occurs with high likelihood in patients with adverse outcome. So it's a likelihood scoring when you're comparing at the single-cell level between 2 different kinds of patients. And so here you can go see what kinds of cells these are, and you can zoom in further to see what the cell is. I'm gonna skip over this. So if you zoom in on this cluster that I was just showing you, you'll again see substructure, and in the substructure, you can pinpoint the populations that are adversely affected versus the populations that are not. So, for example, here the eosinophils have higher mortality likelihood score and the CD16+ cells have higher and the CD16 negative have lower. And sometimes it's very interesting, but the trend can actually totally flip when you look at a higher level of granularity. When here, if you look at the T cell panel, the T cell panel, if you look at the overall T cells, it's highly associated with good outcome.. But if you zoom in and you get these different sub-compartments of T cells, some sub-compartments are pretty significantly associated with bad outcome or mortality. And so, for example, this IL-17 interferon gamma plus granzyme B plus. And you can actually use this as a marker in early stage disease to maybe change intervention or treatment. And, but we ourselves haven't gone down that road. But what else you can do is you can integrate many people's samples this way. So you've actually engineered some features as a result of this process. These engineered features can be used as a vectorization of the patients, and now you can make a fake plot of the patients rather than the cells. So here we went from all these cells to what the patients look like. And the patient landscape is actually fairly simple. The people on the left do well. The people on the right do pretty badly, and the And then people in the middle can kind of go either way. And so on this basis, because there is such structure here, where we were actually able to train a simple classifier that actually predicts the outcome of these patients. And we were also able to find associations. For example, I told you about the trend of more T cells, the less likely somebody is to die, but looking at another level of granularity, the more of these types of T cells, the higher it is that the chance of adverse outcome. And so this allows us to construct sort of patient manifolds and allows us to give sort of predictive insights. The next thing we're doing in both of these is actually trying to study the dynamics. So in order to study dynamics, we've been using neural network frameworks. So in this part of the talk, I'm going more towards our neural network methods. So last year in 2020, we published a method called TrajectoryNet, and TrajectoryNet is based on this neural ODE framework. And the purpose of TrajectoryNet is trying to learn continuous dynamics. So for example, cellular trajectories and inferences about the population in time points we didn't measure, because in these single-cell modalities we have coarse-grained time sequences. So patients could be measured every 2 or 3 weeks, cells could be measured every 2 or 3 days, but they're not continuously following these kind of entities. So just to differentiate from things you might have seen, you might have seen RNA velocity. RNA velocity gives you an instantaneous arrow of change. This arrow can sort of lose meaning if the entire population is progressing, like with a disease, because it's not available in these empty spaces. So we're trying to come up with a more global picture of the dynamics. Actually, more similar to what we're doing is this Schiebinger et al. paper from— actually from the Regev lab where they do optimal transport. But optimal transport between time points actually just kind of gives you a matching that these cells might have gone there. It's sort of a linear match, whereas we actually want to learn where they went in between dynamically. So this is why we developed TrajectoryNet. It's based on this neural ODE framework which is based on the realization that if you have a residual network, a residual neural network, it's actually an Euler integrator. And then at that point you can remove the layers and replace it with Callstone ODE solver to get continuous dynamics here instead. It's actually quite a brilliant idea, but when you use this in a probability distribution like that of cells, it's useful to use the framework of normalizing flows. Normalizing flows means how does this distribution flow to this one. Usually this has been used in generative models because they begin with a simple distribution you can sample from, and they apply invertible transformations to match the likelihood of your data, and then you go backwards to generate data from this. You start here and you go there. And you use this kind of change of variables actually to calculate the probability density of your data also. The probability density needs the change of variables because there's volume differences and you're trying to normalize your data to 1. So one interesting thing was that if you, in the continuous setting, a lot of the complexity of the normalizing flow actually goes away. You have this instantaneous change of variables formula The determinant's replaced by a trace. And this becomes quite effective. And so this is like having a normalizing flow with continuous changes to your distribution. But can you use a continuous normalizing flow just off the shelf to understand cellular trajectories? No, you really can't because it's not constrained in any way. So it can produce crazy paths, circuitous paths, circular paths. It has nothing to do with— how your cells may be progressing biologically. So what we sought to do, which we do a lot, is trying to bring regularizations and restrictions sort of from our biomedical domain knowledge. And so what we want are sort of more realistic paths than kind of circuitous paths. So the main regularization that helps us achieve this is actually to penalize the derivative at every step. We want the derivative to have low magnitude at every step.. And we were actually able to prove that if you do that, you get something called dynamic optimal transport. Dynamic optimal transport already gives you something that's much more biologically feasible. So for example, dynamic optimal transport would deform this Gaussian into this S by placing points directly where they go on the S rather than sliding down like this. So this is actually already pretty feasible. But we understand more things about cellular manifolds. For example, we understand that this kind of low-density path is probably implausible. So we have a manifold regularization, which is actually just penalizing by distance to the kth nearest neighbor all along your path. We can also incorporate something like RNA velocity if you have a robust measure of RNA velocity. As the instantaneous rate of change where it's available, you can penalize to agree with it. And given these different regularizations, you can improve the behavior of your model. So here we're taking this population and it's branching like this. With the density regularize— without the density regularization, you see it's just kind of pushing the population, whereas with the density and velocity, you start getting this arcing and branching behavior. Much better. And so we can use this to sort of reanimate our cellular populations and try to assess single-cell trajectories from this. But once you have these trajectories, it can still be a problem to understand causality. So you have these trajectories and you can look at different genes in these trajectories, but Which gene is causing which gene? It's still pretty hard to understand. You can use Granger causality. Sometimes that works and sometimes that doesn't work. So once we learn the dynamics, our key goal is actually to try to do some causal inference with this. So if you have any suggestions for that, I'm happy to collaborate or work with anybody. And that's kind of where we're at with these, both the AMD system as well as the COVID system is trying to incorporate the longitudinal and dynamic samples that we've been getting from these measurements. The final thing I actually wanted to talk about is just something with a totally different data type, just to show some of the other work we do. And this is called a geometric scattering autoencoder. I was telling Wolfgang that we just submitted a paper based on this, but I don't have the slides for the specifics of that project. The main idea here is that molecules can be encoded as graphs, and then that can offer actually some sequence and secondary structure information. But representing graphs is not super easy. So we all know, for example, that the adjacency matrix is a terrible representation because it's not permutation invariant. So if I relabeled the vertices, you get totally different representation. You can try graph neural networks. But graph neural networks use a lot of local averaging of features to come up, and then concatenation of node features to come up with these representations. And as a result, they can lose a lot of the global information in the graph. So they've been shown to be at most as powerful as, for example, the WL kernel, which looks at a lot of local neighborhoods. So one example could be if you have these kinds of signals, between these two graphs, then the local averaging might make the representations almost similar. But biologically speaking, these could be very different molecules because this, this molecule doesn't have this cycle that this other molecule has. And so you might have distinguishing power lost when you're trying to represent a graph. So instead, actually, we use a diffusion framework again based on the graph. Graph diffusions can be actually useful. So this is where you can get in one step of diffusion. This is where you can get in two steps of diffusion. These diffusion steps give you long-range connectivity, so they could be useful in distinguishing connectivity structure. So what we are doing here is taking molecular graphs and making a diffusion operator out of them by adding self loops to this graph. And this gives you the same random walk probabilities, but on your molecular graph. And there is a construct called a geometric scattering transform that's based on what are called graph wavelets. Graph wavelets, these graph wavelets, or so-called diffusion wavelets, are just differences between dyadic scales of random walks on the graph. So you take two scales, you just subtract them. So what our framework actually does is it uses multiple levels of these wavelet transforms and aggregates all the coefficients and takes all the coefficients for all of these signals that you might put on a graph, a signal centered at every single vertex, as a structural representation. And this goes inside the inside an autoencoder with a decoder. So this entire structure is meant for taking in these scattering transform coefficients, autoencoding them, but then penalizing also by a molecular property. And once you have this, you can come up with a latent space that's pretty continuous with the space of graphs. And in our original work, we presented a way of turning this back into a graph. Which we've actually now changed to be adversarial. So if we have low dimensions like 2, 3 latent dimensions, it's actually fairly easy to sample. So we can directly sample from the latent space of the graph. So at this point, we've penalized the graph so that it has low edit distance, smooth meta-properties, as well as invertibility by way of this inversion network. So given these properties, what we can do is we can regenerate graphs that are sort of continuous. So our first study was RNA sequence folds. These are non-coding RNA sequences. They're folding. You can get simulators like ViennaFold to give you different folds, but you don't understand the global landscape of it. So we tried to unearth sort of the energy landscape of this and generate trajectory intermediates.. And we can see that on both RNA graphs like this, RNA from Seq3, and a toy graph trajectory, that basically the 2-dimensional intermediate layer of our graph scattering autoencoder most faithfully captures the structure of these over the WL2 kernel, graph edit embeddings, just the scattering coefficients, graph autoencoders, graph VAEs. So this is just a simple graph that we evolved in a linear trajectory. And this graph is very interesting because it's called a riboswitch. It's a bistable graph that has two dominant conformations. And you see it's most easy to tell that this has two conformations from this embedding as opposed to any of these. So very, very simple landscape structure that we're able to sort of sanity check on. So with that, what we've been doing is actually generating folding trajectories and showing that they're fairly continuous. And more recently, what we've sought to do is generate molecular graphs that correspond to drugs from these. So these might not be organized by folding trajectory, but rather they're organized by similarity and by binding affinity. And we actually just submitted that to ICLR yesterday. So these are some of the types of work that we do, both with deep neural networks and graph spectral theory and combinations. Of those. If you're interested in any of the work that I talked about today, it's all on this GitHub. So you can play around with it, give us suggestions, whatever you want, and let me know if you have any questions.

Thank you very much for this whirlwind tour of all the different achievements and ideas. I think there will be questions popping in. I maybe have a to get started there, sort of high-level one, which is, I mean, a lot of what you've been discussing is sort of discovering patterns, structures, or paths and so on. What is your general approach towards inference, or like gaining confidence in which of these patterns that you find are significant in some sense and may be worth following up, and maybe which other ones are maybe more ephemeral or just noise in the data?

So usually— sorry— we think of a pattern as sort of significant or sort of real if we can replicate it in different data sets. So that's why when we were finding the patterns in our— sometimes the problem is in a lot of these single-cell RNA sequencing data sets, there will only be one data set like that. We had some 18 patients. Where we measured the retina of these patients, but there's no other datasets like that. There's maybe one other dataset that has like 2 patients. So that's why we kind of have to be creative. So that's why we went to these other diseases that we thought were related, and luckily we were actually able to find very similar signatures in these other diseases. So we basically look for replications, or, you know, sometimes computationally, for the fact that we can find those in different subsamplings of our data, but that's basically how. And a lot of our methods involve a lot of benchmarking or using of metrics. So this RNA sequencing folding idea I told you about, we've— public data sets. We found 4 or 5 different sequence databases of folds of different RNA sequences they generated, and we tried to make sure we can find things that are known about that structure. But sometimes that's not greatly satisfying because people might not know a lot about a particular RNA fold. So we also have to simulate a lot of data.

I think somebody's— We have some questions coming in. So maybe let's start with Ulrich's question. How do you determine the intrinsic dimension of FATE?

The intrinsic dimension in FATE is determined by spectral entropy or von Neumann entropy. And that's what we use for t so that we try to maintain that many dimensions of information in the operator. Of course, when you come up with a visualization, you're reducing it to 2. But if you want to use the operator for other things—

Leonardo Ayala asks about the scalability or computational complexity of FATE.

It's faster than UMAP and t-SNE. We— the results are in the paper. The reason it's faster is because it's actually not that difficult of an algorithm to speed up. So the idea is it's based on these diffusion operators. It has no eigen decomposition or anything like that. So what you do is you basically say instead of computing the diffusion probability directly from point A to point B, you compute the diffusion probability from point A to a small set of landmarks. And from the landmarks to B, and then the intermediate steps of diffusion are from landmark to landmark.

And so— So you're not n squared? That's right.

So it's below n, right? Yeah, yeah, yeah. We always show the complexity results in the papers that use FADE because that's an advantage of it.

Yeah. Joel Hancock asks if I understood correctly, the initial normalization step where you convert distances between points into weights on edges is equivalent to choosing a manifold metric. To do different choices strongly impact everything that follows.

Do most sensible choices for that metric. Usually for high-dimensional single-cell data, we just use Euclidean distance. And this is what people often use also in t-SNE and UMAP and things like that.

So those aren't necessarily— The question was about if you use different metrics than Euclidean, would you get different results?

Yeah, I think you would. I think you would. But it might be apparent which distance do you use based on the domain knowledge of the data type. So we weren't thinking super hard about what to use for single-cell data, for example, because it seems like these are vector measurements. But you could. You could use you could use like a cosine distance or a Hamming distance or something, and that would give you different normalization. But by the way, somebody else had a question in there. So usually the initial pairwise distances that you compute are not usually affected by the number of dimensions because all the nonlinear dimensionality reduction methods I know reduce the data to PCA first, and then they take distances of like 20—

between the 20 PC components or something.

Fred Hampricht asked whether the geometric scattering autoencoder is invariant to the permutation of nodes. That's a good question. It can be invariant. So the geometric scattering transform is invariant because it uses statistical moments of these wavelet coefficients. But in our recent paper, the one I submitted yesterday, we did use it like that because we're trying to generate molecules, drug-like molecules of different sizes. But in this story I showed you, it didn't need to be invariant because it was folds of the same sequence. Okay, so, so if you don't do this kind of statistical moment aggregation, it's not. If you do the aggregation, it is. And you can do something in between actually if you want. If you think it's like should be invariant to local permutations, you can locally average the, uh, coefficients.

—on the work on the drug-like molecules. If I got it correctly, you're just looking at the connectivity, sort of the presence of bonds or not. Or do you also distinguish different bond types or different atomic properties of the—

So we are doing some of that, but in the second step. So we first generate a base graph And then from that graph, we try to generate bond types or fill in the atom types and things like that. Yeah. Because then you have the graph, and then it's another different kind of GAN that kind of generate, can generate bond labels and things like that. But the main thing you need it to be is like a valid graph, valid molecule-like graph, for example, like one node can't be connecting to a million nodes. And actually, we didn't have to put any of these rules in. We use adversarial training. So the neural network knows that from the large database of drug-like molecules that you don't have high FAN-out.

It won't generate high FAN-out nodes. Julio Sait Rodriguez asks, can you elaborate on the application of other single-cell data modalities or even in a multi-omics fashion?

Yeah, so we've been working on approaches that can integrate along with single-cell RNA sequencing data, spatial, as well as single-cell ATAC-seq data. So we've been working with our cycleGAN framework called MEGGAN that was published a couple of years ago to translating between all three of these modalities. But at the same time, we've been— we just had an approach accepted at Machine Learning for Signal Processing for creating an integrated diffusion operator from two different modalities so you can come up with one visualization. The idea is some modalities have a lot more noise than other modalities, and that noise can be different locally as well as globally. And so you kind of don't want the confusion in one modality to infect the information in the other modality. So we're somehow trying to intelligently combine the information from both modalities.

In creating a diffusion operator. Are there any further questions? I guess now is your—

everybody's chance to ask. I guess somebody— oh, I see. So somebody, Leonardo, also asked, "FATE needs to compute distances of every data point to all others." Also t-SNE and UMAP need to do that too, just so you know. So that wouldn't be different. They all use an affinity matrix to start with.

But didn't you say that you could circumvent that by using the landmarks?

Yeah.

That's what makes it faster. Yeah. You made a comment in the beginning that I found very intriguing, and I didn't completely understand that how what you do with the diffusion distance affinity kernel sort of deals with the fact that the underlying true manifold is sampled non-uniformly, or that you have this variable densities. In your data, which may be the case that the underlying manifold actually is sampled in a non-uniform way.

Oh, it definitely is. The underlying manifold is sampled in a widely non-uniform way. So any kind of uniform manifold sampling assumption is likely to highly distort the geometry of the data. But actually, I've talked to Fred about this before, but the diffusion operator in specific, even the original diffusion maps paper gives you a way of kind of decoupling density from geometry by tuning down the effect of density. You can actually take off the effect of density by different degrees of kind of degree normalization of your graph, which is what also the adaptive bandwidth kernel does. It tries to take off the effect of the density. Because what happens is if you have a random walk operator and there are huge density differences, then all the walks will go to the dense area. They're just like a attracted to the dense area, kind of like balls. If you're rolling a ball down a hill, it always go down to the bottom of the hill like this. So for fate, it's important, or for fate and diffusion maps, it's important to decouple the density from the geometry. But by the same token, this could seem subtle, but when t-SNE does it, it's actually losing density information. That's why it spreads all the points out. Because it's trying to lose density information. So we don't necessarily lose density information. We can maintain density information because we can maintain some kind of distances. It's just we don't want the random walk geometry to be distorted by the density. So in the geometry discovery phase, we take it off. But when we put the points into the embedding, they can be very close together in a dense fashion in a way that they're usually not, especially in t-SNE.

And because of high dimensions, you don't really have to deal with because you're always working on a PCA-reduced version of the data.

Yeah, we usually are starting with a PCA-reduced version of the data because the nonlinear dimensionality reduction methods all go with sort of a cell-by-cell affinity matrix, so they're not directly dealing with the original features of the data.

All right, I think the hour is pretty much up. There's no more questions. So I'm really— thank you very much again from all of us here from Heidelberg and everywhere else where people are connected from for this fantastic overview over your work.

Yeah, no problem. I was telling them that I think Heidelberg was the last European place I visited before the pandemic. So hopefully the pandemic will end. I'll get to go. I was also invited for a talk in Osnabrück, and it's also remote. So I'm like, oh, these great places that I can't go to now.

Things are getting normal now again. I hope next year everything will be back to some more sane state.

Yeah, sounds good.

Thank you so much. All right, thank you.

Have a great day, you and everybody.

Thank you for being with us. Goodbye.

Goodbye everybody, thank you.