Scientist Stories: Thomas Walter, Machine Learning to Analyze Subcellular RNA Localization Patterns

Hello and welcome to my webinar Machine Learning for the Analysis of Subcellular RNA Localization Patterns. I'm Thomas Walter. I'm a professor at the École des Mines de Paris. And also group leader at the Institut Curie. And the work I'm going to present here has been done in collaboration with Florian Müller from Institut Pasteur and Édouard Bertrand from the IGH in Montpellier. So why would you be interested in mRNA localization? Well, in the '80s and '90s, a few mRNAs have been discovered that were asymmetrically distributed in many organisms. But at that time, this was still considered to be an oddity. Then in 2007, a seminal study showed that up to 70% of messenger RNA have non-random subcellular localization during Drosophila development. Since then, there have been many studies actually that showed that also in mammalian cell lines, messenger RNAs can have preferential localization. But the extent of this phenomenon is not known, and the underlying mechanisms are also not very well understood. So if you want to study messenger RNA localization, the method of choice is Single Molecule Fluorescence In Situ Hybridization, or smFISH for short. In this technique, there is a set of primary probes that is designed to hybridize to the target RNA, and a second fluorescently labeled probe is then hybridized to the primary probe. And for this reason, the mRNA molecules that have been targeted appear as small dots and we can see them here in this microscopy image. So this is basically a cell and all of these dots here are individual RNAs, individual transcripts, all corresponding to the same sequence, so to the same target RNA. And so there are many variants of this method actually, and the team of Édouard has developed now one technique that can be applied in high-content, in high-throughput mode, so which is scalable. And that means that we can now probe for RNA localization in many genes at the same time in medium or large-scale screens. And so just for an example, so we have here a 96-well plate and every well here would correspond now to one single molecule FISH experiment and would allow us to appreciate the localization of one particular RNA species. Now, the typical questions that can now be addressed with such a screen are: which localization patterns exist? So what are potential preferential localizations of messenger RNAs? And so that's an unsupervised problem. And the other question would be, once you know all the possible preferential localization patterns of RNAs, which RNA localizes where in the cell? And that's then a supervised problem. And so we hear already that in principle we can approach these questions with computer vision, so with artificial intelligence, machine learning. And this is the workflow that we have proposed, and others in the field have also proposed similar things. So you start from this image here, and the first thing you do is you would actually segment the individual cells. And so here we zoom into such a cell, and here we have the individual RNA molecules, and those have now to be detected. And so that means that we have for every cell a cloud of points, right? And this is illustrated in the third step here. And for these points, we can now— for this point cloud now, we can represent this actually with a feature vector. It's to say, we calculate certain measurements on these points point clouds. We use features from spatial statistics, and so this feature vector now describes the point cloud, right? And once we have done this, we have basically a vector. So we have represented each cell by a vector that is indicative of the spatial distribution of RNAs inside the cell. And if we are now in this unsupervised regime that we have just been talking about, what we would then do, for instance, for exploratory analysis, we might actually project this to a low-dimensional space using a UMAP or t-SNE or even PCA, but normally would go for more modern methods, of course. And then you get something like this here, right? And this is then So a plot basically where every point corresponds to one cell and the particular location of the point corresponds to the localization pattern. And this you would then kind of explore and look at the corresponding images or the corresponding point distributions in order to make sense out of that. You can also really run unsupervised algorithms, but you can also approach the supervised learning problem. Supervised learning, that means that we have to predefine the classes and then to give examples for each of these classes, right? And this is shown here. So in pattern 1, for instance, the RNAs, they all go to the protrusions and then Pattern 2, we have something which is close to the nucleus, right? And so on. And so you have several of those patterns. You have to provide the examples and then you can train a classifier that will actually assign to each cell represented by the point cloud features one out of these different patterns. And just to give you now, I mean, all of this is cartoon. So just to give you now a real-world example,. So this is data from a screen. This is our 27 localized messenger RNAs and also some non-localized messenger RNAs as a negative control. And so we can now actually look at the difference. So again, so every point here is one cell, and the different colors are actually manually annotated patterns, right? And we can now look what this corresponds to. The red ones, they correspond to intranuclear, where the RNAs are inside the nucleus. This is a random pattern, so this is kind of like a negative control. These are protrusion patterns where the RNAs go to the protrusion, so to an extension of the of the cell. Then we have nuclear edge where the RNAs are close to the nuclear membrane. Then we have perinuclear, so they go to one side of the cell rather than to the other, and in addition, they are in proximity of the nucleus but not as close as for the nuclear edge. And here we have a very interesting pattern which is foci. So here the RNAs, they actually build clusters, so little clusters, and these little clusters can correspond to translation factories. And this was one of the important biological findings of the paper that we published in Developmental Cell in 2020. And so all of these developments have been done by a PhD student in the lab, Arthur Amber. And so Arthur, together with Florian, they also developed a new version of FISHQuant. FISHQuant has been actually proposed to the community already a couple of years ago, but this is now newly programmed really from scratch in Python. Before it was MATLAB, and it's now really scalable, applicable, that runs on the cluster. It also has bindings to IMJOY, so can be used then with a graphical user interface. And it has been now really applied to a number of screens and a number of datasets. And we hope that this is continuing to be, that people continue to use that. And so that basically just came out in RNA in 2022. Okay, so let's now think about artificial intelligence, deep learning in particular. And if you think about deep learning, one of the bottlenecks clearly is that we have too little annotated data. This is a very common problem in particular for classification, and for classification we rarely have sufficient data to train on, right, a deep learning algorithm. Machine learning, this is okay, but for deep learning it's complicated. And so we tackled this question actually by starting to do simulations. So simulations, they can be used for many things. They can be used for validation of algorithms, for benchmarking, And they can also be used for training, and this is something I would like to show here. So first, we actually built a library of 318 cells for 3D segmentation of nucleus and the cytoplasm. And once you have these volumes, so this is not simulated, these are real cellular volumes, We can then place points in these reference volumes according to a priori laws of localization. So for instance, we can put the RNAs close to the plasma membrane, we can put them close to the nucleus, we can put them inside the nucleus, or we can make them cluster together to build these foci. We can have them polarized or go to these cell extensions. And of course you can come up with other more complex patterns if you wish to. Now here in this cartoon I do as if these were completely pure patterns and that's not true. So in reality what we do is that we draw the positions from a pattern-specific distribution, and we mix that with locations that are taken from a random distribution. Because we know that normally the RNAs, they never have a completely 100% behavior, 100% tendency. It's really a tendency and not a clear signal like that with 100%. Importantly, in this formula we can actually control the pattern strength, which is the mixture between random distribution and pattern-specific distribution. This is of course interesting because we can, by doing that, investigate the limitations of the algorithm. And this is now how this looks like. On the left, we see original images, and on the right, we see examples for simulated images, and we can appreciate that they look reasonably similar. So yeah, so that's good news. That means that in principle, we have unlimited amounts of data that we can now produce with our simulator. And for this reason, we were thinking, well, okay, let's do this now and train some neural network on this. And yeah, so we built the first training set of 200,000 images, 60% training, 20% validation, 20% test, with 7 localization patterns, 6 specific that I have shown and 1 random, of course. All of these localization patterns, they're systematically simulated in all 318 cellular volumes. To avoid dependence on morphology. This is an important point because simulations, they can— they do not only generate more data, but they can allow you also to reduce the effect of confounders. And one confounder is cell morphology. Another confounder, potential confounder, is the number of RNAs, so the expression level. And so this is varied between 100 and 1,000 per cell.. And then we also have different pattern strengths, and this makes a relatively heterogeneous dataset from which we can build classifiers. And our first approach was actually to learn representations with convolutional neural networks, is to say, to first simulate these points and then to do as if, if these were just ordinary images. We had 3-channel images, like 1 channel for the cell contours, one channel for the nuclear boundary and one channel for the detected RNAs. And as these are images, we can use any neural network that works on images, right? And so here we use SqueezeNet, but you could use in principle everything. And so here the idea is not to actually use the output of the neural network, but to look at the intermediate representation. Right before the softmax, and this is a vectorial representation and should therefore be informative on the point clouds. And so in principle, this is our representation that we are going then to use, and you can then project this to a low-dimensional representation and appreciate how usable this is in practice. But actually we then also, I mean, that's a little bit of a not so elegant approach because you have lists of coordinates, you would transform these lists of coordinates into images and then the images you treat them. And so that's a little bit, yeah, I mean, it kind of works, but it's not so elegant. And so for this reason, we were actually turning to another technique which is perhaps more elegant for sure, but which also works very nicely actually. And so here we would actually start from the point coordinates, directly a list of point coordinates, which are the input to these networks. Those are then aligned and then there are many transformations which are calculated. But then we also calculate— this was an important point— we have some attention mechanism inside the network that allows us, actually allows the network to focus on a subset of the points. This was, for instance, absolutely key for the foci pattern where in there's only a small fraction of RNAs that actually define the pattern, right? And yeah, and so in addition, we need some more input. Of course, we need the membrane positions themselves because otherwise the network could not relate the RNA positions with respect to the membrane positions. And we also use some graph-based information, local distance distributions, in order to enrich the descriptions and in order to use inter-point distances also. Yeah, so this is a paper that actually has been accepted for ECCV, the BIC workshop, and is going to be online very soon. And this is the result. So the result for the unsupervised analysis, and that looks again pretty similar to what we have seen with the handcrafted features. So the overall picture is the same. We get one very clearly defined cluster, then here one that is even nicer actually than for the handcrafted features, and the rest actually lives in one big cluster, but nevertheless inside this cluster, this is just because that data doesn't have to cluster necessarily. That just means that it's a continuum of changes between the different patterns. And so that looks pretty similar to what we had seen before. So that's very promising, but then we wanted also to put numbers on this, and we actually did some supervised analysis this time. So we asked, can we actually predicts, so classify each cell into one out of these classes with the encodings. And here we compare the handcrafted features with the learned features. We see that for most classes they are actually more or less equivalent. So the deep learning really gets a very good encoding just by simulation, so without any other definition of features. The only exception to this is actually this protrusion pattern here, and the reason for this is that we didn't include it into the simulation. And this means it's also one result, of course. So, if you really want to classify the patterns with high fidelity, then you better have them inside the simulation. But you do not have to be very careful to simulate patterns that are very close to the reality. It's actually quite robust with respect to that. Okay, so with this I come to my conclusion. So I have shown some computer vision approaches for the analysis of single molecule FISH data. I've shown some— an approach based on handcrafted features and another one based on deep learning. That directly operates on the point coordinates. That's an elegant solution. So we have a tool for virtually all of this, including the simulation, and that's called FISHquant. And so please download it and use it on your data. That would be really very nice for us. And, and yeah, and then of course, I mean, on the biological side, the application of these these tools can really reveal new localization patterns and suggest also novel mechanisms for the spatial control of gene expression. This is of course a very important subject. So these are the people who have contributed. I would really like to highlight the work of Arthur. He has really worked a lot these last 4 years on that, and he's about to finish his PhD. And I also would like to highlight Édouard and Edouard and Florian. So I have excellent collaboration with them on this topic. So thank you very much for your attention, and I'm happy to take questions, of course.