Scientist Stories: Valentine Svensson, Transcriptomics for Aging

So yeah, so we're going to Valentin, who's already been asking technical questions on the chat, or been giving technical advice on the chat. And Valentin was a PhD student in EBI. I think actually, I think maybe my first sort of EBI EMBL PhD student, and one of the, the only one, or one of the few. And Valentin was the GPLVM guru of the group basically throughout that whole time and was incredibly creative with new computational methods for new data types, I would say. So yeah, Valentin, over to you. Looking forward to your talk.

Thank you so much for that introduction, Sarah. Yeah, so I did my PhD with Sarah under the EMBL program between 2013 and 2017. Sorry, can you hear me actually? Sorry.

Yeah, we can hear and see well.

Okay, great. Yeah, and I should say that I ended up working with Sarah through a really random route, doing lots of other stuff before, and there's not a week that goes by without me thinking about how lucky I was to just randomly end up in such a great environment and great place. So I'm, yeah, I'm really happy about how my life turned out after working with Sarah. So I'm going to talk today about a methods development project that I did at my postdoc that I did after I worked with Sarah. So after the PhD with Sarah, I went to work at Caltech working on methods stuff in general. And as I said, we did the last experiment for this data one week before I moved to the East Coast where I'm now living, and I got back this data from it in late February 2020. So working on this has been kind of my COVID project, and I'm happy to have it at a state now where I can present it to people. So you'll be the first people to see this stuff. Before I get on to what I actually want to talk about, I should say that I'm currently working at a biotech company called Circuit Therapeutics. And just some promotion is that at Circuit we are using medical records, human genetics, and in vitro models to rethink drug development for complex diseases. And we are trying to find more people to join us, so if you know anyone who is interested in this kind of stuff, you can get in touch and then we can tell you more about actually what we're doing. But enough of that, back to some actual science. So what I got interested in through a number of projects in Sarah's lab is this general idea of how do cells change their state and how can we have these massive phenotypic differences in the same kind of cells when they are exposed to a stimuli or something else is changing that's causing them to start doing something else, starting doing something differently. And how can we investigate this and relate it to real time, which is why I want to talk about temporal expression. And something to keep in mind are time scales. So a lot of things where you're looking at cell state changes is kind of you do something and then a day later you look at it, but we want to actually get it down so that at least We can see things at this time scale of transcription, so you want to measure stuff at at most 1-hour intervals. There's all of this cell-cell variation, of course, and we figured like everything else, we could probably solve this with single-cell RNA sequencing. To develop this method, we did a model system, like an experimental model system. What we used is this microglia cell line model because we know that there's a massive cell state change when you expose these cells to LPS. You can see this very clear signal 24 hours after you do this. As a side benefit with this microglia, you have a very clear visual phenotype, so we can do stuff with microscopy before we go into the sequencing readouts. This happens 24 hours post-treatment, so we need to figure out a way to measure transcriptional changes as often as possible for 24 hours so we can see how the cells went from their unactivated state to the activated state and exactly which path they took to get there. The way we designed and achieved this is by culturing these cells in a plate with 60 different wells of microglia. Then we add LPS to a different well of the plate with half-hour increments. So every half hour we add LPS to a new well, and then we can look at how the gene expression for all of the genes we will measure relates to the exposure time that the well has been— has seen in terms of the LPS treatment.. And this is a simple enough procedure that would be very annoying to do in person. So what we're using is liquid handling robotics to automate this process. And I thought robots are really cool, so I'm going to show you a video how it works. So the cells are kept in an incubator that's on the right side of this robot. And every 30 minutes, the robot picks out the plate, takes up some LPS that's stored in a cool block, just puts it into the next time point well. Then it just takes the plate and puts it back into the incubator, and then we wait for, for the next treatment time point. So this just keeps happening over a weekend, and then you collect your cells and then you're done. So in this plate we have 61 wells of, of this microglia for the 60 time points that are stimulated by the, by the robot, and the negative control that we can compare things to. Then we collect the cells and we do live imaging of them afterwards to do visual phenotyping and do statistics on those kind of cell count numbers. But I'm not going to talk about that part of the project here because we all love single-cell RNA sequencing. And what we do is we take the plate, we perform multi-seq, multi-text single-cell RNA sequencing, put that in a 10x, and then we get 61 samples of things are only sticking back. For this experiment, we got around 10,000 cells after removing QC stuff. So of course, this is a different kind than in the previous talk. Christine also mentioned that when you're looking at time courses, you have to think about things a bit differently than the usual style of analysis. So we developed a model for for investigating this. This is a variational autoencoder model based on SEVI. It's a temporal model. So what a variational autoencoder does is that it kind of takes your data and it represents the cell-to-cell variation by making a low-dimensional embedding for that data that has, you know, which kind of cells are similar to each other, which will lead to co-transcription of various genes. So you have this z that represents that cell-to-cell variability. You would give that to a neural network that then decodes that representation into the gene expression levels of all of the genes in your data. What we added here is that we also have a temporal latent variable. So this neural network that decodes the gene expression levels also takes that temporal variable. And the reason we need this is that, well, we could use the time points that we know, but the thing is, as with everything with cells, is that they're not completely synchronized and they have then different timings in terms of how they're responding to the drug treatment. So with this temporal latent variable, we give us priors, our known exposure times from the experiment. Then we say that the different cells on average, they will probably be within a well, will probably be at around the same response state. But we can also allow them to vary between response states so that in a given well, some cells might look like they're an hour early or an hour later. And we can learn this from the data. So we take this model and we take the data that we generated. We fit this, and what I'm showing you here is all of the cells in the data ordered by their collection time points. You see this big block of cells to the left? Those are actually cells that failed to demultiplex from the MultiSeq data, but we can still use them when we're analyzing things. We just need a model to tell us which, well, it came from depending on the gene expression profile. So we give them a very wide prior. So the red dots correspond to the exposure times that we know from the experimental design. The black dots, which confidence intervals are the latent temporal state of the cells as they're growing. We can see here that of the ones that we successfully demultiplexed, I could show with my pointer here on the top, these are cells that seem to be faster than the average cells. Here on the bottom, you see cells that are a bit slower than than other cells, and we can actually investigate what kind of genes causes them to be fast or slow. But the question again is that we want to get at these temporal expression curves. We want to see what kind of patterns do genes follow over time. But we have this neural network that takes the cell-to-cell variation that's not temporal, that's a pore variation, and that's what gives us the expression levels. We want to find some way of removing the non-temporal cell-to-cell variation. We can do this by integrating out the non-temporal variation, and we can do this because the variational autoencoder model gives us posterior distributions for all of the cells in terms of all of their cell survivability states. So we can use Monte Carlo integration to estimate this integral for every gene that that we analyzed using the model. And this way we can immediately evaluate a temporal function for every gene. And this is super fast because everything is implemented on GPUs. So you give it a gene name, you wait half a second, and then you get that expression curve for a given gene. And you can actually extract all of these curves for all of the genes at once. And I'm just showing on the right here a random set of of genes. So the kind of question now is, okay, we have many thousands of these curves. How do we kind of figure out what's going on between them and how they relate to each other? And you can actually use a trick that's common in, like, if you're looking at spike data from neuroscience, for example, it's something people do. It's something that I found works pretty well. So we can evaluate all of these temporal functions. We can actually then evaluate them on a pretty dense temporal grid, say like 500 time points or so, that evaluate those function curves. Then you can take those curves and perform a variation of PCA on them, and what that will do is that it will identify as the loadings in PCA the basis functions for these curves. And that means that— so every dot in this plot to the right is a gene, and they are a representation of the curve of that gene. So two dots that are close to each other here have similar curves. And to kind of filter these sets of genes, we can compare, you know, what would that data look like if I had used an isotropic prior? So what looks like noise? And that way we can filter out genes where we have, you know, 5% false discovery rate of the genes having a temporally a temporal meaningful expression pattern. And this way we identified that out of the 15,000 or so genes we looked at, 295 of them we have high confidence of their temporal expression patterns. So yeah, that way we can actually do something and look at these expression levels. So I'm going to show you some examples now. Hopefully the text is big enough. I need to zoom in on my end. To read it. I need a new prescription for my glasses. Right. So on the top here are 4 cytokines that we found that shows temporal expression patterns. And you can see how they're high at 0 hours of exposure time and they're going down. The thing is to keep in mind is that, and this is something we should have thought about when we designed this experiment, we should have added a step of adding transcriptional inhibitors after the robot was finished because we have the plate, we do this exposure time thing, and then we do the imaging and we transport the plate to sequencing facility and then they collect the cells. And that is a process that takes a couple of hours. So when it says zero here, that's really kind of 2 hours from the start point. But we can still learn a lot about what kind of things are happening, at least in the later stages of this temporal expression. On the bottom, I'm showing as a kind of, you know, so things in this PCA plot, things that are far apart have different patterns. And you can see these two receptors where one is in one corner and one is the other corner. They have this complementary expression patterns over time where one goes down slowly over time and one seems to have been downregulated very quickly in response to the LPS, and then it's growing back up to homeostatic levels as time moves on. Similarly, we can look at some classical activation response genes. These are like interleukins that get expressed in pretty much every cell if you stimulate them with LPS. It's IL-6, IL-1 beta, IL-1 alpha. And it seems here, for example, we can learn that IL-1 beta kind of peaks in expression before IL-1 alpha. That's kind of an interesting— like how these relate to each other in terms of transcriptional waves. Again, relating to how we can see which genes are similar. This is like colocation analysis in genetics, where you look at how things are co-expressed across the genome. But here we're looking over time that these two genes are correlated with each other, but correlated in a temporal manner where curves over time look similar for these genes. So it's likely that they're regulated by some common factor. Okay, so that's just what I wanted to show. I still need to kind of write up all of the results for this, but I'm very happy that we could present this for the first time to this group of people. But I really like getting to know the liquid handling robotics, and it really enables these experiments that are not, you know, from a labor perspective, ethical. You know, you don't want to force technicians to do these things overnight and over several days. And it was a nice way to see that we can use these variational autoencoders also for temporal analysis. And I find these variational autoencoder models have been very flexible and really great for many, many kinds of experiments, and I really recommend people look into them. And yeah, we can just extract these temporal gene expression functions and we can we can learn about the regulatory relationship between them. So yeah, just acknowledgements. So I did this project while I was a postdoc with Lior Pachter at Caltech, who many people don't know this, but he has a wet lab in addition to being a computational biologist. Lisa Sled is a lab manager in the Pachter lab who taught me the cell cultures and stuff like this so I could do these experiments. Josh runs the Protein Expression Center at Caltech and he gave me access to the robots and helped me making one of these programs that runs for 30 hours. Matt Thompson runs the Single Cell Center at Caltech, where Jeff Park and Tiffany Tsu made the multi-seq libraries. So that's it for me.

Thank you. Really interesting talk, Valentin. I have to say that you missed a talk. Probably you were, I don't know, probably in bed when Natsuhi Ku gave his talk this morning. If you can stop sharing now. And he basically, for polyIC stimulation, so not LPS but polyIC, he was using GPLVM analysis. So it's innate immunity, so this related pathway, but with a GPLVM. So what's the benefit of the variational autoencoder over the GPLVM?

The biggest one is the speed and the scaling. So when I develop the computational tool for this, I do it with kind of arbitrary data and you can run it on a laptop with a million cells.

That's one of those. Okay. Yeah. We should ask you to— is— I mean, I want to test your model, Valentin.

On my laptop. Yeah, so it is on GitHub. I— yeah, it's not written up in a paper, but if anyone knows my GitHub, it should be like in the most recent edited repository, so it should be runnable.

Okay, now that's really cool. Yeah, Vitaly, sorry I interrupted.

Go ahead. Yeah, Valentine, I want to ask you a more detailed question about potential drawbacks of this way approaches, amortized inference approaches. What would you name as top 3 issues with it? Like, it sounds great the way you and others present it. It sounds like everyone should be using them, but what are the issues?

So yeah, the biggest issue that I have on a day-to-day basis is We current— there's currently no— nobody knows yet. There is like people are doing research into it, but how to handle nested experiments. You can do experiments where you have, you know, one sample is treated, one is untreated, or something like that. But a lot of the times what we have is like, well, we have 3 patients and then we collect 1,000 cells from each patient, and we have 3 healthy controls, so we collect 1,000 samples from those. And that replication structure is something that we don't know how to deal with yet, which is most— which is a lot of experiments. So that's like the biggest drawback, and you kind of have to fall back to another analysis method to work with that. Another, in terms of just the algorithm itself, and you asked about amortized inference, is they are not as good as like other, like, as exact as if you can make, like, a maximum likelihood method or use a Bayesian, like, sampling method. So it is kind of crucial. And of course, I didn't want to put in all of the details here, but it's crucial to kind of do checks where you look at the actual counts you observe and see, like, is my model actually, like, successfully replicated structure, or is it, like, is failing. So you need to be a bit careful to make sure that you're not seeing a mirage. You have to like, okay, but would the data that I'm simulating actually be similar to the data I'm seeing? And you have to check those things. And I'm sure there's a third thing as well, but this was a long answer, so I'll pretend you asked for 3 points, 2 points.

Yeah, so Jimmy has a question. Or you can come, come here.

Hello, Valentine.

This is Jimmy, the PhD student in the lab. So I'd like to ask a bit more technical questions. So about the temporal SEVI, I'm not sure how to integrate that time information into SEVI because after the encoder you get into the latent space, right? So you want to make sure the latent space catches all the time information and bring to the decoder. So, and in the latent space you can't contain any temporal information. But I'm not sure how do you manage to do that. Probably you can. I know it's not published, so probably you can roughly—

No, so that's the point of— so, you know, if we say that in a normal VAE you have 10 dimensions that represent the variation between cells, imagine that you take one of those dimensions and you put the priors from your time points there. So this is very similar to the GPLVM stuff where we, at least in the malaria paper that we worked on, we put priors on that as well in terms of the time points we had for the mouse treatments. And by putting those priors, you're forcing the model to, okay, variation between time points will be locked to this dimension, and then the other 9 dimensions will be orthogonal to that. So it will represent variation that's not temporal. So that way you're segmenting out the temporal variation from the non-temporal variation. I see your point. Yeah. And then with the neural— so we want smooth curves. So like a tiny— this is the default in SVI at least, but when you make the neural network, you need to make sure that it has these activation functions, and you need those to be smooth. So you need to use like arctan or or something like that. It doesn't work well if you use like ReLU. Yeah, but if you do that, then you get these smooth curves.

I see. So one more question is after you— I can sort of get the point, but I saw your results. Normally your relationship is kind of monotonically, like monotonically increasing or decreasing. I'm not sure whether you test the temporal ACVI, can it catch more, you know, complicated trend.

Awesome. This is, so I've been looking all over, like, you know, data in general, like public data and other experiments and stuff. And yeah, my dream is to find a bunch of these, what do you call them, like things that go up and then go down or go down and then up. It seems they aren't really there very often. So that's a thing. And so, you know, it would be great if things worked like that. But it seems what happens more is you stimulate cells and then there's kind of a big burst of like things getting expressed or things being—

I mean, I think it's true for something like innate immunity, but for cell cycle, let's say, you know, G1-S, G2-M, like you do have the non-monotonic relationship.

Right.

And also, I know you're regressing that, but if you were focusing on that, it would be—

Yeah, no, in particular in development, it's something you see a bunch. It's just, yeah. So if you do like a developmental model or if you would do this over the course of an iPSC differentiation or something like that, that's maybe something you'll find. But yeah, I think those transient relations are, at least in cell state changes or are pretty rare. But yeah, I was hoping to find a bunch of those in this experiment and like, oh, look at how these waves of transcriptions are then going over time. And it's a bit of a disappointment I didn't find that. But I think this is more when you're doing this 24 hours cell state change, it's more monotonic things that happen. And then over a very long time, the things decreases.

Yeah, yeah, that makes— yeah, great. Thank you, Wala. Thank you for a great talk.