Unlocking Cells' Secrets: Diffusion, Deconvolution, & Discovery with Siyu He, author of Squidiff & CORAL

Hello, and welcome back to The Cognitive Revolution. Today, I'm excited to share an in-depth conversation with Siyu He, postdoc at Stanford in biomedical data science and lead author of 2 notable recent papers that use intricate machine learning systems to shed light on 2 of the great questions in biology: what causes what at the cellular level, and how do the interactions between individual cells give rise to higher-level tissue development and disease. The first paper, SquiDiff, allows us to predict how cells will respond to perturbations by conditioning a diffusion model trained to predict transcriptome states— or the level of gene expression that a cell exhibits across more than 30,000 individual genes— on semantic directions derived from other experiments. This approach, based on a very modest amount of data, can save researchers months that would otherwise be needed to culture specific cell types, and can also shed light on intermediate states that can be extremely challenging to measure directly. The architecture will be familiar to students of image generation models, and also reminds me quite a bit of the mind's-eye models used to reconstruct images from brain scan data that we've covered on previous episodes. This is another data point showing just how general-purpose today's architectures really are, how models with intuitive understanding of important problem spaces like transcriptomics can perform many orders of magnitude faster than physics-based simulations, and ultimately how much future discoveries in biology will be made initially in silico and then confirmed by much slower and more costly wet lab ground truth experiments. The second paper, CORAL, makes it possible to combine low-resolution tissue data and high-resolution single-cell profiles into a single integrated view. This is a particularly intricate system that uses graph neural networks and ultimately deconvolves the lower-resolution tissue data into a detailed cell-by-cell picture, thus enabling closer and more insightful study of the tissue as a whole. This project makes notable use of synthetic data and reminds me of the project we covered a few months back from Michael Levin and his co-authors which had connected gene, drug, and disease information to make previously unknown connections. Perhaps most telling, Siyu and team completed both of these projects under the supervision of Professor James Zhao, who we recently had on the show to talk about the Virtual Lab Project, which used a team of AI agents equipped with specialist biology models to develop new nanobodies capable of treating emerging COVID variants, and also the Protein Language Model Interpretability Project, interPLM. That all of this work could come from one group in such a short period of time supports the broad theme that everything is working in AI. And with just a bit more acceleration, perhaps driven by the inevitable generalizations and scale-ups of the techniques we discussed today, we really could see a century's worth of biology progress in just the next decade. As always, if you're finding value in the show, we'd appreciate it if you'd share it with friends, Post about it online, or leave a review on your favorite podcast platform. Of course, we always invite your feedback and suggestions too. You can reach us via our website, cognitiveevolution.ai, and you're always welcome to DM me on your favorite social network. Now, I hope you enjoy this deep dive into cutting-edge architectures that are advancing the frontiers of cell and tissue biology with Stanford researcher Siyu He, lead author of SquidDiff, Siyu He, postdoc at Stanford in biomedical data science and lead author of the recent AI for biology papers SquidDiff and Coral. Welcome to The Cognitive Revolution.

Thank you, Nathan. It's great to be here and very excited to be here to share my research to a broader audience.

Cool. Well, we've got a lot to unpack. Let's maybe just start with a little bit of big picture context setting. I think the audience, and I can also just speak for myself, I've been obsessed with AI for the last few years, like studying it intensively. I feel like I have a pretty good general understanding of the landscape. On the biology side, first of all, the landscape is even bigger and more complicated, and I haven't spent nearly as much time in it. So I always kind of like to just start off by trying to get a sense for where you think we are in the big picture and where you think this work is in the big picture. And I guess maybe just tee you up a little bit more by saying the Squid Diff paper is about the transcriptome state of the cell operating at the level of a cell, which is quite interesting. And then the Coral paper is even zooming out to a little bit bigger scope of analysis than that and looking at samples of tissue. And I have kind of had this general sense that the grand challenge in biology is maybe a couple of different things. One big one is just figuring out the super intricate web of what causes what and what upregulates what, what downregulates what, how does an intervention ultimately lead to a change in outcome? And unfortunately for all of us, that's not only super complicated, but also seems to work at multiple levels of scale. So we have the ability at this point to do a pretty good job of saying, well, here's a sequence, what does that translate to in terms of the shape of a protein? And we're even starting to get decent at how do these proteins potentially fit together, but it still seems like we've got a long way to go. when it comes to how does all that stuff add up to the function and evolution over time of a cell? And then even probably more work to go when it comes to understanding how that aggregates up to the tissue, the system, the whole organism level. So I've been watching this space where you're operating because it seems like that's just a really critical frontier. And you've taken a bite out of each of those problems with these 2 papers. But with that kind of preface, tell me more about how you see the big picture landscape, how you understand the grand challenge, and how you have decided to orient yourself with this work toward solving them.

Yeah, I think the major motivation about these 2 projects is whether we can utilize the AI models really to apply onto the current technology of biology. And the goal is trying to understand the cellular systems in trying to understand the disease mechanisms. And further, we will provide like the treatment strategies for the disease and how to really like using the AI models to such kind of like different sort of datasets, like in technology, like for example, like single-cell RNA transcriptomics that I mostly address is like in the first paper and then the spatial transcriptomics that I mostly address is in the second paper. So these are 2 of different studies, but like they try into like very similar goals, like whether can we use AI models to accelerate those understanding of the cellular activity and then to figure out like the disease-related stuff and then their responses to the drug treatment.

Yeah, perfect.

Okay.

Well, let's get into the first one then. The first one that caught my eye was Squidiff. Am I saying that right? Squidiff? Squidiff? Interested actually in where the name came from, just as a bit of trivia before getting into the science itself.

Yes, right. So basically because I have a paper first, like we called it Starfish. And then I was thinking of like, oh, probably I want more models, like they call it the sea animal. So we have like Squid, but like it's a diffusion model. So that's why we call it Squid Diff. But it also can seem like it's simulation-related, sort of like stimuli-related quantitative inferences. of the transcriptomics, like using the diffusion models. Yeah. So you can try to actually guard in the whole ideas about like what model is. And the Coral, yeah, is a second one, but it's also a name around like the sea plants.

Gotcha. You got a theme, you've got a whole cinematic universe in the making of various models. Yeah.

Hopefully I'll get more.

Tell me about transcriptomes in terms of maybe what we need to know that's not obvious. What I know coming into this is the transcriptome is basically a measure of what genes in a cell are being actively transcribed at any given time. So my loose understanding is And it's amazing we can do this now at the level of a single cell. The technical achievements of the resolution of some of these techniques are really super amazing. So we can take literally a single cell, I assume, but correct me if I'm wrong, that this is like a totally destructive process. So this is the end of this cell. We can then pull out the RNA that has been transcribed from the DNA at any given time, and we can just look at how much of these different RNA segments are there. And that basically tells us the sort of state of the cell at that moment in time. Like, what is actually— what are the active processes that are happening in that cell? But that's a rookie explanation. So, you know, tell me what I'm missing from that or what more I need to know.

Yeah, I would suggest maybe a good way to start about this story is to think of like the central dogma of biology. We call like the cell is the smallest functional unit of the living systems. And then the key information in our cells, like we call chromosomes, they carry the DNA. And then so in like human bodies, like all the cells or like almost all the cells that they share the same DNA, what makes them different is the transcription processes that like the DNA transcribes to RNA and then RNA will further translate it into protein and then protein will fold it into specific structures like what the AlphaFold is trying to predict it. And then like they will have various functions. So the process is like related to transcription is we could also like gene expression. That is an indication of a lot of like variants of the cells in the human body. And one of my postdoc advisors, Stephen Quake, has said like the cell is a bag of RNA, and which actually indicated that the complexity of the cells are able to distinguish from the gene expression. Protein can Protein expression can distinguish the complexity too, but I think like the current technology, I would say like it's still like maybe more prevalent in the transcription levels because like protein probably like are much noisy. Even we have like more and more advanced technology too for protein, people are mostly like starting with the transcriptomes first, and which help us to understanding the cellular activity and disease mechanisms. And the second point that I also want to mention is that the development of the new technology, the single-cell RNA sequencing. So I probably not go to details about what exactly the whole process look like, but the major thing is that we dissociating the tissues into individual cells. And then we will probe or like to measure the molecular expression, like the gene expression into the RNA. Sorry, into the cells. And those mostly can profile like around like 30,000 or 60,000 of the genes, which is like very high dimension. And it's like also high-throughput technology, which making AI models like a very good way to address these kind of data because usually we will have like a matrix sort of things. We have like columns and rows, which are like cells and genes. And they are kind of like indicating a lot of like things in individual cells. Yeah. So, that's, I think, really a phenomenal technology came out and kind of like opened areas to the quantitative biology. Yeah.

So, 30,000 to 60,000 different cell— or sorry, different genes can be measured in just a single cell. Yeah. I mean, that is an amazing level of resolution. And I think it's always helpful just to kind of consider these, like, the dimensionality of the inputs. This is true in the AI architectures that we're going to get into. And so I guess just to be very clear, the starting point for this whole process is a transcriptome is like 30,000 to 60,000 numbers that represent the intensity of the expression of the individual genes. And that's measured through the RNA that's pulled out of the cell when it's going to be processed in this way. Is there anything like, do we have any reason to think that there's like, that the transcriptome is like a complete picture? Or are there things that are sort of known to be missing from the transcriptome? Is there a way in which the transcriptome doesn't tell you the whole story of what's going on with a cell?

Of course. I think transcriptome, like, though they will share kind of the heterogeneity of the cells, like, I think there's more information related. For example, we're looking on the epigenomics in the cells and like the chromosome things that happens in the cells that differentiate across the individual cells. And also there is like protein, as I said, which are also distinct with the gene expression, but like sometimes they kind of share similarity. And beside that, there will also like mitochondria-related or like ribosomal-related things. I think like cells are very complicated. That's why people haven't really figured out like what exactly they're going on. And the transcriptome is probably like, is one of the data that we can get at a whole genome level. So it's a very good way to start and to understanding like where cells going. But I would say like there's more other information that can be also considered into the model design and the whole story design as well. Cool.

Okay. So now getting into the Squid Diff architecture, let's first maybe just describe what it is that we're trying to do here. Because one thing that will become clear as we go through this is this both of these papers are pretty carefully designed architectures, actually more complicated than what we see even from frontier language models today, which is an interesting fact. I want to get into as well, why has the bitter lesson maybe not fully come for this sort of work yet? But let's just start off with what is it that you hope to create a model to be able to do with SquidDiff, and then we'll like actually unpack the strategy and the parts of it.

Okay. Yeah, sounds good. So, right. So I think the motivation of this project is like, because usually I'm a wet lab person that trying to getting those single cell sequencing from like, you know, the whole processes of the experiments, which is like very kind of painful because you need to waiting the cells like to grow and then you prepare whole processes that is like usually waiting for like months, but also it's very expensive when we're trying to do the experiments, but also it's very expensive at this time point. What I'm thinking, can I actually like using the generative AI models to creating those like a virtual or like digital recreation of the transcriptomes? So this is the first motivation. And then I realized there is a potential, like we can maybe manipulate that the the major information is like, for example, because we have like a lot of condition-generated AI models. So what if we just like manipulating these major conditions, we probably not like, like observe like exactly, but like we know what they are. And there is maybe a way that we can try then. And then that's why we designed the SqueakDiff. And so it's not just like maybe creating the transcriptomics, but also addresses like some sort of interesting questions in biology that whether there is a perturbation happen, like chemical perturbation or like gene perturbation, making the cells like to be different. And then what they look like in most of the cases in experiments, like we are not able to get them immediately and also very large scales of the data. So that's, I think, the way it can help. The whole field.

Hey, we'll continue our interview in a moment after a word from our sponsors.

Yep.

So you said it takes potentially months to culture cell type and condition of interest, and then you said it's expensive to do the sequencing. Can you cash that out? Is there like a, you know, established dollar amount or is it still like a relatively bespoke process where it's just like your time is the expense of Doing those single-cell sequencing processes?

Yeah, so I think it depends on the exact tissues or like the samples that you are dealing with. And usually I was working on the organoids, which is like engineered tissues derived from like iPSC, for example, like induced pluripotent stem cells. We were making like those engineered brain tissues or like vessel tissues and doing like disease modeling. So the whole process is like usually takes months. I know like some of the brain organoids, they need like even a year or like more, more times to create. And definitely like there will be some sort of mistakes like during the, you know, the experiments and you will need to like start over again. And that's why kind of like very painful. And this is like initiative that I really want to have a— models that can help me to, you know, just take over the task. And, but also there's another concern, you know, in some of the cases there is a limitation that experiments cannot really perform. So that's why I think such a model will be very useful.

Okay. So let's do just a little bit more on like the actual runtime. how this thing gets used. You might have a particular cell type, and you might have a question of interest, like, what happens if I apply this drug, or I apply some radiation? Or you could correct me if I'm wrong on the assumption here, but upregulate or downregulate a particular gene, something like that. And you basically want to know what happens. And the form of the answer to that question is ultimately a transcriptome. What the system ultimately outputs is a transcriptome, not that's measured, but that is generated or predicted by the AI model. What does that do for you? So, I mean, correct again, always correct anything that I'm missing there. But then what does that do for you? Like, when you now have a transcriptome put out from the model, what do you do with that?

That's a good question. Yeah, usually, like, when we have the transcriptomes from the data, like, we are using kind of like some sort of analysis of the single-cell RNA sequencing to understanding what's going on. And I think the first step usually is like to annotate those cells. And usually people will do with the dimension reduction and also like using the manifold folding methods. to seeing like how they look like, maybe like clustering technology as well, like to kind of like classify the cells into different cell types. And then because we also have the gene expression into individual cells, so we knowing like what specific genes are they expressed. So we know like the related signal pathways they are, and then we are understanding like what's all going on. So that's usually the way like people analyze the whole kind of data. Yeah, but there's multiple ways I think in the field they're trying to really quantitatively understand the whole stuff.

And how well do we understand that sort of thing? Like, if you have a transcriptome, regardless of its source, like it could have been measured in, you know, an actual wet lab experiment, or it could have been generated now by a model, what can we— with what level of sort of granularity or specificity or confidence can we say what's actually going on in that cell? Okay, sure, we've got a number now for 30,000 to 60,000 genes, but could you say, like, oh, that's definitely cancer, or the cancer is now— it's not cancer anymore, or it's healthy, or it's not healthy, or far more— I mean, obviously you can do a lot more specific things because you can get down to this sort of You know, gene-by-gene level analysis, but what is our ability to sort of aggregate that up into the things that really matter using all those techniques that you just listed?

Right. Usually, like, I would say if the cells are similar, like they're in a similar state, they will have like similar sort of groups of the genes that express, like we call like co-expression sort of things. And in that case, like we are able to like classify or like group in these cells in into different clusters. Each cluster will kind of usually corresponding to a cell type. And in that case, we are able to knowing what is the high variable genes in the individual groups. And other sort of method people usually consider is the marker genes to really distinguish what's going on. So marker genes are those genes identified by the whole biological community that they recognize is a major, or like, hallmarks for the cell types. So then, for example, there is cancer-related markers. People, if they identify that, they will consider, okay, they probably is like a cancerous stage. And similar to what the pathology people, that they also trying to using the marker, like the protein and antibodies, like to figure out like what's going on in the tissues. Yeah.

Okay, cool. So there's clustering, there's these kind of indicator genes that are like reasonably well understood. And now I think we can get into, you know, how it is that you're going about training a model to come up with these things. So I guess maybe for starters there, what does the dataset look like in terms of Like, where does it come from? You know, how much of this stuff is sort of out there in the community for you to just grab that other people have published? How much do you, if any, have to actually gather yourself? And just like literally in terms of just, you know, megabytes, gigabytes, whatever, like how big is the dataset that you were working with in this project?

Yeah, so that's a good question. Usually, I would say it really depends on the projects. And as I said, like, we have like a matrix sort of format of the data, the like cell by genes. And we have like 30 to, you know, 30 to 60,000 of the genes, which is whole genome level. And then we will have the also thousands or like even actually millions and billions cells. But it depends on projects, I think. And in my In my models, like, I'm taking those, like, maybe small datasets and specific datasets instead of, like, the whole things altogether, which is because also to consider the complexity of the models. And that's the current development things. But I would also thinking of, like, using a more, like, a larger scales of datasets into the models and making it, like, more fun. foundation related. And in terms of the data resources, we usually are kind of— I mean, the community is like shareable. They will kind of like publish their own data into some space, and then we will utilize those publicly available data. And at the same time, we're also collaborating with wet labs, and then they help us to creating those like novel, maybe new datasets or maybe have validation as well for the model. So it's very flexible. And I would say the data source is one of the major challenges right now in the field because we will want to have high-quality data and also have— because most of the data, they are good, but sometimes they're missing some information and will make the models hard to develop to making them consistent.

So, by the way, is it annotated at all? I understand when you describe the grid of cells and transcriptomes, what I'm understanding there is basically one vector for every cell that indicates the strength of activity for each corresponding gene in the in the long list of 30,000 to 60,000. Is there additional metadata around what kind of tissue it was or the patient's condition or anything along those lines that you also tap into here? Or is it literally just pure kind of raw information?

Yeah, we say they have metadata and it's about the cell types which annotated maybe by myself. or by the other authors, and they would say maybe annotated publicly already. And it has matching information, for example, like what kind of tissues it belongs to and what disease stages it has. So I think there is multiple ways to label these data already. Yeah, that's one of the requirements in the model.

Gotcha.

Okay.

So did you say, sorry if I missed it, but Like, is it a sensible question to ask, like, how many individual cell transcriptomes constitute the training data for this project?

Yeah, so right now I would say, like, the model will take in any kind of large amount of the data to be trained. I'm, like, not providing a trained model. Definitely, like, we require people to provide their own trained datasets. So it really depends on like how datasets you are. And then usually I think I'm just testing how long it needs to, and around like 5,000 cells in kind of that kind of datasets, it just requires like 15 minutes to run, to train. Yeah. Wow.

So you can get this working with as few as 5,000 different cells.

Yeah, yeah. But I think too few cells will make the models like to be underfitting or maybe overfitting too.

Yeah. Okay.

Wow.

I mean, that's a small number. So do I understand correctly? Like that's kind of where it starts to work and in practice you used more and in like fullness of time or, you know, maybe future work you could use like a lot more still.

Of course. So I think like the future direction of like the, this project is I think we want to building a bigger one which can kind of like cover all kind of, I mean, for example, we are working on organoids as like a case studies in the paper, but we are thinking of like envelop all kind of like organoids together. So we will have like foundational models for the organoid things, but we can also make a foundation model for tissue or like real human tissues too. So it very depends on how the design of the exact datasets we want to use. But the model is very powerful to taking any kind of data, I would say, and flexible to address the needs of the people, of the researchers.

Hey, we'll continue our interview in a moment after a word from our sponsors. Okay, so then let's talk about the architecture a little bit. I guess one kind of just super brief summary of the structure of the architecture is there's basically 2 core parts to it. There's the VAE, the variational autoencoder, and then there's the diffusion model portion. The question occurred to me before even getting into how those work, because certainly folks in our audience will be even more familiar with the transformer, as probably the whole world is starting to become quite familiar with the transformer. Why not use a transformer for this? If somebody just came to me out of nowhere and said, let's say you want to predict the next transcriptome state for a cell, given a current state and a perturbation or whatever, that would be my first instinct would be like, yeah, we could probably do it with a transformer. Transformer seems to work for everything. So why did you end up going a different way with this project?

Well, first thing I wanted to maybe apologize is there is like mistakes in the first manuscript that is not actually the VAE, it's a semantic encoder that connecting with the conditional, the diffusion models. So we have like the major, 2 major things. One is the DDIM models and then the other is like semantic encoder. And the major generative processes is that we taking the Z semantic, that's variable of semantic, and that's kind of from the encoders. And then we take a Gaussian noise, XT, and using the denoising processes of the diffusion into a final version of transcript tones. Yeah.

So I'll dig into that and unpack it a little more, but just for intuition's sake, like why not? a transformer architecture? Why this more complicated and less familiar architecture?

Yeah. So transformer and diffusion models are both very popular models as a generative AI model. I would say there's already a lot of applications in biomedicine with the transformer as well and with the diffusion. So the difference is that The diffusion model maybe takes more stochasticity of the data because the generation things is more flexible in that case, and they will kind of maybe modeling the more complex distribution. And in terms of transformer, I think it will be powerful. And also there is a lot of similar works that are trying to generating the related stuff, but I think the difference is the input of the model is a little bit different because usually the transformer will take the sequence data, like the sequence of the genes. Usually in our area that we're trying to rank the genes first. So it's not like a count values of the data, but ranking the genes. And then the output is also the ranking of the genes, which is probably different with like what the original data look like. And then the transformer-based models, like a lot of foundational models right now, they trying to getting an embedding of the data and then further to use those embeddings to kind of doing downstream tasks like classifications and prediction task, right? So I think they are kind of like different this time point. And another reason that I wanna using diffusion is that, like, I really— I noticed at the moment that when I started the project, the diffusion model is not really utilized in the generation of the single-cell transcriptomics. That's why I'm trying to really consider, like, explore the ability of diffusion models and conditional models that really can manipulate the conditions to generate the virtual cells. Yeah.

Gotcha. Okay. So on the architecture itself, I mean, you kind of described it briefly a minute ago, but I'll try to take a shot at describing it. I'm actually not quite sure of the distinction between the VAE and the semantic encoder. You can clear that up for me in a second, but either way, when we've seen this on the feed. I've looked at lots of different architectures over time, and actually, maybe the one that this reminds me of most might be Mind's Eye, which was a project partially out of Stability AI, where they trained a system, and it was, again, kind of a carefully designed system to reconstruct the images that somebody was looking at based on the fMRI scan of their brain at the time they were looking at it. And that also involved a diffusion model, and it involved basically creating, in that case, a diffusion prior for the diffusion model to work from. And there's something pretty similar going on here where the first step is to try to understand what really matters, in short, in terms of the transcriptome of a cell. In general, the VAEs, as I understand, are basically trying to pass the raw data through some narrow bottleneck such that it can then be recreated out the other end, capturing as much of the original data as possible. So typically they're trained on a reconstruction loss. Where the point is to say, how can we sort of compress, abstract, get this data into its sort of most semantic form, still preserving all of the information that really matters so we can get back what we put in, but then use that highly semantic form for other things. So that's kind of part one. Anything I missed or anything you would complicate on that description?

Yeah, actually, that's a very good question. I would say there are different kinds of architectures about the models. And if we think of a combination of VAE with diffusion, I would say they are like the latent diffusion models. And there are also related models that are trying to address this application in transcriptomics as well. The difference is that these kinds of models, They usually like training separately and then they kind of like using the VAE to do a dimension reduction and to getting a latent representation first and making the whole generation processes like more efficient. And the way that Squeezie is trying to address is like to training a unified model so they are able to train together. And then because the whole process is including the way to learning the noises, but also the way to encoding the semantic information. So I would say it maybe have— it has the powers to really like making semantic information extracted from the data. And then also with the stochasticity that the model of the diffusion can do, and finally making a more realistic expression data with the cells.

Okay. That's an interesting note. So this system is trained entirely end-to-end under one loss function. And the big distinction there is typically in other contexts, the VAE is trained standalone separately, and then kind of can be mashed up with other things downstream as needed. Okay, so, well, that brings us then to part 2, which is the diffusion model, which I think folks will be fairly familiar with from an image generation context. I always like to get procedural with these sorts of things. And so the way I do that in the image context is say, okay, let's say the goal is to train a model to say, given some starting point, what would this image look like if it was a little less noisy? And then with the conditioning, what would it look like if it was a little less noisy and a little bit more like this thing that we're trying to create? And what I think is really genius about that, of course, is it allows for mass production of training data because you can just take all the images on the internet and gradually add noise to them and then train the model to denoise them. And then eventually you get to something that can start with pure noise and just go step, step, step, step, step all the way to finally a high resolution image. And with conditioning, you can get a high resolution image of something specific that you want. So a lot of same mechanics going on here with obviously the big difference being that the conditioning is coming out of this semantic encoder. that is saying, like, this is the kind of cell that we want in the sort of lower dimension, most sort of semantic way possible. Now it's up to the diffusion model to map that back out into the fully detailed picture of a particular transcriptome. Again, what am I missing there, or what more do I need to understand?

Yeah, it's a very good question. Like, I think also because when I tried to create the model, so thinking of like what's the differences between these models with the other imaging-based diffusion model. So I would say like they are very different in with the data structure first, because like usually most of the imaging data, they are 2-dimensional, have like XY, you know, like directions. But for the single-cell transcriptomic data, we are dealing with just one-dimensional data. Like, the object is the cells, right? Like, the cell and then a gene expression. So with that, like, we're not able to use most of the way that the diffusion model dealing with the image, like U-Net or like image-based networks, to learn the noises. We are using just like MLP to really like learning the noises, but we also consider the residue connections that trying to incorporate the event of the times and also the semantic features. And in terms of the semantic features, I would say it can kind of like providing like implicated information. So it's not just like cell types that included in this, in the data. They will also have like other sort of, you know, the information related, like for example, the disease stage and the conditions that related. So it's like a unified conditions, which we saw like maybe can be manipulated in the space and not in the gene space. Yeah.

Is there anything special about the noising process? In images, it's pretty simple. I've seen that in contexts like protein folding, there is a sort of highly specialized noising process that's required because you can't just— the naive approach of just adding noise to a protein structure just takes you to something totally incoherent, I guess. And so there's like a more sophisticated way of doing the noising. Does that apply here too, or are you able to just apply like a relatively simple noising process?

Yeah, I think the way to dealing with the noise is maybe quite simple. We didn't really consider too much of like the noising issues happen, but the differences we also consider is most of the expression data, they are usually in a single-cell version. They are very sparse. So there are a lot of like zeros in there, and we will need to kind of like take care of that. So I think what we're trying to address is like we're not like studying all the genes related. We filter out some of the genes that are probably not really useful, and we're looking at the high-variable genes only. And another advantage of diffusion model is that they are not able just to reconstruct in those Gaussian or simple distributions. They are able to have creating very complex distributions that most of the gene expressions can have. So the model itself have the powers to model these things.

So then in terms of how we actually run a sort of in silico experiment on this. We've got the setup of this semantic encoder that takes in a raw transcriptome and converts it into a more compact semantic, more semantic representation that abstracts away those details and tries to get at what really matters. It seems like the key thing to understand I found here as I was reading the paper is that you need to do an in silico experiment to be able to ask the question, what would happen to this cell type if I did this sort of stimulus to it? Is you need to be able to encode what that stimulus is in this semantic latent space. And so what I understand is happening to create that sort of direction is you're basically creating a delta or basically doing subtraction in the semantic space between other pairs that you may have. So I think people probably have seen examples of this from language models or even image models. There's like the really famous example of if you have the embedding for man and woman, for example, and then you subtract those, you can kind of create the direction from man to woman or the direction from woman to man. Obviously, those are going to be the negative of one another in this latent space. Then you can do interesting things like if you have the embedding of king and you add the delta of man to woman, you find that, hey, that at least in many experiments looks like the embedding for queen. And so you have these sort of these like delta vectors that can be synthesised from contrasting pairs and then applied to other starting points to get to some other place. So I understand that that's kind of the core mechanism, if I'm understanding correctly. As always, correct me. But what I think you're doing in a lot of cases is saying, okay, I have cell type X and I want to know what would happen if I apply stimulus Y to that. But it would take me months to culture that cell type and then do all these experiments. But I do have cell type Z, and I do have the ability to apply the same stimulus to cell type Z. And then I can— and maybe some of this stuff is just also out there as published data. You don't even necessarily have to run the experiment. You can tell me about that. But if it's easier for me to do this experiment in cell type Z, then I can get that Delta vector, and then I can go apply that delta vector to the cell type X, and then I can run this diffusion process. I can see what the transcriptome would look like if I had hypothetically applied this stimulus to that cell type X without even necessarily ever having to touch the cell type X. So again, how'd I do? And, you know, what'd I miss?

Yeah, thank you so much for the introduction. I would say like it's really, yeah, it's exactly what it is. It's like we using the vector operation and to address those manipulation with the semantic variable. But I would say like this is assumption actually, it's not exactly the real cases I would say, because usually like the biological processes, they're complex and usually not very linear. But I would say like the latent variables probably like are more linear compared with to the gene expression space. So we're using the linear approximations to, you know, like really just resemble to the real cases. And it can kind of like dealing with some certain like maybe simpler like conditions. For example, in my projects, like we are providing some scenarios, for example, this cell differentiation, which like, for example, day 1, we have like iPSCs, which is the stem cells. And then at day 3, we will have like a more, you know, like a more mature type of like mesoderm cells. And what does the individual look like? And in that case, like if we have the, you know, the semantic variables at day 0 and then day 3, we can actually using the linear interpolations to getting this like day 1 and day 2. And to you know, to kind of like creating the transcriptomes look like in that days. And this is like sort of a way we did. And the second applications I would say like is the perturbation task. You know, like in some of the case, we will have like gene perturbation, like we making one genes to be maybe like upregulated or downregulated. And then in other cases, we're making gene B that kind of like regulated. And What's going on with like if we have A and B together, even like I think that the result will be like nonlinear and the model of the diffusion model is able to model this like nonlinear, sorry, like nonlinear processes, but the latent variables are still able maybe to, you know, to manipulate by this like vector operations because I would suppose that the the space of the latent will be more smooth and more structured. That's also an approximation, I would say. So it's a risky thing that we directly just rely on the results from the model. So we need to kind of validate it as well. But so far, we apply it onto the open-eye cases and we see very exciting results, like showing some cells, you know, they're very transient cells that during the development and during the differentiation into IPSC into the endothelia and into the fibroblasts into the blood vessel structures. Yeah. And then to figure out this issue, I would say like we will also consider variant of the SQUIDIF as our second project. And instead of like, you know, just a simple semantic encoder or like simple Schematic variables, we consider a more time series analysis. Well, that kind of related to how we generating the video. So like video generated diffusion models, say we are training onto a more complicated or like datasets that has the time series information and we're learning implicitly about the development things and then kind of take more on the linearity of the datasets. Yeah.

Yeah, that's interesting. It's funny how some of these techniques that are developed for, you know, what were initially such whimsical, you know, kind of, I was following these things back in like 2021 when, you know, the legendary Twitter account Rivers Have Wings was, you know, training these diffusion models and they were just generating very sort of bizarre, but sometimes quite intriguing art. And it was like, is this useful for anything? So it turns out it's actually got direct application to biology. And it seems like we're maybe just a couple years behind. Obviously, the image generation and the video generation has gone extremely fast to the point where now you in many cases would have a hard time distinguishing the video generate, or certainly the image outputs from models from real photographs. And not quite there on video, but sometimes it's hard to distinguish and sometimes it's still a little weird. But man, we've certainly come a long way. So to project that could happen in the biological domain as well over the next 2 to 3 years is nothing short of a paradigm shift, right? I mean, to be able to get that kind of accuracy out of a simulation seems like it would just totally upend the field. And it seems like it's almost certainly going to happen, right? I mean, what do you think is the— I mean, there is this issue of the nonlinearity, which is a fundamental one. So Maybe I should just ask a little clarifying follow-up question. I wasn't quite clear on exactly to what degree this has been validated. I understood that there was this 3-day sort of cell transformation process and this model, because you were sort of able to essentially scale the delta, right? You could say, okay, I have a cell at day zero, Something, I can measure the transcriptome now. Now I can apply some perturbation. Now I can come back and measure it at day 3 after it's differentiated into a different cell type or whatever. Now I have that direction in space. Are you calculating the 2 intervening days as literally just a third of that vector and two-thirds of that vector? Is it as simple as that? And then how, if it is that simple, Do I understand correctly that you also have gone and actually taken cells at those 1 and 2-day time steps and compared to see how well they line up? And if that's all right, then how well in fact do they line up once you actually took the real measurement?

Yeah, that's a good question. I think we have different types of validation strategies in the model. For example, For example, we have some scenario cases. For example, in the diffusion applications— sorry, in the differentiation applications, we definitely have the real experimental data from day 0 to day 3. And then in the training processes, we only take day 0 and day 3, but we will head out those 1 and 2 as a test set. So when we're getting these models to be trained and We will have the results for 1 and 3, as what you said about manipulation with the semantic variable. And regarding those predicted values for the gene expression at day 1 and 2, and then using the real case to compare and get the scores. And in terms of other cases, like the gene perturbation and drug perturbation, we also do the similar thing. For example, we're not— we're having the whole, you know, the experimental data about like perturbation on gene 1 and gene 2, and then also 1+ 2, but we will hide out those like 1+ 2 and then just testing whether they exactly can predict. And then the third strategy I would say is the real cases that we're showing in the end of the paper, the blood vessel organizing. So in that case, Because we need to culture— sorry, we need to culture the organoids, right? So from day 0 to day 11, if I remember correctly. So we cannot stop it. We cannot like collecting them like day by day because the collection of this data to the single-cell sequencing will actually destroy the whole sample. So in that case, we only take day 0 and then we take the final days. And we were just using the interpolation processes to generate the intermediate stages. But we have some public available data that have the time series data as well. And we will compare whether we can identify similar biological things. So exactly, we discover some novel cell states that we haven't really thought about. And in that case, we will also study about like what specifically genes are being triggered by this sort of differentiations and then compare our findings to the experimental data from other publications. We will definitely see a consistent finding. So making the models like to be strong to like indicating that the ways to, you know, the powers to finding these new things. Yeah.

Okay, cool. So, and you mentioned too, it's an important point that like the manipulations that you can do in the semantic space are these like vector operations, which are inherently linear. And that feels like a problem because we know how much nonlinearity there is in biology, but you do have the diffusion process, which can capture nonlinearities. So it's like sort of, you can hope that'll work. And at least in some number of cases, it seems like it does. And I suppose it just kind of remains to be seen, like, in what cases might those sort of linear vector operations just fundamentally break down in the semantic space? Did you, have you seen anything like that where you sort of say, yeah, well, we tried one, you described the validation successes, but were there any validation failures where, you know, you're like, I guess this is just like fundamentally not linear, or like somehow we're not capturing it in the semantic space? Any instances like that?

Well, I think like for now, the cases that we are testing are always like kind of efficient, I would say. We're not seeing that, but we do see that, you know, the intermediate stages prediction is like not as good as we thought. That's probably because, you know, we are modeling the semantic variables resemble about like the factor, for example, iPSC to the mesoderm cells, like they will have like different type of growth factors to be cultured cells. So we have actually different, you know, different vectors. or like different directions that making the— there's many ways that making the final stages of the variable. So the point is we just doing an approximation and then making them to be like linear. So I would say if we have more information, for example, we learn what the semantic variables for the individual growth factors look like, the whole models will be Or like being better a lot. But for now, we don't have that information. So this is the way that we try to address this. Another point that I probably mentioned, not probably related to this question, but we also realize the limitation of the models. For example, if we having some data like about like drug A and drug B, and we want knowing like what's the result look like, we training on the A and B. And then we can kind of like using that vectors A and B to other cell types, right? So we won't having A and B on any type of cell types and knowing what the responses is. But what were the case that we have drug C, because we don't know exactly the vector or like the signaling factor of drug C, so we cannot like to model this like unseen drug. And to address these issues actually, because also I was asked by the reviewers in the paper, we're thinking of like consider another variant of the model. So we're having now like adapter that can encoding the drug information that is using— we're using this mouse structure of the drug and also have this dosage information of the drug. So we can embedding this drug information related to structures and dosage and then also concatenating with the semantic variable, we will have like a new version of the semantic variables. And then in that case, even if we don't have the application or like training onto the drug scene, we can still kind of predicting the results if we have the drug components of the drug scene.

Yeah. Interesting. Okay. So do you think that this is, as it stands today, something that can't effectively accelerate science. I understand how it would if it's accurate enough. I mean, at least I think I do. The way I always kind of think about these things is long term, this can change, but short term, we have sort of a finite amount of wet work experiments that we can do. We've got finite lab space, we've got finite grad students and postdocs to be hands on and actually do the cell culturing, and there's all this know-how. And so for a short period of time, that throughput is relatively fixed. And sort of what determines how much actual scientific progress we make is, do we run the right experiments that actually yield the interesting results that we can learn from? And so the great hope for these sort of in silico experiment models is like, obviously they run orders of magnitude faster. I assume that maybe you could tell me how long it takes to do one diffusion process, but I would assume it's like less than 1/1000th, maybe less than 1/10,000th of the time that it would take to actually do the physical experiments. Is there a single answer to how long does it take to do one of these?

Yeah, I would say like this model will actually help the researchers to accelerate their research. I would say, you know, the way that we're getting this like single-cell RNA data, I probably like need at least like 1 week to, you know, from the dissociation to exactly getting these like raw files about like the sequence. But if you want to get the datasets, like with other conditions, like, can we just use this grid model and using maybe like an hour to getting what's going on with other conditions look like? And then in that case, like people maybe have some intuition about like what's the direction of the project look like. I would not say like trust the model, like, you know, totally, but like we can have some, something that information happen and, you know, in some of the projects, for example, people develop those organoid things because organoid is a good way to be like a drug screening platform. And so they are studying about different ways to creating those organoids. So I guess if we're having the vectors for the individual growth factors, we can able to predicting what's going on with the development of the organoids and also provide information about like those organoids respond to the drug, like different drug components. And that's one example. But in other cases, for example, in a real case, for example, in the patient's level, you know, like for the patient with the cancer and they want to know what kind of drugs are mostly useful for them. And it's kind of like risky for them to just like randomly taking Some drugs, right? So we need to testing. And then I think it's showing powerful applications in predicting how these cells respond and whether they will be useful for that patient. So like, will be have side effects as well. Yeah, I guess will provide like, you know, efficient outcomes and predictions and not necessary to waiting, you know, long times to see how the outcome Yeah.

Yeah. The context of a single patient is really interesting too. I hadn't even really considered that. I was just kind of thinking like general scientific inquiry. You can sort of run a bunch of these simulations and identify the ones that seem most promising and then prioritize your wet lab work accordingly. And that in and of itself seems like it could really—

Yeah.

accelerate the pace of discovery making that you could do. But then it becomes like even potentially more of a game-changing technology at the point where you're applying it to an individual who has potentially their own idiosyncratic situation going on in their own cells. So, yeah, that's cool. Do you think that— so I guess next steps for this are like, Scale-up is obviously always one good candidate. And you mentioned the sort of separating out the interventions to have like a distinct semantic encoder for them. And then you also mentioned the sort of video to image kind of time series to single sort of before and after analogue. I mean, I guess that's enough, right? You'll have plenty to work on to tackle those. Do all those, I guess, one thing I'm always struck by when I talk to people who are doing this sort of work is the pace of papers is just crazy. And you're evidence of that, because we're not just talking about one today. We've got a whole other one that we're going to spend a little time on very soon as well. When you look ahead to all those next steps, do you feel like it's just a matter of work? At my software company, the software engineers used to say it's just a simple matter of programming. And what they would mean by that is like, it's going to take us some work, it's going to take us some time. We're going to probably find some things that are going to challenge us a little bit along the way, but we'll definitely overcome those. And in the end, if it's a simple matter of programming, we'll definitely be able to get it done. And it's just kind of a question of putting in the work and getting over the little stumbling points. Is that how you kind of think about these next steps? Are they like, are you that confident that these are going to work? Not to say that there won't be little surprises along the way, but like overall, would you say you feel very confident that these next steps will all kind of lead to better models and, you know, better predictions and all that good stuff that we want?

Yeah, I think like it really depends on the whole community, like the collaboration, between scientists and not just like our own projects because, you know, we need like an amount of the data. And I mean, people have their own expertise in their domain area and then will be very helpful to accelerating the whole work to be, you know, into the next step together. So like we need definitely biologists to understanding the whole thing, like whole mechanisms of disease. And we need like machine learning person, like to creating those models. And then we also need more people like statisticians and other even artists, like to really kind of making our, like, you know, our maybe our model models and works like to be more fashion. Yeah, I would say it's really kind of like a working together things and collaboration is very important. Another case I remember, I think because we are dealing with the application of AI models into healthcare and biomedicines, we already care about how the accuracy is and we will be responsible for the patients. It's serious that we need to figure out how the model exactly can do and not in the opposite ways. So I think a lot of processes are required before it can go to exactly clinical available and testable. I think we need to experimental test and clinical test a lot of like pre-testing before exactly to publish the whole models to be applied into the real cases, you know, in patients, I would say. But in the lab scenes, I think there will be more flexibility to, you know, to testing because we will have ways to, you know, getting more experiments to testing the whole processes. Yeah. So it's all about the teamwork things.

Any last thoughts on SquidIF before we move on to talk about Coral a little bit?

Yeah, I think that's all the things. I really appreciate that you ask so many questions. I think really catch the key points of the Thank you so much.

Yeah, my pleasure. All right, well, then let's do the coral paper a little bit. This one, honestly, is even like a little more challenging, I would say, for a non-biology background person like myself to understand. But what intrigued me about— I wrote down several things that we can go through that sort of intrigued me about it, but maybe the most fundamental one is this challenge of connecting different levels of resolution or different levels of scale together into an integrated understanding that in, in so many, you know, ways, that's like sort of the fundamental challenge of, of science. It's like, you know, we going down to the very lowest level, I don't know exactly what we want to call the lowest level, but maybe it's string theory, maybe it's quantum mechanics or whatever, right? We have like pretty good theories, certainly at the quantum mechanics level at this point, of how a very small number of particles will interact. And then we can go up and look at atoms, and we can look at molecules, and we can look at proteins, and we can look at cells, and we can look at all the way up, right, to the economy. But there's often a gap between those where it's like, yeah, we don't know how to necessarily aggregate up these smaller units of analysis into the bigger thing that we care about. That's like even in economics, the micro and macroeconomics divide, right? We don't have a way of aggregating all the individual economic actors into an economy. So we sort of have this other top-down approach that's like measuring these aggregate statistics and trying to predict where they're going. And it seems like we have a lot of— I don't know how many layers you would identify, but it seems like we have that problem in biology at maybe the highest level of difficulty. And so I'm always intrigued by anything that I see that kind of tries to take multiple levels and integrate them and create an understanding that sort of bridges these scale gaps. So with that prompt, maybe tell me how you came to this, sort of what the motivation is behind this work and what you're trying to do with it.

Yeah, sure. I would say, like, this one is, like, more technical. So it kind of, like, addresses technical issues when we deal with the spatial data. So maybe I will first like to start by introducing about the spatial transcriptomics that majorly, like, addresses in this project. So we have discussed about the single-cell RNA sequencing data in the first paper. But as you know, like, the single-cell RNA data, they have one of the very obvious shortage is like we dissociate in the tissues, and which means that we don't have the location information of individual cells. This is very important because, you know, the cells are actually not individual things, like they are interacting with each other, and it's very important about like knowing what's the cell's neighbor is. So in that case, like we lost this information and it kind of like making the study of like architecture of the tissues or like the communications very difficult. So with that, people creating the new technology we call the spatial transcriptomics and also proteomics as well. Totally, we can say like spatialomics data recently, you know, just came out like maybe 7 years ago. And now there is like emergence of these current technologies about the spatialomics, and then more and more data came out. And requires computational ways to understanding these data. And one of the very significant issue of current ways to understanding this thing is like we have spatial mix, but like those data are not have the same resolution and they are also not the same sizes. You can consider as this, like you having an image of yourself, like a photo of yourself, like it's grayscale, high-resolution images, and you have another photo of yourself, like at a different time point or like a different, you know, like space. And then it's colorful, but it's like a mosaic level. Can you actually understand, can you actually making it into a better way? Like for example, you can getting colorful high-resolution images. So that is actually the goal of these projects. Well, but I think the photo is different with what we're dealing with because the photo always have like 3 channels, like RGB channel. But in our cases, we have very high dimension channels. For example, the spatial transcriptomics, as I said, we have whole genome level, 30,000 to 60,000 of the genes. But because of that, they will lost in information of the spatial resolution. So it's like mosaic and low resolution. But then in some other cases, we will have like high dimension or high resolution of the proteomics. So for the proteomics, like we— the resolution in the space is very high, but the limitation is that we can only get limited number of the proteins to be measured. So in that case, like they are not perfect in both of the datasets. And then we will have limitations to really understanding or like dissecting the spatial structures with the low resolution of the space. And very difficult to distinguish those like cellular heterogeneity with the low level of the proteins. Can we actually combine them together? And then we can have like a more comprehensive understanding about the tissue architectures and then how they are coordinated with each other and how they respond bound to the disease and how can we provide new strategies in the treatment. Yeah, that's the total goal of this project. So we wanted to address this issue with 2 of the technology. I would say there's new came out, like have the high resolution in both protein and in RNA, but still there is a high need that we need a way that addresses this like resolution differences. Oh, sorry, I forgot something. So in other cases, when we get— when we measure these like spatial proteomics and transcriptomics, the difficulty is that we cannot actually use the same slices. So usually these slices are at a different location. They are very adjacent, but still they are kind of like shifting in these cases. Like what I said in the photos, like we taking photos with the different locations and then we're making some more difficulties in the data dealing.

Yeah, it's just a good primer on the difficulty of doing all this sort of stuff, especially if it has to come out of actually a living person. I mean, it's a little easier, I guess, if it's coming out of an organoid or whatever that's developed in a lab. But you've got, again, in a living person, you've got a procedure to actually get the tissue sample. There's different, as you said, different times that those measurements might be happening at. They might be actually just distinct parts of the tissue nearby, but not the same. Then you have the challenge of the different kinds of measurements. And so this basically creates a very confusing picture that is not integrated into a single understanding. This kind of reminds me also a little bit of— I'm not by any means an expert on this, but in an episode maybe a year ago where we looked at a bunch of different applications of the new, what was then the new Mamba architecture to biomedical imaging problems, there was a similar thing that came up where it was like, we want to be able to integrate different kinds of scans, whether it's whatever, there's all kinds of different scans. You've got your MRIs, you've got your ultrasounds, whatever. You've got all these kind of different ways of trying to get a picture of the body, and they have their different strengths and weaknesses. And if you have 2 of them side by side taken at different times when the person was in a little bit of a different bodily position, whatever, Now you can have a real challenge to try to figure out how do I integrate these pictures, right? So one of the interesting applications that we saw in that Mamba use cases episode was taking different scans of different types and figuring out how they needed to be deformed in space so as to create one coherent single view that could allow a clinician to look at one thing and kind of see these different signals in an integrated way. So it's sort of a similar problem here where you've just got these different angles of trying to see what's going on, and it's very hard for us. This is another example of just where AI is like, so— and there's other techniques. Not all the techniques here are learned techniques. It's quite a a mix of things. But it's another indication. This has been like a huge theme of this podcast experience for me, is just seeing all the different types of problems for which humans just don't have good intuitions. And at best, we can get like a few experts to, with a lot of time and effort, develop the intuition. But then they can only process, even if they do develop that intuition, they can only process so much, right? And what we see just over and over again in so many different domains is that AIs are able to learn the sort of intuition that is needed to make sense of these things. And then obviously, once they do, they become dramatically more scalable. So those are just big themes that I think very much are echoed in this work. Another, I guess maybe you could— I do want to go to the synthetic data question in just a second, but before we do that, because that is another big theme distinct from the ones I was just waxing on about, what are the big sort of contributions here? One big one that I noticed was simply taking one of these higher scale things and deconvolving it, which is a word you don't hear too often. We hear a lot about convolving and less about deconvolving, but taking something that is higher dimensional but lower resolution and actually going down and saying, okay, this is what we now think each of the individual cells from this larger sample actually looked like. So maybe give us a little bit more on sort of what the practical upshot and value is that comes out of this whole setup.

I would say the deconvolving definitely is the major task that the model is trying to address. But the thing is also the model tried to providing a comprehensive analysis across like integration of the different modality. And, you know, transcriptomes and proteomics is like one example. We can also, you know, transfer to other type of modality as well. So besides this like high-resolution to be like deconvolved, I would say like the, in terms of the comprehensive analysis, As I said, like, it's very important to understanding like how the cells are communicating with each other. So in that case, the model will also modeling the interaction level of like individual cells. So we creating a graph neural network-based model and to also inferring the interaction between the cells. Another point is that once we had high resolution of this data, we can always like to exploring the single-cell level modality and, you know, the latent features of them, and then making the, you know, the spatial regions to be more like organized, and then figure out like the structures that happens in the tissues, and then study the tissue architectures. And here we call like, we identify the functional domains. So I think this is the major things that in the models. But, you know, I also forgot something. Maybe we can also investigate this spatial variability kind of like happening inside of the tissues. Yeah. Yeah.

And maybe just for people like me who are not well schooled in all this, I don't have a good intuition for like how, you know, obviously the body is like highly differentiated. You know, it's a a wonder of nature that we start off as even just one cell at one point, and then a blob of not super differentiated cells, and then all this differentiation happens. And then we've got, in the end, just many different types of cells. And some of them seem to have sort of gradual transitions, but then other times they have more more abrupt transitions. And so from my rudimentary study of biology in years past, I do have some sense of how segmentation sort of happens based on gradients of certain signalling factors. And it seems like a key assumption, if I understand this work all correctly, is that this is going to be fairly gradual. There's a couple of ways, I guess, that happens. One is that the— the graph neural network is sort of modelling cells interacting directly on their neighbours. And if there is a way that they're also modelling sort of longer distance interactions, I didn't see that. So that was interesting that it seemed to be like a very sort of local causal model. And then there seemed to be also a part of the loss function that was meant to try to keep things smooth so that as you go through a tissue and you see, like, okay, well, over here it's like the tissue is this way, and over here it's this way, there's some sort of an assumption that this is going to happen gradually through space. So maybe need to be deconfused about some of that. But the thing that also kind of— I'm having a little bit of a hard time sort of reconciling in my own mind is like, for certain parts of the body, it seems like the sort of barrier or the sort of boundary, let's say boundary is a better word than barrier, is pretty clear. When I think of a bone and then the tissue away from the bone, it seems like I know right where that bone stops and where the other tissue starts. And it doesn't seem like there's a smooth transition. Maybe there is, and I'm just not like, Zooming in, you know, close enough to see it. But yeah, I mean, that's more of a prompt than a question, but like, what am I missing there? Or, you know, what sort of additional information would like help me have a better intuition going forward for all this stuff?

Of course. I think like the tissue actually depends on like the type of the tissues that we are studying. Like, for example, as you said, like the bone is having more boundaries and then also like the blood vessel have like boundary as well. But in some of the cases, for example, in the tumors, for example, they have maybe not that kind of clear boundaries. They have some like infiltration of the tumor cells into, you know, the normal tissues. So I guess like it really depends on the tissues that we are studying. And in some of the cases, like it's not clear to identify their like structures directly, like, you know, from the morphologies. But if you're looking on the moleculars, they actually having some interesting patterns that existed, and we are only able to seeing that with the measurement of the molecular information. And in terms of the thing that you concerned, you mentioned arise that are related to how the models can capture these boundaries and also address these interactions, I would say like we are based on the quality of the spatial data we can get, like from, you know, both the high resolution and high spatial resolution and high gene resolutions. You know, like the model itself is able to modeling these interactions between the neighbor cells, but probably like not a little bit detailed in the models that we are not only just taking the most adjacent, we're also taking the nearest neighbor cells. And then we can adjust these like k values and then making this like, you know, the network graph like more bigger. But I would say it's not necessary to make it too long because we were just focusing on the most neighbor cells. in communications in these cases because there is like so many complicated things happen in the tissue. So the way we try to simplify is really to understanding the cell itself and also their neighbors look like. And in terms of the, I think it's the graph Laplace loss that I added in the model, we are making them into like more smooth, but I think it's like a regulation terms that we're trying to make models to be more realistic. And we have some other loss related to the reconstruction of the data, and as well as like the KL divergence things. And in that case, it will have a balance between the smoothness and the sharpness of the boundary. Because usually if we have a single-cell level of the data, they will have a clear boundary because the high resolution of the single cells able to distinguish at cellular level. So we were exactly seeing a boundary of the cells that related to the bone and also related to the blood vessels, which even give us like more higher or detailed structures that indicating the inner structures of that organs and tissues.

Gotcha. So the reconstruction loss incentivises accuracy, and sometimes there is a sharp boundary, and so that incentivises capturing that. And then the smoothness sort of tries to smooth things out in general, and that sort of acts as like a regularisation. And I mean, we've seen this in plenty of different setups, like a clever compound loss function can take you far. So you hope to get the best of both worlds there. And then did I understand also that basically the sort of k value that you mentioned, that's like a hyperparameter for how many nearest neighbours each cell will be connected to? So you could sort of scale up or down with just that single hyperparameter. Like, I just want to look at a very small number of the most adjacent cells, or Or I could look a little bit or even more kind of expand the radius with that hyperparameter.

Yeah.

Okay, cool. How about on the synthetic data side? This is something that I think in the context of language modelling has been— the community's been on a bit of a roller coaster with this where it's been like, oh, we have a data wall. Oh, don't worry, synthetic data will take us there. Then it was like, oh, synthetic data creates bad models that sort of have mode collapse or whatever. That idea got a lot of traction for a little while. I would say at this point, it's like we're pretty well past that notion, and it's pretty clear that just having models solve problems and doing reinforcement learning on that is working very well. Thank you very much. But in the biology context, yeah, I've just had a lot less exposure, and I have a lot less intuition for what sort of synthetic data exists, how much should we trust it? Can we train models on that and be confident? Also, even just the mix here, I don't think it was all synthetic data that was used, but there was some of it. So I'd love to just understand kind of the role that the synthetic data plays and sort of what are the limits today to like the quality of that data. And then, you know, as a result of that, like how much of it we can really use in a project like this.

Yeah. In my project, I also utilized the synthetic data as well as like experimental data to validate like how the model can work as what we supposed. And in terms of the simulation part, we usually like having Those like well-known technology or like models in the field that people creating. And for, you know, for this, for the both the single cells or like spatial as well, they are more realistic in related resemble to the experimental observations. And mostly probably similar like with other fields, like we are interested in like how their distribution look like. And then in those of like simulation models, they are actually to resemble the similar type of light distributions and sample that from the design, you know, structured distribution. And in my cases, SQUIDIF is like, you know, using the splatter. It is like well-known simulation ways to the single-cell RNA. But in the spatial, you know, the CORAL model, which dealing with a lot of like spatial data, we are actually working with our collaborator who is an assistant professor now in the University of, like, Connecticut. And he's designed a model called SA-Design, which published in Nature Biotech, I think, 2 years ago. And then this model is, like, able to sampling from design distributions that having the spatial information as well as having similar observations with the spatial and gene expression. Yeah, so this is about the synthetic data we are working on. We're having real data as well, and that is a little bit different because we have pre-knowledge about like what this tissue looks like, and then can we actually identify some interesting patterns that existed. And we're always having domain knowledge like to validate the model.

So safe to assume that just the reason we need The synthetic data, of course, I guess, is that it's just hard to gather the real data. Sometimes these things sort of feel like a little bit of a hall of mirrors, and you hope it's not a house of cards. There feels like there's something that can kind of be circular about it, I guess, where you're sort of saying, well, we've got this ability to generate this synthetic data, can we learn from it? But when I see that pattern, I also sometimes think, well, geez, if we can generate that synthetic data, don't we in some sense already know what we need to know? So, yeah, again, I feel like I'm just constantly asking you to help me sort of get less confused. But what is it that you are learning from the synthetic data that is not already in some sense learned by the process that generates the synthetic data in the first place?

That's a very interesting question. I would say, like, we actually were using this synthetic data to validate our models, like, really to capture the results we want. So we were not, like, exactly using the same type of generation process. We were having, like, a data generated, you know, synthetic data generated from a well-known or, like, a common way that people observed. And The good thing about this simulation data is we having the ground truth of that. Like, for example, we are interested in these interactions, or like we interested in the domain things. We know like the region or like the domain look like, or like the region look like. So we can kind of like validating whether the model have the ability to exactly identify that domains and also can deconvolve into high-res But the real thing is, you know, like in the real cases, we don't have that ground truth. And then once we can validate in the model, are able to perform well in synthetic data, we can transfer the models into real cases and then we can able to trust the results came out for the real data. And then we don't have any further information about that. So this is all about the way that we first using the synthetic data to validating the model, and then we will transfer the model to real data. And then they need to train separately. It's very— it's totally different things. And as I mentioned again, maybe emphasize is that most of the synthetic way to getting, you know, is different ways what we designed. So we will make sure that there's like no loop actually happens, you know, in the whole projects.

Yeah, gotcha. Okay. Yeah. When ground truth is hard to come by, things can certainly get really tricky. Do you feel like there's talk these days about a Manhattan Project for superintelligence and whatnot? And then when it's very fuzzy, often, like, well, what is that superintelligence supposed to look like? And what's it supposed to do? And are we going to be able to keep control of it? And so on. And I sometimes feel like when I talk to folks like you who are doing such detailed work, especially in biology, I feel like maybe a worthy Manhattan Project would be just to get all the data that we might really need to scale these things up and figure them out. And I suppose that could come about in multiple different ways. I have to imagine a lot of the data that we need is sort of locked up in medical record systems, and it's hard to share because it's privacy rules. And obviously those have their place, not to suggest that they don't, but it seems like liberating some of that data in some responsible way could be just a huge unlock. And then maybe even just like a major investment in the form of potentially just like government subsidy if we're going to subsidise AI development. Because I didn't finish the thought on the superintelligence, but what's it going to do? Whatever. Can we keep control of it? The number one most compelling answer to me generally when people get excited about superintelligence is it's going to cure all the diseases, we're going to live longer, healthier lives. It's going to be amazing, maybe double your lifespan, whatever. Okay, so I'm ready. I want that for sure. And I'm just kind of wondering, maybe instead of building a trillion-dollar data center and trying to create a superintelligence that can solve all our problems, maybe what we should do is get all the data so that people like you have a lot easier time of making the discoveries directly, and we don't necessarily have to go through this undefined sort of amorphous superintelligence. And maybe we end up with both. But I guess the question there is, how different could your life be in, say, a year or 2 years if we had a strategic nation-level project to make the biological data available That would facilitate the next couple of levels of scale-up and improvement in the sort of work that you're doing?

Yeah, this is a very interesting question. I would say there are actually many, or more and more projects came out trying to kind of making the models big and then making the super intelligent models to very resemble the human bodies. And for them, I think, like, for example, I think Google DeepMind, they have creating projects like virtual cells and they want to having a unified models that can making all these different type of cells and organs and then in different scales together to really having a digital sort of copies of all the cells. And also I know that CZI, the Chan Zuckerberg Initiative, they also launched another project they call like Billion Cells, and they will have like a lot of cells together to building such like a similar model as well. So I think there's like very exciting, this is a very exciting period that people are really utilize those like foundation models and then large data, like large-scale models to address these questions. And we have GPT and people are also creating the biological version of the GPT, you know, to answer any questions related to the cells. And then in other cases, like people interested in the, you know, the clinical questions or like the health, healthy questions. So we can have like virtual doctors and there's like many ways. And I would say there's like a lot of applications, very excited time points that we are trying to working on now. And, you know, people are also working on digital twins style projects. We can maybe have a copy of our own self and then we know what's going on if we're making different choices at some time point and then what we will look like after that. That is also very interesting. And yeah, I think there's a lot of very exciting and interesting questions that came out and I would say we're excited to see how the model will go into 1 to 2 months, sorry, 1 to 2 years. And looking forward to the fields, how they will develop.

Yeah, it's an exciting time for sure. And a little scary, but this sort of work is much more pure upside, it feels like to me. I mean, obviously anything like in some possible scenario could be abused, but I don't think we have to worry about Squidiff or Coral getting out of control. It is focused work that is answering specific problems. And it's a good reminder that such work still happens and still has value. And it's not all just about embracing the bitter lesson and maximizing the cluster size that you can throw at something. So I really appreciate that about both of these projects. I think that's about all the questions that I had for you. Is there any other next steps you want to sort of tease or any other thoughts you want to leave us with?

Yeah, I would say like, I think I'm very optimistic about like how the AI can help with the healthcare and also biomedicine. I know people may be like scared about like how, how like fast They can grow. And at least like for now, I don't think there is like a way ready to, you know, scare about like how the AI can destroy, you know, the world of probably like for now. Yeah. Hopefully I think the researchers like who are working at the intersection of the AI and plus the healthcare will making our development of the healthcare like to be more, to be quicker. And then people will have a better life. You know, the styles after the studies, like all of our projects. And that's all I think I want to share. And yeah, thank you so much for this great opportunity and very excited and delighted for discussion with you. Yeah, thanks so much.

My pleasure. See you, Huh. Thank you for being part of the Cognitive Revolution.

Thank you.

It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please don't hesitate to reach out via email at tcr@turpentine.co, or you can DM me on the social media platform of your choice.