Scientist Stories: Johannes Textor, Immunological Intelligence

Hello and thank you to everyone around the world for joining us for this week's inaugural Immunome Lab Meeting. My name is Kristina Bood. I am the science editor at the Human Immunome Project, and I will be your moderator today. For more than 2 years, we've been hosting the Global COVID Lab Meeting webinars as a way to provide our audience with the latest insights on the COVID-19 pandemic from experts around the globe. These meetings have helped all of us understand the myriad ways that scientists are trying to study and combat SARS-CoV-2. This virus and our work to thwart it have also illustrated some of the lingering mysteries of human immunology. In many ways, scientists still don't understand the variation in how different people's immune systems respond to vaccines, therapies, or infectious and non-communicable diseases. Solving these mysteries is one of the main goals of the Human Immunome Project. Deciphering the complexities of human immunity and developing AI models of the human immune system could lead to far more effective vaccines, diagnostics, and therapeutics, and in doing so, usher in a new era of human health. In this new Immunome Lab Meeting webinar series, we are going to showcase the research of leading scientists who are working at the frontiers of human immunology and machine learning to see how the power of AI can be applied to understanding more fully the diversity of human immunity and how this can be applied to improve health. We are very excited to kick off this monthly webinar series today, and we hope that you will help us spread the word to your friends and colleagues. It is my great pleasure to now introduce today's speaker, Dr. Johannes Textor, an associate professor in the data science and molecular life science departments at Radboud University and Radboud University Medical Center in the Netherlands. Dr. Textor and his team study the adaptive immune system through the lens of computation and learning by building in silico models of the human immune system and combining these with various types of experimental data, they aim to understand the essence of what makes the immune systems of each human capable of learning and adapting to changing environments. Dr. Textor holds a VD grant from the Dutch Research Council, a program grant from the Human Frontiers Science Foundation, and was a visiting scholar at the Simon Institute at the University of California, Berkeley. During today's presentation, please send me your questions using the Q&A function in Zoom. I will ask our speaker a broad selection of your questions after the presentation, and then we will have time for discussion. With that, it is my pleasure to now turn it over to Dr. Johannes Textor.

Thank you for this nice introduction. Can you hear me? Is my sound working?

It sounds great, yes.

Great, thanks. So I'm going to start to share my screen. And yeah, it's a great pleasure to kick off this meeting series of the Human Immunome Project. And I'm very pleased and actually honored to be the first speaker in the series. So today, I want to talk about the concept of artificial immunological intelligence. And I thought, because it is the first meeting in the series and the entire project is about an AI model of the human immune system, I wanted to start with some kind of historical perspective on intelligence systems in the body. So AI, as we mostly understand it, is a computational methodology that in its modern version at least is based on and inspired by the architecture of the human brain. The human brain can be seen as a network of neural cells and their connections.. And this has been used very, very successfully as a template for computer algorithms. It is often forgotten that actually there is a second system in the body, which is the human immune system, which is, yeah, I would argue at least as fascinating and complex as the human brain. And so here we can see a picture of the human immune system. This is an intravital movie, and the cells that you can see here are T cells that are moving in the lymph node of a living mouse. So, these two pictures next to each other, they make one important difference between the brain and the immune system very clear. T cells, and actually most of the cells of the immune system, are constantly on the move. So, they're not stationary in one location, but they actually travel throughout the entire body. This is not only true for T cells, but also for B cells, dendritic cells, and most other cells in the immune system. The second important difference between the brain and the immune system is that communication between these cells is mostly local. And that means that cells, they communicate to other cells that are next to them, for example, by receptor-receptor interactions or cytokines that have a more local range of action. So these two systems, they are both intelligent. They're both capable of learning and creating memory and adapting their behavior to the environment, which is one way to define intelligence. But yet they have completely different architectures. And so what I find fascinating, and many other people too, is to understand the implication of this difference in architecture for learning and information processing. So my talk today is going to be about 3 different things. So first of all, I want to take a bit of a historical perspective because this is the first meeting of the Human Immunome Project. I want to first address a bit the question of what is or what could an AI model of the human immunome actually be, or how could that look like? So that will be a more general introduction to this concept, and I'm going to show some work that has been done by researchers in the past on exactly this question. In the other two parts, I'm going to briefly showcase two pieces of work that have been done in my own research group. The first one is where we built an AI model of a part of the immune system to understand a specific process in T cells called negative selection. And if I get to it in the third part, I will show briefly our work to use computational models to understand the movement of T cells. So I've picked these two parts because they align well with these two very important aspects of the immune system that differentiate it from the brain. This is distributed decision-making by lots of different agents and movement, which is a very important aspect. Okay, so I want to get started by showing the basic concept of what we understand information processing in the immune system to look like. There is a term that has been coined many years ago, which is called the immunological synapse. And so an immunological synapse is an interaction between a cell that presents some antigen. So this could be this cell here on the right. So for example, the cell might be infected by a virus, and in this case, it would present small peptides that are taken from the virus sequence on MHC molecules on its surface. And then this detector cell, which would be a T cell in that example, would use its very specific receptor to inquire this peptide that is being presented. So this exchange of information happens at this interface between the MHC bound with the peptide and the receptor on the detector cell. These interactions are very diverse because detector cells have very different receptors and they are extremely specific, meaning that each detector cell can only recognize a very tiny fraction of the possible peptides that can be presented. And this basic interaction is what we call the immunological synapse. Now, if you now take this concept from one synapse to a system of many synapses, then you can come up with this model, which is a very simplified computational cartoon of how information processing in the immune system works. So one way to define this would be by using the analogy of pattern matching. So basically, each of these circles here represents an immune cell. This could be a T cell, for example, or maybe even a B cell that carry receptors that recognise very specific molecular patterns. And I've just simplified this here as patterns with the stars, as we know them maybe from pattern matching. So these cells, they are heterogeneous, they can execute different actions. For example, this cell could be a killer T cell that might mount an immune response if it recognises a ligand, whereas this cell might be, for example, regulatory T cells. The initial state, the system is simply a collection of a lot of different cells, and humans typically 10^10 T cells, for example, that are very, very different from each other. And by this large diversity of receptors, they can cover the entire space of possible antigens. Now, how does a system like this learn? A system like this learns by a process that we know in machine learning as reinforcement learning. So it will be shown peptides or cognate information, and along with some context signals— it could be, for example, cytokine— that triggers some kind of action in T cells. So, for example, if we understand the development of repertoire throughout life, then we could model this as certain peptides being presented by self, for example, those would be presented in the context that encourages T cells to tolerate these peptides. Whereas peptides presented together with so-called danger signals, they might encourage these killer T cells to respond. By doing this for a while, we will get the expansion of specific clones in the repertoire, and we will also get a diminishing of other clones that never get to respond over time. Over time, the system will have an imprint of the environment, and that imprint is a result of this learning process. That's called reinforcement learning in machine learning, but it's done in a very different way than, for example, in a neural network reinforcement learning algorithm. So this basic idea of using these so-called artificial repertoires to represent information processing in the immune system is fairly old. It has been actually introduced at the end of the 1980s, and it was known under the name of artificial immune system. So basically, an artificial immune system is any computer algorithm that is inspired by actually the immune system broadly. And the context of this talk, I'm going to restrict myself to algorithm is inspired by immune cell repertoires. So any computer algorithm that is based on the concept of repertoire and pattern matching, that you could call an artificial immune system. So these systems have been invented, I would say, in the end of the 1980s, and you could see them as the counterpart of artificial neural networks in immunology. So why would these systems be useful? Who needs them? And why do we actually do research on this kind of systems? So there are two different ways in which you could use such a system if you had one. The first would be that you could use a system like this to directly simulate immune responses. If the system was actually able to very accurately, you know, map the actual information processing by T cells, for example, then you could use it as a sort of prediction algorithm. So for instance, here you can see peptides. They are all peptides that are predicted to bind to the HLA-A2 molecule from HIV. And the blue ones are peptides that are not known to be immunogenic. So, these would be peptides that are never— there has never been any T cell response documented against any of them. Whereas the red ones are peptides where we know that they are immunogenic, at least in some people. They contain a few famous ones like this peptide here that has been studied a lot. So, we still don't fully understand the difference why some of these peptides are immunogenic and the others are not. That has been actually a very, very studied question, but there's been some limited progress, but still we can't really understand very well what separates the ones that are immunogenic from those that are not. If we had a way to simulate this whole process, if you could just say, well, I'm going to use my artificial immune system and I'm going to give this peptide to it, and I'm just going to simulate whether there's going to be a response or not, then this would have immediately very many useful applications, one of them being vaccine design, right? So for example, if I could simulate the response of this person to a vaccine, a T cell vaccine in this case, in which I include certain peptides, I might be able to predict which person will respond to which peptides. And that prediction should actually be personalized because every person has a very different repertoire of T cells. So if we could have an algorithm like that, and if you could measure all these sequences that are expressed in humans, then we could use this model to predict immune responses and, for example, use that to personalize vaccines. Now I'm saying if we had it, because in reality there is no such model capable of doing this at the moment, and there's actually quite good reasons why we all expect this to take a substantial amount of time until we have something like this, if it is even possible. But it's just the kind of big goal that we are working towards. Another goal, and that has actually also been considered side by side with this more biomedical approach from the beginning, is that you could use an artificial immune system to protect your computer, right? So this is an article from the 1990s where they suggest that by using analogies from immunology, we might be able to build more robust computers that would be less prone to, for example, intrusions by hackers or by computer viruses. Viruses. If you honestly want to evaluate the success of this idea, then we have to now see that 25 years have passed since this article, and while our computers have become somewhat better at protecting themselves against viruses, this is not based on artificial immune systems at the moment. So you can see that this idea initially was quite popular, but has not really ever taken ground or has not really ever become mainstream. But then again, you could say the same thing about artificial neural networks 25 years ago. Okay, so I think by this I should have given you an idea about what artificial immune systems are and why they have been developed. So there's another interesting parallel to neural networks, and it is that one thing that we do know about these systems is that we have to make them quite large in order to do something meaningful. So the state of the art back in the days when these earlier systems were built was maybe something like this. So you could have an artificial immune system that consists of 10^5 cells, right? So that would be the size of the immune system of a tiny tadpole. If you wanted to scale this up to a realistic size of vertebrate immune So then this is several orders of magnitude away. And there have been people studying this and back 20 years ago, maybe people really concluded that this would be too difficult. So basically that there's no way that these systems could ever work because they just would have to be too large. And so it would not have any potential to ever become useful. Again, I want to point out that a lot of people would have said the same thing 25 years ago about artificial neural networks, and yet that turned out to be solvable through various advances in engineering. So also in the field of artificial immune system, time has not stood still in the past 25 years. There have been some developments, and my group has been active in this field and many others. And we have developed techniques, for example, data compression methods, which I'm not going to explain in detail, but which are able to create simulations of large immune cell repertoires much more quickly than what used to be possible. If you want, if you're interested in this, you can have a look at the code here. So there have been some developments, but still we are not at a point where we can actually build these systems at a scale that we need and actually do something useful with them that would match the success of artificial neural networks. That's also why I'm so excited about the Human-Emu project that hopefully will really make progress in that direction. Okay, so this was my kind of general introduction to the field of artificial immune systems, and now I want to showcase some work that we did using artificial immune systems broadly defined. And the first part, I want to explain this work here, which was done by Inge Wortl. She was a postdoc 5 a PhD student and a postdoc in my group, and now she's an assistant professor. And Inge studied the question of, do T cell repertoires learn something during negative selection in the thymus? So this process of negative selection is something that we studied using an AI model, simplified one, of the human immune system. So I thought it would be a nice fit for this meeting. Okay, so I'm just going to recap the very basic immunology that we need to understand this work. So most of you probably know this, but T cells, as I mentioned before, they have very specific receptors. These receptors recognize short peptides presented by MHC molecules. For example, the peptides presented by MHC class I molecules, they tend to be about 9 amino acids long. These peptides are produced constantly in every cell, every nucleated cell, by chopping up degraded proteins in these proteasomes. These proteasomes produce peptides, and these peptides are loaded onto MHC molecules using the TAP. So this is something that you could see as a window into the normal metabolic processes that happen inside every cell constantly, and in normal circumstances, you will only see normal peptides showing up there. If, however, for example, there is a virus that has infected the cell and is starting to produce abnormal proteins, then this will also mean that abnormal peptides from these proteins are going to show up here. So from the point of view of machine learning, the T cell receptor's task is to discriminate normal proteins, normal peptides presented on MHC, from abnormal peptides presented on MHC. Initially, when a T cell is made, there's a highly sophisticated process that generates the receptor of the T cell that involves random rearrangement of gene fragments and even insertion of random nucleotides. So that means that there are initially lots of many, lots of different T cells with different specificities created in the bone marrow. So now this random generation of T cells has the advantage that you can potentially make a T cell against pretty much everything. But there's also an inherent risk, and that risk is that you could make T cells that are harmful, that are basically going to attack your own body. And the textbook theory about how this is prevented is that T cells, after being created in the bone marrow, they would migrate to the thymus. The thymus has a specialized cell population that presents lots of different peptides from different proteins, and it's going to expose that T cell to an environment where lots of different peptides from self are going to be presented. If the T cell reacts to any of these peptides presented in this specific environment, it's going to die. And only T cells that never react to anything, they are allowed to eventually leave the thymus and become mature T cells. So this would be a textbook idea of how you first create a population of random T cells that are potentially autoreactive, and then you remove these autoreactive T cells from the thymus. However, there has been some recent data that made clear that this is really not that simple. So actually, if you look at the population of escaping T cells, there are lots of T cells there that can still recognize self-peptides. And it might be, for example, because not all of the peptides that you could make in the body can be presented in the thymus within a reasonable amount of time. So we asked ourselves this question here, If only a subset of the peptides is presented in the thymus, is then the resulting system still capable of some learning? So can a system learn to tolerate self and discriminate self from foreign by just being shown an example of all the self-peptides, so for example a subset of the self-peptides? And to answer this question, we built now an artificial immune system, basically a simulation of the T cell repertoire, and we let this simulations solve a toy problem. This may seem like a very unusual approach, so let me first explain what we did and then why we did this. So we made an artificial immune system that looks like a T-cell repertoire, but instead of discriminating peptides that are made of amino acids, we let the system discriminate words, words from different languages that are made of letters. So there are a few reasons that we did this. So the first reason is that we can tune this problem such that the information here is similar in, at least in quantitative terms, to the information that is processed by real T cells. So we can make, we can chop strings, words, up into small sections. And there are 20 different amino acids and about 26 different letters depending on which alphabet you use. So we can tune this to be quantitatively similar in the amount of information that is presented. The second, but maybe much more important, reason is that we actually understand this data. So we can look at this data ourselves, we understand what the differences are, and then we can actually use this to deduce some properties of how the system learns, if it learns. In this other problem, we don't really understand very well yet how self and foreign are discriminated in the first place. That makes it harder for us to use this directly as our input for this artificial immune system. Okay, so what we did is we took the Bible, and we did that for the reason that the Bible is translated into very many different languages, which was useful for this research. And we built an artificial immune system that can recognize strings from different languages. So how did we do that? So we need 3 different ingredients. The first is we need the actual data, and as the data, we used 6 letters of text in any language. So for example, it could be 6 letters strings from English in this example here. That's our sequence. So that's basically the data that the T cells see. We use 6 letters because that's about the amount of amino acids that the T cell sees, which is actually not buried in the MHC binding groove. So this would be a similar amount of information than that is being seen by a CD8 T cell. The next thing that we need to define is how our TCRs, T cell receptors, look like. And what we did here was we used the so-called implicit definition of pattern matching. So each T cell receptor itself in our model is a string, and the string defines a pattern of sequences that the T cell receptor can recognize. This sequence matching function is called affinity, and this You can define this in very many different ways. So what we did here is we just took a definition of affinity that was used by the people who made the first artificial immune systems, Stephanie Forrest and Alan Perelson and colleagues. And so they defined the affinity between the T cell receptor and a string as the longest stretch of contiguous stretch of letters that is identical between the two. So for example, if this on the bottom is my T cell receptor and this on the top is my peptide being recognized, Then here the affinity between the two would be 4 because there's a sequence of 4 contiguous letters that is the same. Here we have 2 contiguous letters at most that are the same, there's an interruption in the middle, so we say this has affinity 2. You can actually define this affinity in other ways, you can use the so-called Hamming distance or you can even look at the similarity between letters, it doesn't really matter all that much for what I'm going to show, but I just want to make The point here that this is a kind of a modelling choice that we make. Now when we make a system like that, we can now build repertoires and simulate negative selection based only on the rules that I just showed you. So we start by creating a pre-selection repertoire. In our case, we just made a complete repertoire of all possible motifs, which is 27 to the power of 6. 27 because we have 26 letters and we can use this underscore to represent the boundary between words. Then we are going to train this. In machine learning terms, you would call it training. So we say that we take this repertoire and we take a training set of data and we delete all of the strings from the repertoire that recognize any of the strings in the training set, and that's what we call negative selection. So that's basically a simulation of this process. Now, after we are done, we end up with what we call the post-selection repertoire,. And this is now a trained repertoire. It has basically been exposed to some information from self, so while this pre-selection repertoire has no relationship to self, the post-selection repertoire contains a kind of imprint of the self. And now we can expose this repertoire to different strings coming from, for example, English and some other kind of language. Here we're using a Xhosa language, which is very different from English, as an example.. And we can now test whether, for example, the resulting cells recognize the Xhosa strings which are foreign to them better than other English strings that they have not seen during training. And yes, indeed this happens. So for example, if we are making a model like that and we show it 500 strings from the Bible in English, and then afterwards we're comparing the recognition of this repertoire against English strings versus strings from Xhosa, then we see that the so-called precursor frequency, so the amount of T cells that recognize each of these unseen test strings, is much higher for Xhosa than it is for English. We can also look at the ones it recognizes the best, what proportion of those is foreign versus self. And so what we see is that after learning After being exposed to a couple of hundred of these training strings, the system actually learns to recognize a foreign language. And actually, that's not that much information. 500 strings, that's about a page of text maybe. So after that, there's already certain decent performance even in solving this admittedly very simple problem. So what we see here is that we can state that the system has learned to generalize. So our artificial immune system is able to learn something. It can learn in a way how English looks like by just being shown some example letters, some example strings. So why does this happen? And this is now something that we can investigate in this toy problem and that we can also conclude from what would then be the requirement for the same thing to happen in a real human system. So the first thing that we need is that the same TCRs tend to bind to strings from the same language. So for example, if I make this graph here where every point is a string and I connect them, if there's many T cell receptors in the model that can recognise both of them, then these lines, they should go between strings of the same language. We can measure this, we can call this, for example, concordance. And here we see that the concordance in this case for English and Xhosa is very high. So alignments tend to go between strings that are in the same language. So T-cell receptors, if they recognize two different strings, they tend to be from the same language. If I would do the same thing with a different test set where I just take one set of English strings and another random set of English strings, then this would not happen because then basically the concordance would be 50%. It would not matter for recognition of a pair of two different strings which languages they come from. So it means, in other words, that this system can only learn to distinguish between languages if there's a difference between a language in the first place. So the languages have to be different for them to be distinguishable by a system like that. And there's actually another requirement that we can see, and we can also infer that from this graph, and it has to do with the specificity. So remember that I had this parameter in my model where I could tune how many peptides each T cell receptor can recognize. If I set this parameter very high, meaning that the T cell is very specific and it recognizes only a single string, then nothing can be learned because a T cell that can recognize only one string will either be deleted or not during negative selection, but there won't be any consequences of that for the recognition of any other string. So this means that no learning occurs in the setting. If I set my cross-reactivity too high or my specificity too small and the t-SNE recognises almost everything, then there can also not be any learning because then they are not capable of picking up on these differences. We have to set the specificity to an intermediate level Otherwise, if this doesn't occur, then even if the languages are different, they cannot be distinguished. So the second requirement, as I mentioned, is that there should be a difference between self and foreign to begin with, and we can actually quantify this somewhat by doing the same thing that we just did for many different languages, and then we can see that the recognition performance of a model like this is very much related to how different these languages are. So unsurprisingly, the model has a much easier time distinguishing Xhosa from English than, for example, medieval English from English. Okay, so these are the two things that we learned from our toy model: that yes, a negative selection can lead to learning in the immune system, but these two conditions have to be fulfilled. We have to have an intermediate specificity, and we have to have a similarity between self and foreign that is not too high. Okay, now let's go to the real thing, because we don't care so much about distinguishing languages, and that's very easy to do, but the immune system has to really distinguish, for example, the peptides of your body from the peptides of COVID or some other pathogen. So to test that now, we use the same model, but now we actually fed it with real data coming from proteomes. So the Human Proteome Project is just published, you can download the sequences of all known proteins, you can run them through prediction algorithms so that you can which ones end up on the MHC class I molecule. And then you can just run the model in the same way that we did before. We use the same mining function, we're just now applying this to peptides and T cell receptors. Now there's a very disappointing thing that we do see immediately when we do this, and that is that actually this second requirement that we had for the similarity of the languages does not actually hold in the case of peptides from pathogens or the self. So while there was a very large dissimilarity in the languages example that we could exploit, actually even the peptides of very foreign-sounding pathogens such as HIV or hepatitis are actually quite similar to self-peptides. And if you think about it, it makes sense because the human proteome contains a lot of proteins that are quite strong homologs of proteins that viruses also have, right? So there's no reason to assume that two random peptides from self, they should be more different than a random peptide from self and a random peptide from the hCMV, for example. So that is disappointing because it means that probably the learning isn't going to be all that great, right? We can also look at it in the same terms as before with this concordance graph. This on the left side was our Xhosa versus English case, which was really nice and easy to learn by our model. But if we expect our model to do the same thing for HIV versus self, then it's going to have a very hard time. So that's going to be super difficult, it seems. And yes, it's also what we see. Even if we train our model on 200,000 self peptides, then it will still have a very hard time discriminating HIV from self. It can do it to some extent, but really not great. So actually, cell phone discrimination by the immune system, that seems to be very hard. That's our first lesson here. Okay, so, and that's actually, yeah, that is sort of in line with the recent data that we know that actually shows that there are lots of self-reactive T cells in the periphery in humans, and that they are probably then silenced by other mechanisms of peripheral tolerance. But this shows that negative selection alone is very unlikely to achieve this, which may seem obvious if you've done this, but it has been in the textbooks for decades that negative selection is what does this. Now, we went a bit further here, and this becomes a bit speculative because something that you immediately ask if you see this as a machine learning person is then, okay, so we picked this training set when we trained our our model with, we picked this at random, we just used random peptides from self. And that's possible certainly, but it's not very efficient because it means that you're going to use lots of different peptides that are actually pretty similar to each other. So they end up deleting the same T cell receptors and that's not super efficient. If you want to make a machine learning model that actually works well on all kinds of subclasses of your data, you also try to enrich for examples that are a bit more difficult. And so we wondered how, yeah, is this maybe something that our artificial immune system could also benefit from if we, instead of using random self-peptides, we used non-random ones? And yeah, so in the computer, of course, we can just compute even what the optimal training set would be. I'm not going to say how we did that, but there's a way, an algorithm that you can use to compute the theoretically best training set of any given size. And when you do this, it really makes a big difference. So you see that these theoretically optimal training sets, they delete these cells much faster than randomly chosen ones. And they also, and that's not super obvious actually, lead to much better discrimination between self and foreign. So this is quite striking. It means that, for example, by showing less than half of the human self, you can now achieve much, much better discrimination between for example, HIV, and self then by using random selection of self-peptides. So it's not quite obvious why this happens. So, but it basically is the case because the peptides that we enrich, that we remove more they tend to be more self-reactive peptides. So by using this optimal training set, we don't lose as many of the foreign-reactive TCRs, but we keep more of the foreign-reactive and we remove more of the self-reactive TCRs. So, okay, but that's maybe interesting, but it's something that we computed, right? So in the computer we can do this, but the question is, how can you do this in an organ like the thymus, which has to basically make the selection? So we also wondered, is there any biologically realistic mechanism that could potentially do something like this? Well, yeah, so for example, one thing that we noticed is that these optimal training sets, they were enriched for peptides that contained rarer amino acids such as tryptophan, which is much less common than something like leucine. So when we, for example, compute the rarity of the different amino acids in the peptide, then we see that there's quite strong correlation here with this exchangeability, so basically by how useful they are for negative selection. And it means that if we would enrich somehow for these rare amino acids, then okay, maybe you don't get the optimal training set, but maybe we still get something that's better than just choosing at random. And yeah, so you see that this does help if you build a bias into your model that it preferentially presents peptides that have rare amino acids in them, this will improve the performance quite a bit, actually. And this is something that you could, yeah, at least, you know, imagine that could happen in the thymus, for example, there's special proteasomes that get activated, there might be some kind of bias in the entire molecular pathway that needs MHC presentation. So We don't know, it's purely speculative, but this is what we call a model prediction, right? So we have seen something in the model, we have made certain changes, and we predict that if there was a bias like this, it would actually help the immune system to discriminate self from foreign. And of course, it's something that has to be tested in experiments to be confirmed or rejected, but this is what we call a model prediction. And I think that's an important thing to recall in the context of building a human immune system model, is that the model is very likely not going to be perfect, there's going to be things missing. So one critical thing is to really try to make predictions that you can theoretically validate or falsify using experiments. Okay, so when we do this for all kinds of pathogens, it really invariably helps. So having a bias for these rare amino acids, that's something that seems to be beneficial. Okay, so I'm summarizing this here, and I just want to state what have I learned. We learned from this model that yes, in principle, negative selection in T cells is actually a learning algorithm, and you can use this as a classifier. So you can actually make a computer model of this process and classify text and actually arbitrary other data if you want to. However, in reality, the self and foreign classes, so basically peptides from viruses and human self, they are very similar, and that makes learning very hard., and to overcome this obstacle, the thymus may have to present some kind of non-random training set. We don't know if this happens, but our model predicts that this would be helpful. Okay, so this was my second part, and I'm just going to go briefly to something that is a bit more an outlook part. So this is a project we're working on, it's not published, but I just want to show this because it's a nice example of how you can study The other important aspect of T cells versus neural cells, which is movement. This video here shows T cells on the so-called under agarose assay and that are moving in a crowd. And so what is this is an in vitro system, but it is a very specific, very typical situation for T cells to be in. First, I want to thank my team that I work with on this project. So this is a project of the Human Frontier Science Foundation consisting of 3 different countries. I'm going to be finished very soon, by the way. So this shows a simulation of a lymph node, and I'm just showing a simulation as opposed to the video I showed you earlier, just to emphasize that the real lymph nodes will be packed with T cells. So unlike the video that you've seen where there were only a few of these stained T cells, this is really something that is completely crowded. So the question is, how can the T cells now thrive in these large dense crowds? We know from other crowded systems that if you have crowds and they have to be in challenging environments, all kinds of things can happen. For example, at a room where there's a small exit door, there can be jams. Preventing people from exiting even though they are trying to exit, which is called the faster-slower effect. And so this is something that we're interested in and studying quite a bit recently is how can T cells navigate in this very constricted and dense and still moving environments. So something that people did in crowd dynamics to get a basic understanding of the crowd behavior of different systems is that they put individuals into single lanes of traffic. So for example, you might think about humans in a narrow corridor, or maybe even ants. And when you do this with different animals, in this case like humans, you can see very different types of behaviors. So for example, humans, they are very bad in crowds. They tend to slow down when you approach higher densities, and they jam. Whereas ants, which are more known as kind of social insects, they can maintain their speed even at high densities, almost at the same level. So we were kind of wondering where do the T cells fall in the spectrum. So we combined some in vitro imaging where we can use microchannels to actually build these lanes to make single T cells move that can only move in one of two directions. And we also have a computational system of the same as a combination simulation of the same system. So to just show you one result is that we found is that yes, the T cells are predicted to behave more like ants compared to humans. So when we look at the fundamental diagram of speed versus density, then we see that T cells at high speeds, they can— high densities, they can move as fast as T cells at low densities. So they make actually these trains, and the trains, they can move very efficiently backwards and forwards. Okay, I'm actually going to skip this last part because I think my time is done. And I just want to summarize that T cells seem to be quite able to navigate in dense, crowded environments. Judging from some of the in vitro data that we've seen, they seem to be quite resilient to crowding issues, and maybe even more so than other self-propelled agents. With this, I want to wrap up by thanking, first of all, again, the organizers for allowing me to speak and also my team, the Computational Technology Group at Radboud. Specifically, I want to thank Inge and Shabaz who did most of the work I was showing and our collaborators and also our funders.

So thanks for listening. Thank you so much, Dr. Texter, for running through all of that really fascinating work. So I'm going to read off some of the questions that have been coming in on the Q&A. The first being, what gaps does the toy letter and word model have when you think about the buried amino acids that are not seen by the T cell receptor?

How could this impact the results? That's a good question. So actually, we did some of that. We included in the model because I showed that, for example, used only 6 strings, 6 letters as opposed to 9. So the whole peptide is 9 amino acids long, but at least position 2 and 9 are known to be kind of buried in the MHC, and that's actually why we took them away and we focused on those 6 letters in the middle which are not buried. But honestly, it wouldn't change the results so much. So this result is very independent of how exactly you set these kind of details. But yeah, it's something that we indeed considered as well.

Right, right. Okay, so we have a question on whether concepts of structural biology like binding energy can be applied to make a model for negative selection, and has this been tried?

Yeah, that is a very great question as well. Yes, it has been tried, and it's hard, right? So you know that there have been very great advances actually in the 3D binding predict world. Structural prediction, first of all, using AlphaFold, but now the problem of predicting the binding between two different proteins is even higher, is even harder, and it's especially hard in the case of T cells apparently. So I'm not a structural biologist myself, I know a lot of smart people have worked on this, tried to indeed make progress in that direction, and seems to be something that will take quite some time.

Yeah. Okay, so your example that you provided was based on T cells. It seems that B cell learning might be even more complex.

Would it entail big changes in your model and your approach to answer that question? Yeah, another great question, but I think these very, very basic things that I've shown you, like the requirement for similarity and cross-reactivity, that should hold in almost every conceivable setting, even for indeed more complicated B-cell binding, but it might be weaker and more difficult to detect. That's my hypothesis, but we haven't addressed this yet. But yes, it's a very important point.

And then we have a question on whether you could say more about the optimal training set detection algorithm, and does it do anything to minimise the difference between self-presented and foreign? And would there be any biological analog that you know of?

Yeah, yeah, this is the technical thing I skipped. So we used— actually, this is a combinatorial optimization problem, so we really use something called integer linear programming to solve this. So it has no— it's like the real optimal thing has no biological equivalent, I think, that I know of. This was really something theoretical that we just wanted to know. That's also why we then moved to something that a kind of more realistic biological interpretation, which was not quite as good, but still good, and at least is kind of conceivable that this might even be the case in reality.

Yeah, okay. And can you define for us what you mean by rare amino acids?

Yeah, so amino acids are not uniformly distributed, right? So there are, yeah, the most common one is leucine, for example, and the rarest one is tryptophan. And leucine is, I think, out of the top to happen, maybe 20 times as frequent. So rare one is just one that if you look at all proteins, it's just found less frequently. And that's, yeah, that's just the fact of biology that some proteins— sorry, some amino acids are more common than others.

Okay, all right. And we have a question on whether you can estimate how many human peptides originated in viruses over the eons.

That's an excellent question. I wonder if somebody studied that.

Honestly, I don't know. Well, we won't make you guess. So we have another question, which is that language processing has been a rich area of research for machine learning and AI. What other applications do you see for the field of immunology from the machine learning AI field?

If I get this question right, it's asking about possible application for immune and Inspired ML?

I don't know. Inspired by the language processing, which has been very—

Yeah, exactly. So language, I think the language is not an application for this. It's just something that we did to understand some very basic concepts. I think this has been a question that the field has debated a lot, like what would be the kind of killer application for an artificial immune system? And there has been none so far. And my best guess is something like adversarial information where you have to deal with information that somebody makes it as hard as possible for you to distinguish things or maybe tries to, you know, make your neural network think that it is looking at a cat while it's actually looking at a dog. And we know that neural networks are very sensitive to these kind of manipulations. And one hypothesis I have is that artificial immune systems might be less prone to this kind of thing. But that's something that we are investigating at this moment. Right, right.

So we have a question that some recent studies have shown that there are viral peptides which are highly dissimilar to self and therefore are non-immunogenic. How would your negative selection model explain this?

Yeah, so that's exactly what motivated this study. I've actually looked at the same thing. People in the past have said the more similar you are to self, the less immunogenic you are, obvious, but that's really not the case. So if you try to show that in any data set, you will indeed find these examples. And the model indeed predicts that similarity to self is not that important. Like, once you're above a certain threshold, it doesn't matter whether you are very far away from self or just far enough away, right? That basically doesn't matter anymore. And so that's something indeed that this model also predicts.

Yeah. And have you thought about including in the system the glycosylation of peptides?

Which also differs greatly? I haven't, honestly. And it's actually a very interesting point. I'm going to think about whether it is something that could be done.

Thanks for that suggestion. And then we have a sort of forward-looking question, which was if you had thought about how artificial immune systems could be validated to know if they are representative.

Of a real human immune system? Yeah, that is a great question too. So yeah, some of the— for some of the models that we make, we do validations and experiments, like for example the thing I showed with this fundamental diagram. That was something that we predicted and then validated. And this other kind of more recognition type models, you would want to predict, for example, immune responses against peptides in some kind of experimental system. It might be, for example, a cross-system where you have less genetic heterogeneity. I think that would be the first goal. And I think that's, yeah, I even don't want to say if it's realistic or not, but I think that would be the first thing that could be tried.

Okay. Okay. And then we have a question on whether you think T cells could be trained to recognize tumor antigens or neoantigens, which may have many mutations that distinguish them.

From normal antigens? Yeah, may have many. So I've also studied new antigens a bit. Sometimes they simply have a mutation in one place, can be enough that makes them presentable, right? So maybe the original peptides was not presented by self, by MHC, but then it's just one mutation and this can then lead to binding. And apart from that, actually lots of tumor neoantigens are not so different from self, like they can have one or two mutations and suddenly be very immunogenic. And that's also, I think, something that we don't fully understand. And I think that's because we don't know entirely the rules of TCR epitopes binding yet. We have used something very abstract, and there's really lots of people who are trying to understand better how, like, which TCR recognizes exactly which peptides. And that's kind of still the roadblock that we would have to clear to make these kind of more quantitative predictions.

Yeah, okay, well, maybe we'll close with, with one more question. And as you As you might be aware, the Human Immune Project is in the process of selecting a pilot project with aggregated data from multiple parties and institutions to advance progress towards creating AI models of the human immune system. And I guess thinking from your perspective, do you have any suggestions of what the ideal project would be?

Yeah, we've had this discussion many times. I missed the meeting in La Jolla, but we had a meeting in the Netherlands afterwards. We also talked about this, and I think something that came up again and again was that we need more data on which T cells recognize which epitopes, because that's something that lots of people have already been trying to address with, yeah, not the— there have been no great breakthroughs yet, and I think one key reason is that we don't have yet enough data. So a larger-scale project like this might be very beneficial to synchronize the various efforts that are being made and maybe collect data that is large enough for machine learning to actually be used at scale.

Right, right. Well, big projects to think about. But thank you so very much, Dr. Texter, for kicking off our new lab meeting series with both a really interesting background on this area and then some of your really exciting research. And thank Thanks for taking all the questions from our audience. Thank you. Thank you also to all the attendees for participating in today's webinar and for submitting questions. We're so glad to have you join us, and we look forward to many more engaged discussions. I just wanted to let you know the Immunome Lab meeting will now be occurring on a monthly basis. And so please mark your calendars now for these upcoming webinars. Upcoming sessions. On Thursday, December 1st at 10:00 AM Eastern time, we will host Dr. Mark Davis of Stanford University. And on Thursday, January 12th, also at 10:00 AM Eastern time, our speaker will be Anant Madhubushi from Emory University. These webinars are now open, so there's absolutely no need to register in advance. Please just visit our website at humanimmunoproject.org to obtain the links. And while you're there, please sign up for the newly launched Immunome Report, which will feature voices of researchers working in human immunology and AI and provide all of the in-depth and interesting coverage you've come to expect from the COVID Report. Finally, please do visit our website and follow us on social media. We will post a recording of today's webinar there, and you can find all of the Global COVID Lab meeting webinar recordings there as well. Thank you again so much for participating today, and we look forward to seeing you next month at the Immunome Lab meeting.

Thank you so much.