Scientist Stories: Ben Glocker, Image-based Predictive Modeling

Good morning, everyone. Welcome to this joint Heidelberg AI and DKFZ Data Science Seminar event. It's my great pleasure to have Ben Glocher here. Ben Glocher is a reader at Imperial College, which is basically associate professor, associate professor, and actually one of the stars in the field of medical image computing, I would say, in our generation. He did his PhD at Technical University of Munich and also was a postdoc at Microsoft Research. At University of Cambridge. And I think if I'm not mistaken, I met you, Ben, in 2006 as PhD student at biomedical imaging. So at that time, our research interests were quite disjunct, but I always knew that Ben knew where the best social activities took place, and I saw him win best paper awards all the time. So now with deep learning as a very generic method, our research interests began to overlap actually, and I started reading Ben's papers with great interest, which is also one of the reasons that we invited him here today. So it's great to have you and the stage is yours.

Thank you very much. Thanks for the kind introduction. And also thanks for having us, having me, for inviting me to talk a little bit about some of our recent work. So there's quite a lot of materials. You will see how far I get, but I want to discuss a few attempts from our side to look at image-based predictive modeling. And in particular, I would like to talk about some recent work that we did on causality and generalization, and also something about uncertainty estimates. So just some general disclosures. So as I work with these companies, However, this shouldn't have any effect on what I say today. So for people who haven't been to Imperial College, so it's located at the heart of London, I would say very nicely surrounded by beautiful buildings like the Royal Albert Hall and the National History Museum. The Science Museum is here on the side. There was Mikhail in 2009, I believe, so maybe some people remember that. There's the Hyde Park, and this is Imperial College. So this is our Queen's Tower. Department of Computing is here, basically in one of the ugliest buildings in the whole area, but at least whenever you look outside, you see all nice buildings in South Kensington. So once this pandemic is over, I very much invite you to come over. So I'm a member of the BioMedia Group. It's, it's a large research group in the Department of Computing. It was started by Daniel Rückert many years ago. And it's now basically a group of people working at this intersection of medical imaging and machine learning. And as Lena nicely said, AI is now basically being used in so many disciplines that a lot of disciplines have come together because we are using the same methodology in the back. There's a little team, Team MIRA, that I call it. That I'm supervising within the BioMedia group. So I try to categorize a little bit what I'm currently working on. So these are kind of the research themes. So we do quite a lot of work on algorithms, so core machine learning and image analysis algorithms for doing something that we call semantic imaging. So trying to get semantic information out of raw image information. So we, for instance, worked on this convolutional neural network that we called DeepMedic, which was, I guess, one of the first that tried to do fully 3D analysis. And this was focused on image segmentation, so lesion segmentation in brains. A lot of the work is centered around image-based detection of disease, trying to detect, localize, recognize whether there are any signs, often just subtle signs of disease in scans. Scans, trying to work on algorithms that are more robust and reliable because eventually if you want to deploy these techniques in clinical practice, you need something that not necessarily works all the time, but where you have some measures of knowing whether something works or not. So we are trying to look, for instance, at methods that can tell you how reliable a prediction is. And how trustworthy a prediction is in, obviously, in the absence of ground truth once you deploy the method. And then there are quite a few people now these days who are more on the machine learning methodology side. So we have some people more looking into theory and trying to come up with new ways at the core of machine learning problems. I invite you to check out, so for the DeepMatic, there's a demo online. So you can try out running some 3D CNNs on some brain images and different types of data. There's a live demo that you can try. So this talk is divided into two parts. So the first about causality and generalization. So this is quite recent work and thoughts that we have put into trying to understand a little bit better what's going on in our world when we talk about predictive modeling. And I think it maybe helps to just like, so everyone is on the same page to quickly talk about the general setup that we are considering here. So when we say predictive modeling in image analysis, typically what we mean is that we try to train a model that takes images as inputs and tries to predict some annotation, y here. So this could be a disease label, it could be a segmentation, it could be a bounding box around the structure of interest. But basically what you're interested from a probabilistic point of view, you try to learn this conditional distribution. Given an image, what is the probability or likelihood for a certain label. So there are some big assumptions that we make when we say we train such a model. So first of all, we assume that there's sufficient training data available, so pairs of X and Ys in the case of supervised learning. And the other big assumption that is often challenged in practice is that we assume that training and test data come from the same distribution. Because only then this model that is trained on some training data actually makes sense in the test domain. And this is often violated, actually. We know that if you deploy a model, performance often drops on new, really unseen data where maybe there's some characteristic or some variation that is different from your training data. So I guess a year ago, or maybe a little bit longer, we started to think about this predictive modeling from a causal perspective. So we got interested in causality and trying to understand how this causal perspective actually relates to things we do in machine learning for medical imaging. So we wanted to understand a little bit better what's actually the relationship between the images and the annotations. Again, in the case of, let's say, we do supervised learning. So What we then looked at, and there's some work on this in the context of machine learning already where people discuss the relationship between your inputs and outputs, which direction of causality is there. And we argue that there's basically two cases. So you could have a causal relationship. In that case, we are predicting the effect from the cause. Remember, this is the joint, and this is the conditional distribution that we are trying to to learn during training. And then there could be the anti-causal relationship where actually the annotation or the label is the cause of the image, and this is what we call anti-causal. So here we are predicting the cause from the effect. Now the interesting thing is the conditional distribution is still the same, right? So we are still interested in, given an image, what is the probability of a certain output label. Now the question is, does it matter? Is it actually important to know what the causal relationship between your images and annotations are? So let's first have a look at some examples, and I'm trying to convince you that first of all, these two settings exist in the real world. So if we look at skin lesion classification, for example, we could say there are dermoscopic images as an input, and what we are trying to do is try to predict, for instance, cancer or no cancer. Or benign and malignant appearance. And let's say our annotations come from biopsy. So we have biopsy-derived diagnostic labels, which is probably the closest you can get to ground truth, to the real like disease status. Now we would argue that this is a case of anti-causal relationships. So the disease status, whether you have cancer or not, or whether there is a malignant mole or benign one, causes the appearance of the image, right? So the images, how they look like, depends on what the disease status is. So the causal diagram would look like this: disease causes image. But the prediction goes in the other direction, right? So we are trying to predict in the reverse. We are given an image, want to predict whether it's a benign or malignant case. So here's another example, brain tumor segmentation, another popular application for medical imaging. Here our inputs could be, for instance, a structural brain MRI and we want to predict some manually drawn contour. We want to replicate basically a manual process of drawing contours around structures of interest. So we would argue that this is a causal relationship. Here the image causes the label, which is the annotation, because if you think about the data generating process, someone sits down with the image and draws contours around them. So if you change the image, the contours would change, which is a good indication that This is the relationship in terms of causality that we should consider. So there is still, of course, the disease, the underlying disease that causes how the image looks like. So this is still the same as before. However, the thing that we try to predict is this segmentation, right? So this is caused by the image. So in this case, the prediction, the direction of predictive modeling goes in the same direction as the causal relationship. Now here's another case example, radiology reports. So consider the case where we look at chest X-rays, for instance, and we have maybe some diagnosis that is extracted from a report. Now what is the causal relationship in this case? And we argue that this is much more complicated to establish because maybe we are missing some important information. It's actually not entirely clear where this diagnosis is coming from. Let's say you use some natural language processing to extract the diagnosis from the report. How do you know how this diagnosis was derived at? Was it based on the image? Did someone look at the image and make a diagnosis? Or maybe there were some other lab tests involved, maybe some blood tests or something even more important than the image itself. And if you don't know this, then you cannot really establish the causal relationship. So it's really essential that you have all the information available about how the data was collected, how the data was annotated, and so on. And this meta-information is pretty much the same that Lena and colleagues some time ago basically argued is really important to have also in the context of biomedical image analysis competitions. So it's the same type of information that is essential for establishing the causal relationship that you would also want to have to make better arguments about whether a data set is useful to establish, for instance, the performance of certain algorithms. You want to know something about the underlying biases in the data, for instance. So far I showed you that we believe there are cases, real-world examples where we can talk about causal and anticausal relationships. Now the question is, does it really matter? So we argue that there are a lot of challenges that we are trying to overcome and many of you many others in the MICAI community are trying to work on these challenges. First of all, there's data scarcity, which basically relates to the problem of having only very limited annotated data available for training our models. That's always a problem in medical imaging. And then there's dataset shift and selection bias, which I will come to later. But let's look at data scarcity. So what are we trying to do in data scarcity is we ideally would like to use all the data that is available. And by all the data being available, we mean a lot of unlabeled data, which is often the data that's more easier to get. And labeled data is expensive, tedious, and difficult to get. So can we leverage unlabeled data? So from a machine learning perspective, you could use techniques like semi-supervised learning, which have been used quite a lot on various prediction tasks. Now, if we look at semi-supervised learning, So basically from a causality perspective, it's quite interesting because the label data is basically these pairs of X and Ys. The unlabeled data is only giving you information about the Xs, right? So you have unlabeled images. So you get more information about the distribution over the images. Now remember that we are talking about two causal relationships that we can have in our predictive model, either a causal or an anti-causal one. Now, interestingly, if you look at the conditional distribution that we are trying to learn, so given an image predicting the probability of a label, and let's assume we are in the causal setting, for instance, image segmentation, right, where we predict from image and a contour around the structure of interest. Now, in this case, this unlabeled data is telling us something about the distribution of the cause, And what we are trying to predict is the mechanism going from X to Y. And there is this principle of independence of cause and mechanism, right? So this is a well-known fundamental principle in causality that basically any information you get about the cause doesn't help you, is independent of this mechanism. So which basically means if you think this further, that semi-supervised learning should actually not work in applications like image segmentation. Now, this was discussed not in segmentation but for other settings by Bernhard Schulkopf and others quite extensively, and they made very good arguments and also empirically provided empirical evidence that semi-supervised learning shouldn't work in the causal setting. Now, what's interesting is in our domain is that there is actually quite some work out there on semi-supervised learning for image segmentation. And I think it would be interesting to look at this a little bit in more detail. Why and when does it work? And maybe there are some theoretical obstacles or challenges that actually prevent us to make real good progress on image segmentation using this setup of semi-supervised learning. Just from the causality perspective, it shouldn't work, but I think it's an interesting one to discuss. So the causal perspective also allows you to look at other techniques. For instance, data augmentation. Data augmentation is different from semi-supervised learning. What you're trying to do there is you basically try to generate additional pairs via realistic perturbations, right? So you take your true data, x and y pairs, and then you're trying to basically synthesize more pairs that you can use during training. And from a causal perspective, this should work quite well because generating these additional pairs gives you more information about the joint distribution, and that could be useful when you try to learn the conditional distribution of predicting y given x. So realistic perturbations in computer vision are often like these simple spatial transformations, rotation, scaling, and so on. You can do intensity variations and so on. People now also started to look into learning augmentations. So you use now the unlabeled data to get an idea of the variation that's out there in the real world on your images And maybe you try to learn what type of perturbations you should do on your images. So this could be an interesting one, but it's quite different from the setting of semi-supervised learning. Okay, so this is the part on data scarcity. Causality also allows you to analyze more specifically the problem of dataset shift. And I personally believe that is the challenge in machine learning in medical imaging, also in other areas, but in medical imaging in particular. Dataset shift basically means something has happened between your training distribution and your test distribution, and a shift has happened in a way that you cannot really rely on your trained predictive model anymore. You need to do something about it. And there could be different types and reasons for shifts. And the interesting thing is that causality allows you to be more specific about the types of shifts that you might actually see or encounter in the real world. So we try to look at this by basically introducing a domain indicator. So this is another variable in our causal diagram, and that is some sort of helper variable that says this is the unobserved true anatomy of your patient that you will never really see, but that is what you kind of— it's the physical manifestation of the thing that you image, right? So your patient goes into the scanner, lies down, and then you get an image, and the image is obviously just a measurement of that unobserved true anatomy. Now, if you use causal diagrams and you talk about these different settings of causal and anticausal relationships between X and Y, so this is still the image and this is still the label that we want to predict. We can use causal diagrams to more accurately describe different types of shift and also communicate what types of shifts we might expect in the real world. So causal diagrams are a great tool for communication of your assumptions. So for instance, we could clearly indicate what we mean by population shift. Population shift means that this true unobserved anatomy changes between domains, right? So whether you're on the train or test domain, whether you're in hospital A or hospital B, or whether you use a scanner from one manufacturer or— sorry, the scanner comes at the acquisition shift. But the domain indicator just tells you that your patient population might have changed between domains, right? And this is affecting the P of Z. Now, if you are interested in predicting going from X to Y, so we're in the causal setting. So let's say you do image segmentation. What's interesting is that actually this population shift shouldn't affect you too much, right? As long as your training data has some sort of, support or captures the range of variation or population that you want to be able to predict on, whether your population shifts between domains shouldn't matter too much. Acquisition shift is the one that I just mentioned with the change of manufacturers, for instance, between image devices. And this happens quite a lot in practice. So if you go to one hospital and they use a certain type of scanner or a certain type of generation or a certain type of software for image reconstruction, that has an effect on the images that you get, on the image characteristics. So the domain indicator here is pointing to the image. And if you go to another hospital or another scanner, the image characteristics might change. And then we're talking about acquisition shift, which is very different from population shift. You can have annotation shift. So this would mean that actually depending where you are, people, for instance, use different ways of annotating the data. That is a quite difficult one for machine learning. There's not really a good way of dealing with that. It also indicates, and maybe that goes back to the work by Lena and colleagues looking at data collection issues. If you have different annotation policies between sites, let's say you do a multi-site study and there's some problem with, let's say, the standard operating procedure collecting the annotations, that could really screw up your machine learning. In the, in the anticausal setting, we have things like prevalence shift, right? So the prevalence of having a disease label, the prevalence might change. So this could be, for instance, the case where you train on balanced dataset and then you test on a highly imbalanced population. Again, You have acquisition shift if the domain indicator points to the image, or you can have manifestation shift, which again would be something that from a machine learning perspective is really difficult to deal with. It would mean that a disease manifests itself differently in the anatomy depending where you are. So again, that's something that a predictive model wouldn't be able to deal with. So I guess the take-home message is here that there are all these different types of shifts, and causal diagrams allow you to more concretely say what type of shift has what effect, and it also allows you to then go to a list of mitigation strategies and see what could you do about it. There are different strategies for dealing with these different types of shifts, and that's important to realize when you do a study. Let's say you do a multi-site, maybe even an international study collecting data from different continents. You really want to understand and analyze what type of dataset shifts you might expect. Because that would heavily influence what type of machine learning methods or mitigation strategy you can employ. So just going back to the brain tumor segmentation example, so we had initially this causal diagram for saying what we are trying to predict, right? So we have disease status causing image, image causing segmentation. Now in this diagram, you can, for instance, directly show what type of shifts you might expect. Let's say we do expect some population shift, then we could have a domain indicator saying here that depending whether we are at train or test time might have an effect on the population. Or if there's a different scanner being used, there might be an acquisition shift or both. So you can communicate that again in your causal diagram. Now I quickly want to talk about some mitigation strategies. What do you do if you have acquisition shift? So this is, as I mentioned, I think one of the biggest ones that we have. There is now, for instance, this year, quite interestingly, some challenges that specifically look at multi-site, multi-scanner problems, which I think is still a big issue. We do see segmentation methods fail. If you're trained on a Siemens scanner, test on a GE scanner, you do get these differences. We tried a while ago something called adversarial domain adaptation, where basically the idea is if you have a convolutional neural network that does image segmentation, right? So this is a DeepMedic here, but it could be a U-Net, could be something that tries to learn features mapping to segmentation labels. So dense prediction here. You could add a domain discriminator at training time that looks at the feature maps, and the domain discriminator is basically the task is to tell where the data is coming from. So if at training time you have data from different sites, but you only need one of them being annotated, right? So let's say you have one hospital where you've got all your manual segmentations, you can train your task network and the other data. So data coming, unlabeled data coming from a different site could be used for the domain discriminator. And the domain discriminator of course should eventually not be able to tell where the data is coming from by looking at the feature maps, because then you would train a task network that is agnostic to the data source. So that's the idea of doing adversarial domain adaptation. It works reasonably well. You don't need task labels from your test domain. However, you do need it at training time, and that could be a limitation, right? So you need already the test domain data during training of your predicted model. But it works well. So we tested this on brain lesion segmentation and MRI, and here the issue was that in the first case, we had data coming from an earlier study where they used gradient echo scan to detect microbleeds. And then in the imaging protocol later on in the study, they switched to something called SWI, which is a different type of MR contrast, which also shows microbleeds, but is a very different protocol. So from a machine learning perspective, the two input signals are very different. And if you just ignore this difference, your network is basically failing completely. If you then just say, okay, so maybe there's a change. So maybe we can leave out this sequence and just rely on the other sequences that are in common between the two studies. You can do that and you get better results, but you're missing out the microbleeds. So you're not able to detect these important microbleeds anymore because you removed the modality that shows them. What we could show with this adversarial domain adaptation is that you can actually train on these input sequences and then switch from GE to SWI and still be able to detect these microbleeds. So this is the detection, this is the manual ground truth. So such an approach works, but as I said, the problem is you need the data already at training time. So I'm skipping those numbers, but so what do we do if we don't have the test domain data already at training time, which is the more realistic setting, right? So eventually you want to train a model, you want to go out to a new hospital and deploy it there and run it on data that you have never seen. How can we generalize entirely to new domains? So Qi Shi was a postdoc with us, is now an assistant professor in Hong Kong, has worked on something called domain generalization. So this goes one step further than domain adaptation. In domain generalization, we basically try to simulate domain shift during training, but we don't need the test domain already. But what we do need is at least two or more domains available at training. They don't have to be the ones where we later test on, but they should be two different datasets from which we can simulate the domain shift. So this was tested on a non-medical imaging dataset, and so there's a benchmark for domain generalization called the PAX benchmark. It's a quite interesting one because the domain shift is quite severe. So you're moving from photos to art to cartoon to sketch, but then have the same categories in each of them. So there are these 7 different object categories and you see the differences are quite severe. So these drawings compared to like the cartoon variations or art painting and real photos, there's quite severe domain shift between them. However, there is also some commonality between them. So you might say, see some features or characteristics within a class that still make it recognizable, right? So how do you do this? So we combined several things here in one method. So the first thing is called episodic training. So that's the training procedure that simulates domain shift. And then we introduced some new loss functions. Which has two parts and I will quickly talk about them. But the idea is basically that you first, you look at your predictive model. Remember, we are talking about CNNs here. So you can think of a model doing some feature extraction first. So you get some latent representation, this z, right? So input images go to a z feature encoding, and then you have a task network that sits on top of that, that takes this representation and predicts, for instance, the probability of a class label. So if we separate the network in these two parts, what we then can do is this so-called episodic training, which was introduced here in this paper, where you basically take your training data that you have from two domains and you split them into what you would call a meta-train and meta-test dataset. So these would be from two different domains. Let's say you have photos and cartoons, and these would be the two domains you have during training time. And the idea of episodic training is now actually that you evaluate your task model, and when you update your network parameters, you do it based on different information from the different domains. So you do a forward pass on your network that is currently trained on, let's say, one of the domains, and then you evaluate your loss functions on the other domain. So that's where you basically get this idea of simulating domain shift. You want to know how good the current update is on this domain shift domain, right, on a domain that is very different from what you currently consider to be the training domain. So you update your network basically by alternating between switching between what the meta-trained and the meta-test domain, and that's how you basically have have assimilated domain shift during training already. The loss that we use, so we don't have to go through all the equations, but basically what it does, the first one is called the global class alignment. So the idea, the intuition is that if you take your image and it goes through your feature extractor network, that you then look at what we call the semantic feature space, basically the latent representation of each of those images. What you're saying is that whether you're in domain I or in domain J, let's say photos or cartoons, the relationship between classes should be preserved within the domains. So basically it says, if you look at this, so after these feature representations goes through a task network, you can come up with these affinity matrices that tell you how, or confusion matrices, how much you are confusing one class with another. Basically says, let's say you classify houses and cats and dogs. You say cats and dogs are confused in a certain way with a certain probability, independent of whether you're looking at photos, cartoons, and sketches. So this confusion between classes should be preserved independent of the domain. And that's what you're trying to do is you're basically minimizing the difference between these affinity matrices computed on different domains. That's the one part of the loss. The second part of the loss is a local sample clustering where we now basically say we want to have domain-invariant invariant features. So here we are looking at the features from different domains. So each of those symbols here mean we have data from a different domain, and the colors is basically again the class. We are now saying that cats across domains should fall into a similar region of the feature space. Dogs and houses and guitars and whatever categories there are, they should form each class should form a cluster in the feature space. And that you can do by either, for instance, using a contrastive loss, something that's done recently quite a lot using contrastive losses. You can also achieve this by a triplet loss where you basically say if two data points from two different domains but the same class, they should come together, they should have a small distance. And if they are from different classes, they will be pushed away. So that's the idea of contrastive learning. So this was the setup. So we had episodic training, global class alignment, and this local clustering assumption. And these are the results on the PACS dataset. And basically, I mean, these are just numbers which probably don't mean much, but what you do see is that this new approach was performing better than previous approaches. That's what you usually want for a paper. But also interestingly, we trained this baseline, which was basically just a big deep network that took naively all the available training domains together and just trained a deep network on them, ignoring that they are different domains. And interestingly, the performance of that was already better than some of the proposed domain generalization methods. So I think one take-home here was also to be careful about your baseline that you compare with. So we did an ablation study because we introduced 3 parts. Obviously, it's not easy to see which one has the biggest effect, and we did an ablation study and found that actually you do need all 3. You do need the episodic training and the different loss parts to get the final overall best performance, which I think is what you want to see. Now, interestingly, going back to medical imaging, it turns out that if you do image segmentation on MRI brain, which was our initial motivating example, this technique doesn't really like help in a sense that it didn't make it worse. It actually like improved a little bit on top over our baseline, but not too much. And again, the deep baseline is basically you have 4 domains, 4 different scanners, different protocols for getting these T1 images, and then you do image segmentation. And if you have 3 of them available for training, you just train this network, generalize actually quite well. The reason we believe is that our technique here doesn't, like, give a big improvement is that the domain shift is relatively subtle. So these images are much more closer together in feature space than, for instance, what we saw earlier, photos, cartoons, and sketches. So it seems that this technique that we propose for domain generalization is good for severe domain shift, but it doesn't matter too much if your domain shift is subtle. So going to the last part of the causality perspective, which is about selection. And I think this is again a very critical one and discussed more and more, I think, actively, which relates to any biases that we might bake into our machine learning models. And this goes back to sample selection problems. And again, I'm just trying to make the case that I mean, many people look at this, many people talk about sample selection biases, but causal diagrams again, or the causal perspective, allows you to more formally communicate and describe what type of biases we might run into. So if we again look at our causal diagram, in this case we have images and labels. We don't care about the causal relationship for this diagram, so there are no arrows here, but we use this selection variable S to indicate how we basically made a selection on a data sample, whether it's accepted or rejected in our database. If we do completely random, that's ideal, right? That's a randomized controlled trial where you don't look at the data at all and you just take random samples. That's great, no biases involved. But that's typically not the case. Very often you either do it image dependent, You look at the images, maybe you do some quality control, you filter out some poor quality scans, some with artifacts and so on. So you do image-dependent selection. You often also might do target-dependent selection. So you look at the disease labels or the annotations and you decide based on the annotation whether you include something or not, or you might use both. And then you bake in different biases and you can formulate that, formalize that. And for instance, if you do selection based on the images, you might end up with a population shift or an acquisition shift. If you do it on the labels, you might end up with prevalence shift. So you might bake in all types of shifts and that could lead to really bad predictive models because you might pick up spurious associations between the data and then you have again problems with generalization. So again, just making the case in this example of skin lesion classification, if we communicate in our, let's say paper, report, whatever, what we built is a predictive model of going from image to the disease label. That's what we're trying to predict. We could already indicate what type of biases there might be. So this is an interesting one, I think, because if you think about our earlier assumption that the— our earlier statement that the labels might come from biopsy, right? So this is great because it gets close to ground truth. But when do you biopsy? You probably don't do this randomly, right? And you don't often do this on people who are not suspicious in the first place. So you could indicate this in your diagram that the image had maybe, was causing some suspicion, right? So something did not look quite normal. And that's, these are the cases where you do biopsy. So now if you think about, you download a dataset that someone said, put somewhere up and said, this is a public database for skin lesion classification and their biopsy-derived diagnostic labels. If you train a model on that, you're probably heavily biased because this is a dataset where already images were quite suspicious, otherwise you wouldn't have biopsy. So this model will probably not work in a screening setting where you will see lots of normal cases. Again, this goes back to the point you need that information, right? So people need to tell you how the data was collected, generated, and so on. There's some more work on this which I'm not going to cover. But we looked into what can you do if you have sample selection bias, what can you do if you have class imbalance, and so on. But basically, I tried to make the point that causality matters. I think considering it in our daily work, even from start of projects, right? So when it comes to collecting the data, annotating the data, taking this into account is important. There's a lot of talk about guidelines and regulation. How do we properly report results on AI in healthcare in general. And there's a lot of call for transparent reporting, and we believe that causality could add to that, right? So causal diagrams are a great way to communicate the assumptions you make. So we came up with this scaffold diagram that might be a useful first guide if you wanted to build this yourself for your own project, your own problems, you might want to start with this stuff. There's a little bit more in this report which is currently in press and should come out sometime soon. So this was work done by Daniel and Ian. Okay, so what's next? I don't know exactly how much time I do have. If someone wants to interrupt and either tell me how much time I should spend.

Yeah, so we have 1 hour in total, including questions. So I think we should leave at least 10 minutes for questions.

Okay, so I have another 8 minutes. So good. So let me continue with this. So we wanted to— so this was causality in basically mostly looking at the challenges and communicating and making assumptions transparent, but we didn't really do much, right? So we didn't improve our predictive model. We tried in the domain adaptation case with some algorithms, But the next big thing, and I guess people like Bernhard Schulkopf and so on have inspired these thoughts with their great work around causality and machine learning, is can we actually now use these models? Can we use the causality perspective to build better predictive models? And this goes to this idea of causal representation learning. So I want to show you a very recent— so this is just like in the last couple of weeks came this together, some results that we got shot by trying to put in causality into deep learning. So these are two subjects, subject A and B, and they are both 68 years old. And then we look at a scan two years later. So I hope you can see this if I flick forth and back. So the scan two years later, if you focus on the left, you see there's quite some change in the ventricles. So you do see that there's some anatomical changes here. On the right, you don't see much. Now the question is, what is different? Why did these two subjects have a change to their anatomy quite differently within just two years? So what is the difference between them? So it's not easy to answer that question, but we believe that causality could help by bringing in some understanding or some domain expertise about the data generating process. So we tried this with a very, very simple, probably overly simplistic causal model for brain imaging here, but just to make a case. So this is the simple causal model that we assume. So there are two exogenous variables here, age and sex of a subject. And these have the following effect on the image that we observe. So there's intermediate variables of brain volume and ventricle volume. And this is like our assumption how this fits together. So it's based on, it should be okay. As a simplified model of what you would observe in an MRI brain scan. So we worked on this idea of putting in these causal models into deep learning. So here the idea is that each of those variables is something that you can observe for your subjects. And then you have these mechanisms that basically relate your causes and your effects. And mechanisms are basically assignments. These are functions. That we can learn from observed data. So if we have a database that has all these variables observed, we can learn these functions, basically the arrows here in the causal model. And that's what the structural causal model is about. So it's a set of assignments, these functions that define the value of each of those variables. And they are basically always defined by looking at the noise variable and the parent values, right? So that's how a structural causal diagram is defined. And over the structural diagram, you get a joint distribution over all the noise variables, which factorizes into these— sorry, this is the joint distribution of the joint variable. This is the joint distribution over the noise variables. But we then get this unique joint observational distribution over all the variables. And this factorizes into these individual mechanisms, right? So these are the independent mechanisms in the structural causal model. And in this very recent work, we basically represented these structural assignments with deep learning models. So we used normalizing flows, and for the imaging generation, we used the variational autoencoder. But what we could basically show is that learning from data, so this is the true data distribution. So I'm showing 3 variables here at the same time. So brain volume, age of a subject, and the color here is indicating the sex of the subject. So this is the true data distribution in our database of 15,000 brain scans we use here. It's a population study. And after training our model, these are the distributions that we can get out from our trained model, which is quite faithfully representing the true data distribution. So now if we have learned this, what is interesting is we can talk about generating counterfactuals. So this is the kind of the holy grail of using structural causal models. You can now generate points which are basically answering the question of what if. And the way it works is that you have 3 steps. You have this abduction step. So you first predict the state of the world. So you have your observations and you learn all the noise variables and the mechanisms, the functions, the structural assignments. If you do this, then you have a fully specified structural causal model. Now you can do an intervention. So you can change variables on purpose. So you can do this intervention. You can set, for instance, the age, you can set the ventricle volume to a certain value. So this gives you a new structural causal model. And then you can do prediction with this new model. So you can basically generate, in our case, we can generate an image that reflects the intervention that we did. So this is what we tried. So we looked at brain scans here. So this is just mid-axial slices of aligned scans, but then if we get a new scan, so this is an incoming scan. So here we have the values for the different variants. Attributes, we can now do an intervention and use basically see what happens to the image. And we show basically here the grayscale image that comes out when you do the prediction step of the counterfactual. And here's the difference between the original input and the changed image. And what you do see is actually you do get quite some faithful alterations to the image. For instance, if we do, change on the sex, we get an overall slight increase of the overall brain volume. That's just what the model describes. So sex has an impact on the overall brain volume, but it's very subtle. If you change the age— so this was a 49-year-old— if we change it to 40, not much is happening, but you do see like a slight change of the brain. If you, if you put it to a much older age, like 80 here, you see that the brain overall shrinks. So that we know also that age has an effect on the overall brain volume. However, if you do specifically only brain volume, right? So you're not changing the age, you're just changing the brain volume. You see that the brain volume changes and the ventricles change. So everything becomes smaller. But in the case of age, actually the overall brain volume gets smaller, but the ventricles get larger. So that is what we know from age aging processes in the brain. And if you directly intervene, for instance, on the ventricle volume, you get more of these localized changes. So it's interesting to see that these interventions that we can now do on images actually do represent something that we believe is plausible in terms of what we know about the aging process. There's a live demo, so you can check this out and do it yourself. There's a Google Colab notebook where you can go and actually play with these brain scans and change some sliders, change the age, change the brain volume and see how it affects. Good. So that was the first part. I'm afraid I have to skip the second part. I think I was a little bit too excited about the causality stuff, but at least it gave you hopefully a good impression on that we really believe that causality is something to consider in future. So we're organizing this tutorial at MICAI. So if you're interested and want to learn more about this, there will be like a gentle introduction to causality. We will talk more about how we believe causality could bring in some new fresh ideas into medical image computing. The counterfactuals that I just showed is like a very early first attempt to do something interesting in that space. But there's of course a lot of work on how we can counter biases, how can we deal with confounding, All very, very important topics, I think, in the context of machine learning for medical imaging. So I'm afraid I won't be able to talk about the uncertainty part, but if people are interested and if there are people who work also on uncertainty, I would be very happy to catch up on this or have another session to talk more about this. So I'm finishing here, and I'm looking very much forward to any questions you might have. Thank you.

Thanks a lot, Ben. I think we should definitely follow up on it. We just invited you for the part 2 in fall if you're interested. Yeah, so now I hand over to Jens for the question moderation.

Yes, thank you very much. I had some audio problems. Is everything okay now? Can you hear me?

Yeah, I can hear you well.

Yeah, I had to switch devices. All right, thank you very much for the great talk. I especially love the causal perspective on the dataset shifts. I haven't really thought about that in that way. So before I start with the questions, I would like to invite everyone who's listening to post their questions to the chat. The link is right below. The video stream, and then I will forward your questions to Ben. Okay, so maybe before we start with the audience, Lena, do you have any questions you'd like to ask?

Yes, actually, yeah, it's a bit of a fuzzy question, but I also like the perspective that Jens just mentioned, and I typically fight a lot about how to validate machine learning algorithms because the typical the typical answer that I get is, well, the test data has to be the same distribution as the training data. And I find this a little bit annoying because this is not something that you have to practice necessarily. I mean, of course I understand the argument for it because I mean, you train for a specific case, but in practice, if you really think about clinical translation, basically your test distribution is unknown, right? So like I said, fuzzy question, but what is your view on it? Or how do you think we should validate methods in general?

No, I don't think it's a fuzzy question. I think it's the key question. I mean, this is really, I mean, when you want to deploy these models, what do you do? I mean, you can validate in-house. To be honest, I think most of the validation that is done, I would call it part of the development process because the test data is often just a random subset of the data you have anyhow. I mean, now there are— there is already some calls and a lot of clinical journals are asking for external validation and so on, which is good. It's a good— it's a move in the right direction to ask for independent test data. But again, that means this data needs to be annotated and so on, and you stop then there, right? So I think what is absolutely key is that we find better ways of doing post-deployment analysis, post-deployment validation, and that will be different, right? It will not be like you have fully annotated, you have the same labels that you had available at training time. It will be a different way of validating. You will need to monitor, I think, live. So you need to have systems operating and you need to constantly monitor in real time what they are doing. You need ways of having post-deployment quality control. So we need to find ways of having checks after deployment that could, for instance, predict when the machine goes completely wrong or when something is trustworthy or not trustworthy. It's a different type of validation. It's not the same metrics we will use. There's no dice score to compute after deployment. But there needs to be other metrics, other measures of seeing what's going on. You can think about, you can monitor, right? So you could collect all the statistics over your predictions over time. Now that doesn't help you with individual patients, but it might show you trends. Let's say the brain volume that you predict degrades, goes down over time. That's very unlikely to happen in your population. So maybe your model is miscalibrated or does something weird. And then you need techniques for individual, like patient per patient prediction of whether is this prediction useful or not. I think there's a lot of work needed for that. Thanks.

Maybe we can catch up on this at some point in the context of the challenge initiative.

Yeah.

All right. One question from the audience is, you mentioned that in a causal setting, semi-supervised learning should work.: Are there any theoretical results or guarantees in the anti-causal settings?

No, but so I'm not aware of theoretical guarantees on that. It's if you look at the, so let me see, I don't know, I assume you can still see my slides. Let me just go back to that one slide. Yes, so the problem is, Let me see if I can get this without, yes. So the issue generally is when you talk about these causal settings, you make a lot of assumptions and it's in the first place difficult to show that your assumptions hold because I mean, you make them. And that's, I guess, the first problem with any theoretical guarantees. If you do have an anti-causal setting, however, so we are looking at this case, right, where you predict from— where your causal relationship is going from y to x. What it means is that your unlabeled data is basically a distribution over the effect. And if you look at the distribution that you're trying to learn, so this conditional distribution where you are given x, you want to predict y, in that case, the— it might actually help. There might be information in the distribution over the effect that might help you. Or that might be informative for the semi-supervised learning. I'm not aware of theoretical works on that. If at all, I would definitely check out Bernhard Schulkopf and colleagues' work on comparing causal and anticausal settings. Not just this paper here at the bottom, but they have many, many more works on that space and looking at causal and anticausal relationships in machine learning.

[Speaker] Maybe I can follow up on this. It is often the case, at least I think so, that an image is not unique in the sense that multiple underlying semantics, let's say, have the same expression in the image space. Could that be a reason why this could still work even though it shouldn't?

Yeah, it's a good question. So I guess we would need to be more careful in looking at when did this really work. And again, what does it mean to work? So often you compare to not using semi-supervised learning, right? So I'm sure there are papers and we have done similar works where you look at the fully supervised setting. Let's say you have 100 training images and now you say, what happens if I use another 500 unlabeled datasets and I use some loss function that says I want to be somewhat confident on the unlabeled dataset? So there are these losses that people use. There are these cluster assumptions, right? So this diagram here in the top Basically, it's this typical illustration of you have some labeled cases and your unlabeled cases maybe give you an idea about the underlying space, how samples are distributed, where should the decision boundary be. Should it be going here from, from top to bottom? Then you would say no because there are unlabeled cases. You want to go through the low-density areas. So depending on what type of things you actually use in your semi-supervised learning. I think we would need to look carefully at specific settings where people claim that semi-supervised learning works in image segmentation. There might be also some other things going with like selection biases. What type of data is being used? Is it really randomly IID sampled from the unlabeled distribution, or is it maybe informed by something. So maybe there's some spurious or some signal in there that we have just like not really carefully considered. I'm just saying that, and it's not actually me saying it, it's Shukhov and I saying that if you're in the causal setting, semi-supervised learning shouldn't work. They have generalized this quite well. So they also looked at differences between so-called effect features and causal features. And there's a nice paper on generalizing semi-supervised learning to this setting, but I think that's all I can say at this stage.

All right, another question or suggestion idea that also relates to this. Someone's asking, could this improvement from using unlabeled data stem from regularizing intermediate representations with respect to the broader input data distribution.

Yeah, I think so. Yes, I think it's a good point. So the unlabeled data might have regularizing effects. And again, because if we use these deep networks with like lots of capacity, lots of linearity, it's sometimes not entirely clear which aspect has what effect on your training. And the regularization might be a fair point. And maybe that's already helping. But I think it's still important to realize then you might benefit from it in a different way than you thought. So I find it quite interesting to think about the assumptions we often make. I mean, it's going back to this idea of when people describe how neural networks work, let's say for digit recognition, right? And on MNIST, and then you say you have layer 1, 2, 3, and 4, go on, you learn more abstract features, and one of them is learning the edges, the next one is putting the edges together into more shape type of things, and then you put everything together and so on. And then you look at the feature maps sometimes, and that's not at all what's happening. So I mean, I think it's interesting to talk about the, like, our intuitions and what's actually happening. And I think sometimes it's important to realize if the two don't match, then we might be in trouble if we had used our intuition and assumptions, for instance, to deploy a model, right, and say, I assume this is how it works, so it will work in this setting, but then you realize it doesn't.

Okay, another question pertains to learning data augmentations from unlabeled data, and the question is, how do you learn P(X, Y), so the joint distribution of the data and the labels without any label information.

Is this going about the data augmentation case? So I think so, yes. Yeah, so, so we're not, we're not learning that joint distribution. We are basically, by generating new pairs X and Y, we are informing the joint distribution. So they, so those samples that you generate are typically done on quite constrained perturbations, right? And that's also where some sort of domain knowledge comes in. I mean, why are we rotating images a little bit, right? So why would we take that cat and rotate it by 40 degrees and say that is a realistic perturbation? It's because we have some domain knowledge, right? So we say this is still a cat, after I rotated it 45 degrees and I want my predictive model to be agnostic or invariant to these type of perturbations. So you say that rotating an image by 45 degrees is still part of the joint distribution. You're not learning that joint, it's purely based on your assumption. That's why I think these works are both promising but also dangerous in a way that if you now try to learn augmentation from unlabeled data, How do you make sure that this learning actually doesn't fail or doesn't bake in another set of biases or something, right? But I think it's an interesting idea to look at unlabeled data and of images, how they appear, right? So you could just look at Xs without the Ys and just see what is the variation on these images that I see. For instance, intensity distributions. You could look at the statistics of the histogram, spatial relationships, whether things can be rotated, and then you take this and apply it to real XY cases where you have the pants. So I guess that was probably what the question was about, I hope.

I hope so too. If I'm relating anything wrong, please feel free to correct me in the chat. Another question says, great talk, Obviously, I like the causal model that informed the image generation. Did you try to compare to alternative conditional image generation approaches that do not use causal modeling? What exactly is the benefit of the underlying causal model?

Yeah, so what happens is, so in this paper we do this, I mean, so here we're using a very simplistic causal model for this type of data generation. In the paper, we do this with MNIST as well, where we control exactly what the underlying underlying data generating process is. So I mean, it's kind of knowing, it's assuming that you know what the data generating process is, which is what causality is all about, that you put in these assumptions. If you do just like standard conditional modeling here, you simply will get implausible variations when you produce the data. Not always, I mean, sometimes your conditional model can just align well with like the causal model and you do get realistic things. I mean, people have probably seen works like Faces where you change the hair color of someone or these type of works kind of assume that there's some sort of disentanglement of your variation and you can say there's a factor for changing hair color. But because you're not factorizing according to the data generating process, you have very little control that this works. You might change the hair color, but actually you might change other things at the same time because it's not following the data generating process. So yeah, you can show this empirically. But yeah, I mean, obviously it does need you to think about what the true process is, and sometimes you don't know.

Okay, one person mentions that the group just published a paper at UAI 2020 or got accepted. I don't know what the timeline is there and just recommends this for people who want to go deeper on the topic. And maybe we can close with A little bit of a broader question. Someone asks, in your opinion, is causality the way to go to make AI explainable?

I don't know about the explainability part. For me, it's still a very vague— and I'm not an expert in that field. I think there are people who really look into what makes AI explainable in different settings. I think causality can So for me at this stage, causality is interesting and intriguing because it requires us to make our assumptions clear. While if we don't use causality, we still have to make assumptions, but we sometimes just not talk about them or we don't make them clear. So it's often less transparent. So if you take transparency as one aspect of explainability, then I would say yes, causality is the way to get there. But otherwise, I'm not comfortable to— I don't know where the explainability people would see causality, but I'm sure there are people who would agree that it's an important aspect of it. I'm not sure it's solving it.

All right. I think this is a good point where we can close the question round. Thanks again for the great talk. Thanks.

Thanks for having me. That was very interesting. Thank you.

I would love— we would love to have you again for part 2 on uncertainty. Great. Yeah, maybe Lena, do you have a few final words?

No, I just agree.

It was a great talk and we already booked you for the fall part if you're available. Very happy to come back. Thank you. Awesome. Thanks everyone for joining.