Scientist Stories: Florian Büttner, Machine Learning in Oncology

So welcome everyone. Good morning. This is the Data Science Seminar, and today we are also sharing the seminar with Alice Live Heidelberg. We are expecting a talk from Florian Büttner representing Frankfurt University and the German Cancer Consortium DKTK, and he will talk about trustworthy machine learning in oncology. Today, Oliver Stegele is the host. It's your turn, Oli.

Thanks, Irene, and yeah, well, warm welcome to everyone. It's our last seminar before Christmas and the last of this year 2021, so a special welcome to everyone, and it's my delight to introduce and welcome Florian Büttner today. So actually, I've known Flo since a long time because he was spending some time in Kingston between our group and John Marioni's lab and 2014 to 2016, so it's quite some time ago. So it's really great to have you here. So Flo did his PhD in medical physics at UCL, I believe, in the UK, then a postdoc at Helmholtz Munich where he got closer to biology and molecular biology and machine learning in these approaches before then moving on to the UK where he was a Medical Research Council Career Development Fellow. And then he had a stint at industry, which is I think quite special and not so often the case. He was at Siemens for 4 years before then rejoining academia. Since 2020, he is a professor at the University of Frankfurt and also a Philip Dicker Dicker. And so yeah, we basically have an almost within DKFZ seminar today. So it's great having you here today, Flo. And yeah, the floor is yours.

Yeah, thank you very much for that kind introduction. It's really a great pleasure of being here, and especially being the Christmas special, as I've heard. And fortunately, I didn't prepare anything, but still, I hope machine learning itself, trustworthy machine learning, is festive enough for you. But anyway, so before I go into the details, I'd just like to very briefly give you an overview of the research interests in in our lab. So what we do is sort of the application and collaboration-driven development of interpretable and statistically sound machine learning methods for understanding disease heterogeneity. And sort of what that means is that, you know, most of the things we do really are motivated from either direct collaborations or at least specific applications that we have in mind. And these applications and collaborations usually are centered around oncology, being part of the DKDK and the DKFZ. And the methods we use are usually machine learning based, and we do try and take care of making things interpretable and really statistically sound when we do things. Generally speaking, maybe I can group a little bit the projects that we work on in 3 or 4 different groups, ranging from translation single-centre genomics to probabilistic machine learning, sort of basic machine learning research. The first 2 blocks are where really the focus of the lab is computational proteomics and multi-omics data integration. And here we basically develop new technologies, sorry, we develop new computational methods for new technologies. For example, the other DKT professor in Frankfurt, Thomas Ellrich, he's brought some really exciting expertise in proteomics to Frankfurt. And with him, we developed new computational tools that allow us to use these proteomics data as a discovery source where we go quite a few steps beyond heat maps, and we've done that now for looking at breast cancer, identifying patient subpopulations for AML patients, for DLBCL patients. And we do that not only on sort of a proteomics single fuel point of view, but we really also look into multi-omics data integration. So very often in addition to proteomics, we also collect transcriptomics, you have data on mutations, clinical data, and these data views we integrate and look at interactions between the omics layers. And we're also a little bit interested in single-cell multiomics. I'm not going to talk too much about this set of topics, but I would like you to give a very brief teaser sort of into how we tackle those problems. And this is an example that we've recently published at KDD and now in the Journal of AI in Medicine. And here we show how we can encode prior knowledge in form of gene sets or pathways with sparse priors and thereby infer explainable latent variables. So what I mean by that is, you know, very often when you have like sort of your transcriptomics or proteomics data, you're going to do some kind of UMAP and you see your clusters or some kind of other dimensionality reduction. And like these axes, this, you know, UMAP 1 and 2 or t-SNE 1 and 2, are not necessarily very meaningful. So we want to sort of go one step further and infer like dimensions that are directly tagged by pathways. So we directly know what we are looking at because we think that's a lot more meaningful or can be more meaningful than just having these generic representations. But yeah, today I'm trying to focus on something else, also not translation in single cell genomics, even though that's also really exciting. We have lots of collaborations within Frankfurt and the UK as well on that. But rather, I'm going to focus the rest of this of this talk on probabilistic machine learning. And we're really interested in also developing new algorithms and tools from the machine learning side because we believe there are some really general key challenges that can be addressed with the framework of probabilistic ML. And in particular, that's uncertainty quantification, calibration, and domain shift as sort of one topic block. And this is what I really going to focus on for the rest of the seminar. We're also interested in other aspects like computational learning theory, explainable AI, interpretability, how to include domain knowledge and prior information, which I alluded to in the previous slide, but also stuff, how to build hierarchical models and include different levels of hierarchy between patients and cells and hospitals and so on. Yeah, so as I said, the one really big focus of our group is uncertainty quantification. So how can we make sure that the models we do train are trustworthy in the sense of that they're uncertainty aware, that they know when they know, and that they also know when they don't know? And to explain that a little bit further, I have like here a nice example that is from a study by Titus Brinker from the NCT in Heidelberg actually, and we have studied collaborate with Titus and also Achim Häckler, who's loosely associated to our group now. And the task they look at is skin lesion classification. So they get these pictures from skin lesions, and the task is to classify or say whether it's benign or malignant skin lesion. And now let's imagine, you know, they train a model with a really good accuracy, and then for example, they could have this type of image, and the model would say, okay, this is a malignant skin lesion, and I'm 99.8% sure. So the question is, can you trust that prediction, or when can you trust that prediction? And well, one could argue that you can trust that prediction if in 1,000 similar cases the model would be 998 times right, because then this confidence score here would be corresponding exactly to the accuracy. And this would mean the model is calibrated, and I can sort of trust this prediction in terms of that this has the accuracy it suggests. So now, unfortunately, this is not very often the case because, you know, when you take this image and you, for example, zoom in, then the model may say, okay, well, I've seen now this, this perturbed image and this is a benign lesion, and I'm 93% sure. And of course, that is a problem if the model makes these wrong predictions in an overconfident manner. So it's not necessarily a problem that the model makes wrong predictions because Of course, any model has its limitations, and at some point any model will make a wrong prediction. But what we'd like is for the model to sort of communicate that uncertainty in a way that the confidence scores again reflect accuracy. So if I perturb these images and I zoom in and do all kinds of things and it hasn't seen it before and the accuracy drops to around 50-55%, then I want this to be reflected in my confidence scores as well. I hope this is a little bit clearer now, but just to give you a different example, I have here the classical MNIST toy example to illustrate it a bit further. And so now imagine you have your MNIST model, you want to classify digits from 0 to 9, and you train some kind of like deep neural network. And then you take an image from the test set, say the 6, and you make a prediction, and then the model will say, okay, well, this is a number 6 and I'm 99% sure. And that's, you know, totally fine because, um, you know, you have like an accuracy of close to 1 for MNIST anyway. So this, this confidence score matches accuracy, it's more or less right, so perfect. Now the problem comes when I move away from this training domain. So when I take these digits and I somehow distort them— so in this case we applied some shear operation, you shear the whole thing, like pull it apart And then for each version of this digit, we again looked at the confidence score and what the model predicts. And you can see when we have like these small perturbations, then the model will still be very sure and say, hey, this is a 6 and I'm 98% sure. And that's still okay. The accuracy of this model will be around 98%. It becomes problematic though when the model sees stuff that it doesn't really know. And so at the very end, when you have these highly distorted images, we don't expect the model to make any meaningful prediction because at the very right-hand side, this is this is not really a digit anymore, right? So, but what we would hope is that the model is able to communicate this uncertainty that this is not really a digit anymore. And this should be done via its confidence score. Because we have 10 digits, we would hope for something close to 10%. But what happens is that here the model says, no, I'm still 99% sure, and I think this is a 2. And now this is where it becomes problematic because now the model makes these wrong predictions with a very high confidence. So again, it's not a problem that the model makes the wrong prediction, but it's a problem that it's not able to communicate its uncertainty and makes this wrong prediction with a very high confidence. And so that means we cannot really distinguish these wrong predictions from correct ones. So the stuff we are doing, so we've really set out to address this problem of uncertainty and calibration on the domain shift because we think this is highly relevant in medical settings where, you know, you have like distortions of images as I've shown at the very beginning of the skin lesions. Think of, you know, you go to different hospitals, you have changing patient demographics. So I think it's really crucial for all models to be robust in these settings and able to communicate when they stop being reliable. And so on the right-hand side is sort of what comes out of our approach, and I'm going to tell you in a minute how we get there. And here you can see something closer to what we would hope for. So at the beginning, when the distortions, the perturbations are very small, the model still is able to make the correct predictions as before, of course, and communicate this with a high certainty which reflects this accuracy. But now crucially, when you move far away from our in-domain setting, the model is also able to communicate when it's uncertain. So it's able to say, hey, here for these cases, I'm not sure what kind of image it is. If I have to guess, I'd say most of them are have a similar probability, but it's very close to like, you know, 10-20%. And for example, here at the end, it still says it's a number 2, but it's able to communicate its uncertainty much better than in the previous case. So before I tell you how we get there and show some more results, I'd like to introduce some way on how we can measure this model calibration or this quality of model uncertainty. And one way of doing that is via reliability diagrams. So what we do here is we can quantify the mismatch between model accuracy and model confidence in a systematic manner. So this is taken from an ICML paper 4 years ago, which really brought that back on the table for the case of in-domain calibration. And what you can see here is on the top of this figure from that paper where the authors basically said, okay, let's just train a U-Net model model on CIFAR-100. It's not a very complex model, but it's okay. And let's just look at the confidence score and look how it matches accuracy. And you would hope that the average confidence score should match the average accuracy, right? And this works pretty well in this case. But if you train a more complex model like a ResNet, then you see that the average confidence is much higher than the accuracy. So the model systematically makes overconfident predictions. And you can go one step further from these sort of global average views to sort of more local views and say, okay, I can take the confidence scores and I can just divide them into 10 bins. And then for each of these bins, I want the accuracy to match the confidence. So for example, if I have my bin here between 80-90%, then I would hope that for all these predictions which have a confidence score between 80-90%, the accuracy is also between 80-90%. But in this case, that's not true. In this case, for all these predictions between 80-90% have an accuracy of 40%. So here the model really makes highly overconfident predictions, right? And one way to quantify that is via the expected calibration error, or ECE. And here I sort of take the weighted difference between the accuracy and the confidence for each bin. So basically I take the gap that's shown in red here and weigh it by the number of samples in each bin, and that sort of tells me how reliable my model is, how well it's calibrated. And this is like a measure we're going to use a lot now in the next few slides. Right, so our methods, which published at AAAI this year, is called FALCON, Fast Adversarial Calibration for Domain Drift Scenarios. And here the idea is that we really want to make sure during training that our model later on during deployment is able to communicate its uncertainty in a calibrated manner. So, and for that, we sort of like came up with a new loss function where we added two terms to the standard cross-entropy loss. And the first term was about encouraging a uniform distribution of the scores, of the confidence scores, in case the model does not know. So basically, we take all the wrong predictions it makes during training and maximize the entropy. So this is a predictive entropy loss, and the model sort of learns that it's sometimes good to have like a very high entropy. And then in addition, we want to encourage the model explicitly to be calibrated for all types of out-of-domain samples, sort of these unknown unknowns that we don't know before that we're going to see. And for that, we sort of need one way to come up with generic out-of-domain samples because we don't know what kind of domain drift we're going to expect. Because, you know, if we knew, we could use it during training. If we knew, you know, the skin lesions would be a bit zoomed in, or if we knew that the MNIST digits would be torn apart or whatever, and then we could just use that. But in practice we don't know, and that's what makes it so challenging, that we don't know how the domain shift manifests itself. So the effect of deploying a model in a new hospital, for example. And our solution here was to use a very generic manner of coming up with a set of out-of-domain samples via adversarial. So basically we compute, we use fast adversarial to generate this generic set of out-of-domain samples, and we simply do that by using the fast gradient sign method. So we just look at the sign of the gradient and then add some noise on top of that in that direction. And for these samples, we explicitly optimize for ECE. So we have these two terms. One, predictive entropy. We maximize entropy for some of the model classifies incorrectly. And second, we directly optimize ECE based on these adversarial samples that represent a generic domain drift. So we evaluate then calibration across different types of dataset shift and different out-of-domain scenarios that were not seen during training. And the idea is basically because again, we don't know what kind of domain shift we expect, we just like look at all kinds of synthetic domain shifts that we can think of that the model hasn't seen before. And we also do that for different data modalities and different model architectures. We want to also cover all kinds of domain drift. So we want to have small domain shifts and we want to have very large domain shifts where the model sees data that's maybe erroneous, totally out of distribution, and see how the model performs across this entire spectrum of domain drifts. And of course we want to compare to other methods because, for example, you could think, hey, why aren't you just using Bayesian neural networks? Because they will have a good predictive uncertainty as well, right? Which is true to some extent, but I'll show you in a second why we can improve on that. This is just another illustration of the problem on MNIST, sort of what you've seen before, like in a summarized manner, where you can see like different levels of perturbations of the shear perturbations, denoted by epsilon, ranging from 0 in-domain samples to 100 where we have totally out-of-domain samples. And you can see when we increase these perturbations, what happens to the accuracy is, and on the top left here, that we start with a very high accuracy and then eventually it goes down sort of to around 20%. At the same time, the entropy of the prediction, so uncertainty the model has, does increase. So the confidence scores do get a little bit lower, but the problem is they don't get low enough. And this is what you can see in the ECE. The ECE tells us how low exactly these confidence scores should go. And you can see the ECE is quite low at the beginning when you have in-domain predictions, because the model basically always says 1, it's always correct. But as soon as we move out a little bit here, the ECE goes up really high and we end up with calibration errors of 40-50%, which is massive. Of course, MNIST is not really representative, So we actually focused on ImageNet in this paper, and there we have used just a set of standard corruptions that people apply to ImageNet when they want to look at these domain shifts. They range from Gaussian blurs to speckle noise to motion blurs and fog and whatnot. And what you can see here is that we only have one baseline, namely DeepEnsembles, and that's because Bayesian neural networks are very hard to fit on ImageNet. Basically can't really, or they performed like really a lot worse. It wasn't even worth showing. But even for deep ensembles, which do work and are actually a fair bit better than the standard vanilla approach, we perform massively better across the whole board. So the blue bars are our modeling approach and the orange one is deep ensembles. So we systematically improve on them here. Yeah, I've also included some data on 20 News Groups. This is a text dataset where the task is to classify news articles into 20 different news groups. And that was mainly to show that our modeling approach is not only working for images, but also for other data modalities. And in this case, we fit recurrent neural networks on these text data as a classification task. And also here, you can see that our method Falcon in blue has a much lower ECE of 0.15 compared the baseline of 45%, and even DeepAssembles at 20x. Right, so now we've mainly looked at sort of like domain shift scenarios where we walk a little bit away from the domain but it's still fairly similar. So you know, you can still recognize it's a 6 from the MNIST example, but actually we also want to know what happens when we are like really far out of domain. So when the model gets some input that has nothing to do with the training data it had received. And this we can do for MNIST if we, for example, plug in FashionMNIST rather than digits. It now sees items of clothing, and then what we'd like to see is really low calibration scores. So here, you know, we want to see the lower the better. And you can see that even though the model now sees items of clothing, it still makes predictions with a confidence score of, you know, 75-80% for most of the baselines. And only for our baseline, we get like down to scores below 30%. Same holds true for CIFAR-10, only for our method we get below 50% actually, again we are at 30%, and also for ImageNet we see that the mean entropy, so the uncertainty of our predictions, is much, much higher than the mean entropy for deep ensembles. So what I've tried to show you in this case was that Falcon is like a fast and generalizable approach to train uncertainty-aware deep neural networks So we came up with this new loss function to train the models, and we try to make sure that these models are well calibrated for any degree of domain shift. So from in-domain, where we are very similar to the training data, to really truly out-of-domain, to stuff the model has never seen before. And so what we're working on now is a collaboration with Titus Winkler and Achim Häckler, where we apply this to skin lesion classification and see how we can make sure to obtain well-calibrated models in these use cases. And we've also started to think a little bit about genomics applications with Ollie. Okay, so this framework that I've shown you just now is for training models. So let's say we have already a trained model and we want to fine-tune it with a new loss function, or we want to train from scratch. But actually, there's already in the literature all kinds of manners how you can calibrate train models in a post-hoc fashion. So that means you have a trained model and you don't touch the weights. All you do is you take the logits or the predictions of the model and transform them in a way that they become more calibrated. And those of you who've been around for a while, they will remember SVMs and Platt scaling, which was introduced to make sure that the distance to the decision boundary in SVMs would be calibrated. And the same thing is something you can do with neural networks. And in that 2017 ICML paper that I've showed you at the very beginning, they've showed that that you can use simple things like temperature scaling, which generalizes plot scaling to basically the multiclass classification based on just one parameter, works very well. And basically it's quite well established that we can recalibrate neural networks using these post-hoc calibration techniques. But what people hadn't looked at before is how it looks like when I am faced with domain drift. Scenarios. So, you know, you have these neural networks, you calibrate them, but then you get these samples in from a little bit away from your training domain. So how does the calibration look then? And this is what we looked at in this paper. And this is sort of like an illustrative example again that I want to start with. So here we look at ImageNet, and you have like a doormat as one of the samples in the test set. And if you just take an uncalibrated ResNet and you make a classification, a prediction of this doormat sample in the test set, then the model would say, yes, doormat, 99% sure. Same holds if you use temperature scaling, so like the standard calibration method that most people use. Same thing, and for all methods, same thing. But now it gets interesting if we have some changes in the scene. So here we've taken an image from ObjectNet, and ObjectNet is a curated dataset that was motivated by ImageNet. But they sort of took images of the same objects from different viewpoints or in different contexts or from a different angle. And so you just have like a real-world domain shift, basically. It's still the same object, but just taken from a different angle. And so this is still a doormat, but the angle changes and it's like turned a little bit. And so what you would hope is that either the model still classifies that as a doormat or it makes different prediction, but its uncertainty, because it's quite different from the Dom it has seen, is reflected in the confidence score. And for the uncalibrated model, it says it's handkerchief and it's 91% sure. If you use temperature scaling, this gets down a little bit, but the model is still quite certain or quite confident in its prediction, its wrong prediction with 85%. And with our prediction and our calibration methods, we can actually get that down and the model is able to say, hey, yes, this This is a hanky, still wrong, but at least it's able to communicate its uncertainty. And this holds also for all kinds of other predictions in object detection. Okay, so we looked at that a bit more in detail, and the first sort of contribution in this paper was really to show that if we use post-hoc calibration methods and then look at domain drift scenarios, these calibrated models get highly overconfident under domain drift. And this is again an example on CIFAR-10 where we trained VGG19 model on CIFAR-10, then performed like a few standard post-calibration methods and simulate some domain shift by increasing shear, and then evaluate ECE under domain shift. What we would hope for is that the model becomes more calibrated. And this we can see at the beginning. We have in black the base model. This is uncalibrated for in-domain scenarios with a calibration error of 10%. Once we use any of these state-of-the-art calibration methods, it gets down to a few percent, 2, 3, 4%. But the problem is when we now are faced with some domain shift, then at the beginning, yeah, we still have a good calibration, but as soon as we really walk away from the domain, the calibration error increases massively. And for the uncalibrated model, we are at more than 60%, but even for the supposedly calibrated models, we have calibration errors of 45-50%. So that means when your model says, you know, I'm 80% sure, really it's probably only like 30% sure. And this information on the confidence is of course really important when you have like medical applications. All right, so how do we get to these better calibrated confidence scores? Well, our idea was to say, okay, I mean, all these calibration methods, they rely on a validation set which is then somehow used to fit a post-processing method. And so our idea was to again not use like a standard, just the normal validation set, but come up with a generic validation set that in some way reflects all kinds of domain shifts from in-domain to completely out-of-domain. And the way we we did that was that we chose direction and magnitude of the domain drift randomly and basically thereby had introduced this variability across the domain shift spectrum in the validation set. So that means now we have like this transformed validation set and we can fit all these post-hoc calibration methods that exist as before, just on a different type of validation set.. So it's a very simple, simple idea. And then we evaluate the whole thing on previously unseen perturbations as well as out-of-domain scenarios as well as in-domain scenarios. So let's see how that works. Oh yeah, so this is just like an illustration again of what domain shift and out-of-domain mean. And sort of domain shift is this ObjectNet example where you have these images from ImageNet and just like have these different viewpoints in different contexts and so on. And so these are all chairs in ImageNet, and ObjectNet chairs would look look upside down, taken from a different angle, from above, being put somewhere outside the house, being a plastic chair, and so on. So it's still all chairs, but it's sort of a real-world domain shift. Out-of-domain would be I have like my digits from MNIST and I have like these fashion items that have nothing to do with digits whatsoever. Okay, so if we now look at the ObjectNet example and We look at calibration if we make predictions on ObjectNet on these pictures from a different viewpoint, then we can compare the calibration scores, sorry, this is the wrong label, the ECE for post-hoc calibration methods fitted on the validation set or post-hoc calibration methods fitted to our generic validation set. And our method's always shown in a bit of a darker color and the baseline calibration is shown in gray, like in a lighter color. So for example, temperature scaling, the most common one. We have like standard temperature scaling in light gray and our approach in dark gray. And what you can see sort of is that the dark colors are always a fair bit lower than the light colors. So that means whatever calibration method you choose, you know, just choose whatever you think works best. Maybe it's isotonic regression in some cases, then you will get an improvement for these domain shift scenarios if you use sort of our than our validation set approach. We of course looked at that for all kinds of other corruptions as well. This is again ImageNet and the canonical corruptions here. And again, like sort of the dark bars are always like a lot lower than the light bars. And if you average across all these corruptions, you can see that our approach, which is shown in green, really improves massively over the standard tuning approach in these settings, especially for CIFAR, but also for ImageNet. Maybe one last example for the out-of-domain calibration. Then we also see for SIFA that sort of the dark bars again are a lot lower. So this is really confidence scores here now. So here we see that the confidence scores of samples that have nothing to do with what the model saw during training are substantially lower, reflecting the model's uncertainty much better than with standard calibration methods. And compared to baseline, we see like a huge improvement even for standard calibration methods, and the same holds for emission as well. Okay, so that was all about how we can sort of use existing post-processing methods, post-hoc calibration methods, and improve them for domain shift scenarios. But then, you know, once we evaluated all these models, we thought, actually, these temperature scaling-based methods, they are quite limited in a way because they have a fairly low expressive power, and the problem is they they transform every set of logits in the same way. So if you have temperature scaling, you use the one same temperature to transform every single prediction. And so our idea was to actually improve that a little bit. And our hypothesis was that we thought that if we increase the predictive expressive power of these methods by parameterizing temperatures itself, then we can also get a better performance in terms of ECE. So in other words, we want to compute a prediction-specific temperature. So every prediction the model makes gets a different temperature that transforms the logits, and we want to use side information from the model itself and via the logits because there's a lot of information in the model already somehow. And we thought, okay, why not, why not use it? So we fitted a neural network calibrator that used information from the logits to fit a prediction-specific temperature. And when you do that, you can see that extended calibration errors decrease from like 2% after temperature scaling into 0.5% using our method, and ETS is extended temperature scaling, previous proposed extension of temperature scaling, and still is 50% better than that. And here's like some, some more other baselines, and you can see that overall our PTS method really improves substantially over the state-of-the-art just by computing this prediction-specific temperatures and using this site information from via the logits. Okay, so in summary, standard tuning or standard postdoc calibration results in highly overconfident out-of-domain predictions and uncalibrated models in domain shift scenarios. And sort of our approach yields well-calibrated predictions from the in-domain scenario to truly out-of-domain scenarios, and it can really be readily applied by to any post-hoc calibration methods. And I've just showed you very briefly how we can even improve these post-hoc calibration methods. So what we work on at the moment, an unbiased estimator for multi-class calibration errors, because actually what we've seen is that this use of histograms to compute the EC is highly biased and not very reliable. So we came up with a general framework how we can define calibration error as a proper scoring function, and this is something we're going to submit in the next few weeks and also look, for example, at other more theoretical aspects like how does imbalanced data, how do imbalanced data settings affect the calibration and so on. Right, so I was going to talk about domain generalization, but I'm going to skip that in the interest of time and rather take the next 10 minutes or so to talk about multi-output Gaussian process latent variable models as uncertainty-aware recommender systems. So now it's a bit of a gear switch, and rather than looking at these standard classification models, we're interested in recommender systems. So we want to have a model that makes a recommendation, and the classical example in machine learning literature is movie recommendations. So you have, you know, a set of users and a set of movies, and you know, like, some movies that a set of users like, and then you want to predict what other movies that specific user may like. And for example, you know, classic application is Netflix, what you've watched before, what you may want to watch next, or Amazon or Hulu, I don't know what streaming service you're using. So, and when you make movie recommendations, it doesn't really matter how trustworthy these recommendations are, or when Amazon recommends you the next product you want to buy. All that matters for them is that the average recommendation, the average performance of the recommender system is a little bit higher. Because if the average recommendation is a bit better in your shopping thing,, then the company will make more money because it sells more. But for us, if we look at clinical decision support systems or other sensitive applications, we really also care about measures beyond this average performance. We do care about uncertainty. We do need to communicate our model when it's not certain, when we even want to think about using these types of methods in the tumor board or making treatment predictions— sorry, treatment recommendations. Because if the model can communicate the uncertainty, then the user knows which which individual recommendation is likely to translate into a good recommendation. And then the user can focus onto those recommendations where we already believe because of our uncertainty that the recommendation will not be very good and focus the resources on these ones. So the problem setup is that we want to infer some kind of value, some recommendation for between a given user and a given item. And this can be user of streaming service and movie, or it can be patient and treatment option. And so the simplest approach to do that is collaborative filtering, where we decompose a matrix into a product of two other matrices. So here on the top is our input data. We have users times items, and we have lots of blanks in this matrix, which we want to fill in. And y_ij would be the recommendation for user between user i and item j. And we can model that best, for example, as a product between two matrices A and B, where A_i is sort of a latent representation of the users and B is a latent representation of the items. And then there are some bias terms and this works quite well. Sorry. But you can also make this a bit more complex, introduce some nonlinearities by fitting a multi-layer neural network where you say, OK, I learn embeddings of items and embedding for users and I stick them together and propagate them through to some kind recommendation. Here you have like, you know, very good performance, but the problem is you have no uncertainty information, and this is of course a limitation in some critical or clinical examples or applications that we have. So a natural alternative to these models are Gaussian process latent variable models, where you have like the big advantage that you would actually have uncertainty information. And here in GPLVMs, you basically model the column or each column of this matrix of users and items as a Gaussian process with covariance matrix K. And so yeah, the nice thing about that is that you actually have a predictive distribution here instead of a point estimate, but you have no explicit model of the interaction in the output space. So we don't have currently, with standard GPLM, no way of sort of learning this representation or this correlation between between the items. And of course, there's like a very strong correlation structure between the items, between the movies or whatever. So our idea in this work was that we wanted to come up with a scalable solution to join the strengths of both approaches. So very brief summary of Gaussian Process Latent Variable Models in case you're not familiar with them. So you probably are familiar with GP regression, where you're given some kind of input x and output output f(x), and then you choose a kernel, learn some hyperparameters to describe that mapping. In the GPLVM, only your outputs are given and you learn the inputs. So that's why, you know, you have like latent variables that construct the kernel. And this is sort of representation learning, and you have this mapping from an unobserved latent space to an observation via Gaussian processes. And the latent space is part of that kernel here that you construct, you know, however you want, either as a linear kernel, RBF kernel, or mod. And you can put some prior on x as well, usually like standard normal prior. Okay, but as you see, all dimensions are treated independently. Okay, so how can we include these interactions of the outputs? So we basically borrowed from supervised settings where for supervised standard GPs, the intrinsic model of coregionalization, IMC, is used, where basically you construct this covariance matrix based on a kernel that's working on the inputs and take the Kronecker product to some matrix W that's capturing the interactions of the output variables. If you want to think about how to generalize that to a latent variable model, you know, you could do the same thing, except that your kernel is not working on the observed inputs but on latent variables, and you still have your W output capturing— W matrix capturing the interaction between the outputs. But the big question now is, of course, how do we construct this W matrix in the case of the GBLDM model? Setting. And so our proposal was to also learn an embedding of the outputs. So now we don't only have unobserved inputs, but we sort of also have unobserved outputs in the sense that we want to learn a latent representation of the outputs. And then we can apply also a kernel on these learned embeddings and use this kernel to construct W. So if you want to think about this whole thing in the GP world, then you have your standard multi-output Gaussian process regression where where you have observed outputs and observed inputs, and you have a dependency between the outputs that you model. For the GPLVM, you have an unsupervised task and you have an unobserved input that you model, but you don't have any correlation, you don't have any dependency between the outputs. And then recently there was the latent variable multi-output Gaussian process regression model, a lot of letters here, and there you have a supervised task, but you have this correlation by kernel. So they also learn a latent representation of the outputs. And here in our model, we combine all of that. So we have the latent representation for the outputs, correlation by kernel, unobserved inputs, and unsupervised task. So okay, what's the catch? So the obvious catch lies in the Kronecker product, and it's very expensive to do that, right? But we had quite a simple idea on how we can make this whole thing computationally efficient and scalable. And And that was by coupling inducing points. So when we do inference for these GP models, we don't tend to work on the full covariance matrix because it's far too expensive. We always work with the inducing points that you need a lot less of than the actual number of observations. And if we choose the same number of inducing points, we can go down from this complexity to that complexity. So we basically lose the MB squared here and we introduce a sparsity factor here. And even by that, we are already a lot, lot faster than even standard Bayesian GPLVMs where we don't even have this discretization. Okay, I'm almost done. Just want to show some results of how that works in practice. And so here, what we did in our UAI paper that came out this year was to just look at the standard MovieLens datasets, for example, and CHESTER dataset. And we were really interested in two things. First, we want to evaluate the predictive power, so reconstruction power of our model in terms of RMSE, so root mean squared error, or mean absolute error. And we're also interested in the quality of the predictive uncertainty of our models. So MAE and RMSE, we can compute for all kinds of different baselines. And the predictive uncertainty is a bit trickier. So what we did here was to say we take the standard deviation of our model, so the predictive uncertainty, and just look at different quantiles from the, you know, 100% quantile, which would be all predictions, to the 10% quantile, which would be the 10% of most confident predictions. And then for each of these quantiles, we compute the reconstruction error. And what we would hope is that for increasing confidence, so the lower the quantile is, the lower the error is. And we can see that, and we can do the thing for our GPU VM, sorry, for our model, but we can also do it for the Bayesian GPLVM where we don't have any correlation between the outputs, just to see what the effect is here. And you can see that our model has a better reconstruction error than the Bayesian GPLVM, and both models sort of get better with decreasing uncertainty, so with a higher confidence, which is exactly what you want. And the dashed line here is the state of the art. So if we To be honest, I forgot what it was for Movielens1M, but basically if you have like one of the standard deep neural network recommender systems without predictive accuracy, this is the performance you get. And already for our model, if you just discard the 20% most uncertain samples, we are better than the state of the art. So in a practical application, that would mean we can focus on these bad ones with the highest uncertainty and where like a clinician has to look at himself and the ones where we have a really high certainty, we can know we can trust these recommendations, these recommendations will translate in a good prediction. Did the same thing for MovieLens10M, Bayesian GP-LM we couldn't fit, it was too inefficient, and here we are even better. So we only have to discard like the top 5% of our samples and then we're better than the state of the art and have mean absolute errors here that are better than anything deep neural networks can do. And the important thing is that we can identify these samples where we do worse as opposed to the deep neural network. All right, so what we've shown here was that we could extend the GPLVM to account for dependencies in output whilst maintaining scalability. We have since extended that to the multi-view setting, so we don't have only one view as an input but several views, and we are currently applying that to modeling proteogenomic data and looking at gene-gene interactions and inter-view interactions. You can use them as a either directly work on latent representations or use it also as a method to impute missing data. So yeah, finally I'd just like to make a couple of acknowledgements. I'd like to thank Yincong from Siemens with whom we came up with the GPLVM model, and then Christian Tomani who used to be an intern of mine, is now doing a PhD with Daniel Krämer at TUM, with whom we kicked off all the uncertainty work, Xudong on domain generalization that I actually didn't really talk about, and of course our group at DKT in Frankfurt. And yes, also as thank you to Siemens, with whom I did the early bits of that work. And thank you for listening. And yes, I'm of course happy to take any questions. I see there's some stuff in the chat. Great.

Yeah, thanks very much for a great talk. Fantastic overview on different science questions. I mean, please all ask your questions questions. I'll start with the questions in the chat, and Philipp Winter is the first one. Do you want to ask the question live, Philipp, or should I read out?

Yeah, I can just read it maybe. Thank you for the talk, it was interesting. My question would be, would it make sense to use a dedicated out-of-distribution or anomaly detection method, like for example generative models which learn the actual manifold of the data data instead of using regularization loss terms? Because if you use this regularization or these other loss terms, this typically harms classification accuracy.

So I mean, we looked at accuracy as well, and so we could show that accuracy wasn't affected. So our Falcon models had pretty much the same accuracy as before. So then your suggestion would be to train a model as normal, and then on top of that use anomaly detection, right? But so the problem with that is that we again don't know, we don't sort of get a calibration kind of thing, right? Because we're not necessarily only interested how far away a sample is from the data manifold of the training data. We're interested in how well the classifier actually works on that particular sample, because maybe it's able to generalize to that sample sample, and then we can still use it. We only need to worry when it's not able to generalize. And this is it able to generalize kind of thing, we can directly see in these calibrated confidence scores, which is why I think they are so powerful. So I think anomaly detection or detection is good if we only need to identify samples that are really far away from the training distribution, because then we know the model will definitely not generalize. And if that's the task, then yes, absolutely, then don't use and just think about calibration, use anomaly detection. But if you're interested in sort of in the in-between part, this is where I think the calibration concept is quite powerful.

I see, thank you.

Great, we have another question from Thiago Borges. Do you want to also ask your question live? Otherwise I ask the question. Oh, hang on, this was just the link to the— Sebastian has a question. Apologies, do you want to go next?

Yeah, thanks, great talk.

Did you look at invertible neural networks or normalizing flows in comparison to your methods for uncertainty quantification? Yeah, so one of the baselines we did show was based on normalizing flows, one of the Bayesian networks, and I have to say it didn't really really perform very well and it wasn't scalable very well. So it was— we didn't get it to train on ImageNet, actually. But it is definitely an interesting approach in terms of looking at things. And so the model we used is already like 2 or 3 years old now because we did that work in 2020, 2019. So I think some more stuff has come out and definitely be worth looking into that, those newer things. Thank you.

Yeah, I've also maybe a question related to your last part, the GP model. I was wondering, I didn't quite get it. I mean, do you get around this issue to have a covariance, maybe the scales with a number of items per user? Because that obviously will still limit the number of items you can model eventually by cubicness of this, or did you get over that as well?

I have like this capability. So yeah, so no, it scales linearly with the number of users and the number of items. And this is the sparsity factor. So if we only observe 10% of potential observations, this directly reduces the complexity as well.

And this— how is that done?

So what's the speedup you're using for the covariance Oh, we just use inducing points.

Just using one, you're just saying if you don't—

so if you have something, you don't need to use it.

Okay, I see. Okay, and this is quadratic where m² is the number of inducing points.

Is the— you're talking— great, no, I understand that.

That makes sense.

Thanks.

And you're basically just using the data as inducing points?

Yeah, in this case you don't— uh, no, no, we just use the standard variational approaches where the number of induced points is low, like a lot lower than the number of observed users. So this would be, I don't know, maybe 100, 500 or something for the Movieline 10M. Okay, great, that makes sense. Thanks a lot.

Yeah, okay, are there any other questions? Anyone wanting to ask a question spontaneously now, raising a hand? Otherwise I have one more. Apologies, jumping on the opportunity here. And with the calibration question, I mean, I was wondering what would be the side information you could need to improve the out-of-distribution generalization performance? Would, for example, labels, I mean the inputs without the labels help in an element? I'm just wondering what's the next step to basically generalize to the unknown where you don't know what it is, wondering is there any proxy data you could collect to basically make this out-of-distribution prediction or the robustness to these out-of-distribution problems better?

Yeah, no, it's a very good question. This is actually what we're thinking about at the moment. And I think one idea is also what Philipp alluded to, that you can, for example, also try and learn the distribution of the data and then sort of come up with a measure how far away the sample is from your input distribution. And that could also be an interesting bit of side information that you can stick in.

So if a generative model of the input— For example, yeah. At the moment you don't have sufficient on the inputs, right? So basically at the moment you we don't use that.

Yeah, exactly. Yeah, exactly. So at the moment, we can sort of implicitly condition on the inputs by looking at information that's present in the model. So how the model itself encodes the input. But we can also, of course, look at the input directly, or how a different model sees the input, especially like this type of OOD anomaly detection methods via manifold learning.

Okay, okay, great. This is your last chance to ask a question. Oh, we have one Martin Schneider, go ahead.

I have a question. I'm not from the machine learning field, but I was wondering concerning controls in the number example and in the skin lesion example. Are there controls like for the numbers, letters that are not assigned as letters, just as non-numbers? And in the skin example, just healthy skin? Because I guess you need a reference point to say how certain you are if away from something, if you do not know what something is, the distance may give you a wrong assumption.

Yeah, and I'm not 100% sure if I get the question, but I think the idea is that you're wondering, is there sort of like a specific class in the model that's none of the above, so that encodes like this, or the totally out-of-domain class, right? And we don't do that at the moment. We argue that sort of when it would be something that's completely different, then the model should make a prediction with a confidence score that reflects randomness. So if you have binary classification, confidence should be around 0.5, and we use that sort of as an integrated framework. But yeah, I mean, it's that— that's also like an orthogonal way of looking at things and having this specific class of unknown. But we're not using that in our modeling approach.

Is it normal, I would say, that control is not included, like in the skin lesions, or is healthy skin included in such models? I find it very important because I once heard a talk where it was about about facial recognition of sickness in some kind, and only sick persons were included, or sick faces, into the model. And of course, if you put any face into this model, it assigns a certain sickness.

It has to, it has no other option. Exactly. And that's exactly one issue that we can address with the calibration framework, because we say, okay, if that then happens, and like a health uncertainty phase is in the model, then the model will say, I don't know, it could be any of the sicknesses, any of them is equally likely because none of them fits. This is how we use it. And then I mean, you could, you could like, you know, make a threshold then and say, okay, then I directly assign this unknown class. So you could do it as a post-processing step easily. But, but for that, you'd have to have like these calibrated confidence scores. Yeah.

And I was just wondering, I find it quite rare that controls are included from my perspective.

I'm not into the field, No, that's right.

Yeah, most easy way to have a controller. But I find these scores very, very helpful. I was waiting for such an approach for quite a while, I admit.

That's good to hear. Great, thanks a lot.

I think if there are no more questions, we've already had quite a good discussion, so I think we can close at this point. So yeah, I'd really like to thank you again, Flo, for presenting. It was a pleasure.

Interesting, stimulating talk.

And if anybody would like to meet Flo and have a conversation about anything, maybe, you know, just follow up with Goran or Doreen, and I'm sure there could be things arranged. And with this, Merry Christmas and all the best for the new year, and see you in 2022.