Scientist Stories: Nicola Rieke, Federated Learning for Healthcare

Welcome everyone, this is the Data Science Seminar today with our speaker Nicola Rieke from NVIDIA. I will hand over to Lena Meyerhein to introduce her.

Yeah, it's our great pleasure to have Nicola here today. She studied maths and medicine in Aachen, and we just talked about it, she just couldn't decide between medicine and maths, so she studied both. Quite impressive. Then she did her master's in biomedical computing at Technical University of Munich, and then a PhD in surgical data science with Naseer Nawab in Munich. She's now a senior solution architect at NVIDIA for deep learning in healthcare, but kept the links with the research community, for example, by being very active in MICAI and IPCAI. Yeah, I first met Nicola, I think, at MICAI 2015 when she won a best paper award.

Paper Award.

It also was one of those last present on the dance floor at an Oktoberfest event, so that was a very good overall performance there. And yeah, the stage is yours, Nicola.

Thanks a lot for this nice introduction, Lena. It's a pleasure to be here. I would have preferred to visit you in Heidelberg, so I like to visit you there, but unfortunately these times We have to do everything digital, but the good point is that we also remote people can join. So happy to be here. Let me share my screen. So today we talk about a topic that gained quite a lot of traction in the last couple of years or months. So we're gonna talk about federated learning. So federated learning in particular for healthcare. So it's a very general learning paradigm that was introduced several years ago, maybe 2015 when I met Lena, maybe before. So it was first introduced to a very different domain, but gained a lot of traction in the last couple of years for the healthcare domain for good reasons. So there is a lot to talk about, a lot to cover regarding this topic. I will focus today on the perspective paper that we published last year. So this is perspective paper where several thought leaders of this field, not only from NVIDIA, but also for example, Klaus Meyerheim from DKFZ, George Cardozo from King's College, Shadi Al-Bakouni used to be at TUM, now is at Helmholtz, Bennett Landman, Ron Somers, for example, provided some clinical perspective of it. But also Andrew Trask. So it's really a combination of several thought leaders of this field coming together and discussing the potential of federated learning for this field. So the medical federated learning use case is inherently different from other domains. Let me quickly get rid of the Zoom part here, otherwise I can't see. So as I said, the medical federated learning use case is inherently different from other fields. We have way more, way bigger models. We have less participants in terms of number of clients, for example. So one of the original domains that we have seen is smartphones. There you could expect millions of participants. In our field, it's highly unlikely that you find millions of hospitals that are willing to participate in this kind of collaboration. So this comes all with a couple of peculiarities in terms of technique and in terms of setup. And in this paper, we discussed the different points of view of that. So it focuses on the benefits and the impact for federated learning for medical applications and highlights some key considerations and challenges of implementing federated learning in the medical space. And as you see, so we published it— when was it? 14th of September 2020. Within 6 months around, we actually got a lot of interest in there, so 90,000 views, and I think right now we have 40 or 50 citations within 6 months. So there's a lot of interest in this field, and it's a very, very active field. It's impossible to cover everything in the talk, so I will really focus on this consensus view. That we developed with several thought leaders of this field. So whatever I say, it's not necessarily only my opinion, but it's several really bright minds contributed to this. So let's start. You know, at DKFZ, you all know that AI really transformed the medical field enormously in the last couple of years. And particularly advances of machine learning in general and deep learning in particular really led to a disruptive innovation, not only in radiology in terms of CT, MRI, ultrasound, or X-ray, but also pathology, genomics, dermatology, or completely different data types like electronic health records. Across the field, this data crunching and gaining insights with the help of AI really innovated the field enormously. Led to new applications that were not possible before or accelerated the process or improved accuracy, for example. And this was really across the field of many, many medical domains. So there are thousands of research papers out there now that really leverage machine learning and deep learning for various different medical domains. So really tackling the most complex problems with enormous success, not only in terms of creating new architectures, but also combining steps that have previously been impossible by explicit programming, for example. So they really improve state-of-the-art performance. However, in research, we usually have a very controlled setting. We try to aim for huge datasets to really simulate a real-world scenario, but usually what we encounter in the research environment is a very controlled setting. However, AI also managed to make the transition from being successful from this really research task in an experimental setup to achieving human-level performance on large data and clinical settings. So really in the wild. So on this slide, you can see a couple of examples of clinical applications in terms of dermatology, endoscopy I think is on there, histopathology, and ophthalmology. And what they all have in common is that they train and evaluate on extensive datasets. So for example, if you develop an AI-based cancer detection algorithm in a supervised way, for example, it really requires a large annotated database that really encompasses the full spectrum of possible anatomies and pathologies. So really different manifestations of the demographic factors like genders and age. So if you consider the skin, this can vary very significantly from different regions. But this also includes technical circumstances such as the acquisition protocol. So an MRI is never the same, let's say like this, for example. So depending on the equipment manufacturer, it can also vary quite a lot. And if you just develop something in a very on a very small dataset, you don't necessarily have the guarantee that it will perform well on a different one. So we entered an era of data-driven medicine, which is great. We have a lot more potential now and can discover new directions. However, we really rely on large and diverse datasets that really represent the problem that we want to solve. So currently a lot of research is really driven by data lakes. So challenges like the EndoVis Challenge or the Chameleon Challenge really foster the research in this direction. However, in research direction where we can't send, like create such a dataset and share publicly, we don't see this kind of research evolving that fast as in this kind of, in this domains. And centralizing or creating a dataset that is publicly shareable is very difficult for the medical domain. So health data is highly sensitive, subject to regulations for a good reason, and can't easily be shared. So even if you think about that some data can be anonymized, so de-identification, there are methods later on when you already shared this data that can re-identify a person. So for example, if you consider electronic health records, just a couple of data points can make a single dataset really not de-anonymized anymore. The same is, for example, for brain images. So a brain image like an MRI can be as unique as your fingerprint. So even if your name is not on there, depending on the situation, there, de-identification can happen at some point. So a large reidentification of patient can't be ruled out in general if you centralize the dataset and make it available. So how can we enable a data-driven medicine that respects the highly sensitive nature of health data, but also allows to develop algorithms that are— that avoid demographic bias or how to develop new algorithms on rare diseases. So we really rely on data at this point if you want to develop algorithms that are data-driven. So we want to avoid biases in terms of demographics, technical imbalances, and all the components that could impact the prediction or the accuracy of a model. So there are great examples of really good efforts to create large datasets. So for example, the NHS Scotland's National Safe Haven, the French Health Data Hub, the UK Biobank, the Cancer Imaging Archive, the Cancer Genome Atlas, several medical challenges for the research world, for example. So all these efforts are great and they really foster the research and the insights for different domains, but it's not that easy. So as I said, there are regulatory and ethical challenges, but also technical ones. So in our domain, we can't necessarily centralize the data and create a dataset that would be necessary to really create a model that is able to address the problem in a more general sense. So the promise of federated efforts in this context is the following. So instead of bringing the data to the location where the model is trained, let's bring the model to the data. Let's keep the data in the institutions and the hospitals and train directly there in-house. So the concept of federated learning or a federated effort is to share the insights in terms of model characteristics or model updates. So it allows a collaborative learning without centralizing the dataset itself. Itself. And by this, it addresses a very important problem that we currently have in developing medical AI. So it addresses the privacy and data governance challenge. So in the federated learning setup, each data controller, so each participant client here, can not only define its own governance process and the associated privacy policies for their own data, but it also controls, and this is important, the data access and the ability to revoke it. So it remains in full control of when, at which time point, which algorithm can be trained on the local datasets. So instead of giving the data away, you remain in full possession of them. So in this context, the AI training, machine learning, deep learning, whatever you want, occurs locally at each participant and not in the centralized way. So let's assume there's a hospital that has a model, some data and some compute resources that would like to tackle a specific problem. However, the dataset that this institution has locally is not sufficient to train a model that is capable to address this problem. However, he has a friendly other hospital that he wants to work together on this problem. So they have a common goal, they want to solve a common task. And the dataset in the other institution is different and hopefully complementary to the dataset that the first institution has locally. So they want to work together, but are not allowed to share the data. So how to train collaboratively without sharing the raw data? And this is exactly the focus or the purpose that federated learning tries to address. And in some way you can imagine federated learning as a natural extension of the development that we have seen in the last couple of years. So federated learning is a kind of new learning paradigm, a new way of learning that addresses this data governance and access challenge. So in the maybe traditional, so several years ago way of training, You usually started like you defined your model and you started from scratch. So you had your own dataset, huge dataset, you had to curate it, you had to label it, then you had a couple of iterations of training and you developed your model like this. So this required a huge dataset locally. In our field, so we don't have an ImageNet correspondence for the medical field. So we had to wait a little bit until the deep learning wave actually entered in our field. And this is mainly because we don't have this huge dataset at hand to develop the models from scratch. So what was very dominant in the last couple of years is this kind of transfer learning. So instead of training a model from scratch, you start from a pre-trained model. So this pre-trained model can be trained on a completely different data domain or in a similar one. But the weights already converge to a direction that the model, for example, in terms of deep learning understands edges, corners, and so on. And then you locally train on the datasets that you have locally just in terms of fine tuning. So modifying the weights into the direction that makes sense for your application. This had the major advantage that your local data that you needed was not as big as training from scratch. So you still have a local training, but you adapt to an external model. So federated learning goes one step further. It probably won't replace the cases where you do, so the traditional AI training and the transfer learning itself, but it addresses cases where you can't leverage a pre-trained model for your purposes. So federated learning is still local training, but this local training happens at several clients, at several institutions, and they all collaborate at the same time iteratively on a global model. So instead of just fine-tuning an external model on your local dataset, you do a couple of iterations here and then you share your model updates that is aggregated in a global model update. So important here is that the local steps that are necessary for AI training remain the same. You still need data, you still need all the preprocessing steps that you would usually do for AI training. However, the nice part is you collaborate on a global model and everybody gains from the insights from the others without talking to each other directly. So let's go through a federated learning round just as an example. So let's assume we have 3 hospitals, like a community hospital, research medical center, and cancer treatment center. They all have private data, they all have the same goal to work towards, and they all have compute locally. So in a client-server architecture, for example, you could enable this federated learning, this communication via a federated server. So you start from a global model. You decide we want to work on a segmentation model, for example. You start with an initialization. The model is sent to the local clients. They do the compute locally on their datasets, achieve a local model that are very different. So I did not color code it here, but all the three models are slightly different because they incorporate the insights that were gained on the private data. And after a couple of iterations, so not until convergence, after a couple of iterations, this model insights or this model update or the model characteristics are shared with a federated server. The federated server collects it and combines it to a global model update. So as Lena said, I studied mathematics, so I tend to like, formulas in the slides, but I learned that not everybody appreciates this. But still, just to give you an insight what this could look like. So aggregation in the end is just a weighted sum, can be, for example, just a weighted sum of the delta between the global model and the local model, or the gradients or the parameters, whatever you choose. And here, for example, it's weighted, a fair weighted average. So it's normalized to the size of the local datasets. But in the end, the operation, the mathematical operation that takes place at the server side is not computationally very expensive. And in the next iteration, you send the updated model to the clients again, and they start again with the training and so on. So it's an iterative loop until the convergence is achieved. Is there. So federated learning implicitly offers a certain degree of privacy because the raw data is never exposed. So the federated learning participants never access data from the other institutions and they only receive the model parameters that are aggregated, combined, for example, in the weighted sum over several participants. In a federated learning workflow— there's something showing. I hope you can still hear me. In a federated learning workflow, the participants can even remain unknown to each other. However, there are a couple of things to consider. So there are techniques to interfere this training process. So for example, model inversion that you have in traditional machine AI training for deep learning models. This is, of course, also consistent here. Is in theory possible to do model inversion and gain some insights about the local data distribution. But luckily in federated learning, there are several techniques to counteract for that. So privacy-preserving techniques, I will come to that in a second. So in the end, federated learning is a learning paradigm which enables multiple parties to collaborate on a model or part of the model without centralizing the dataset. So it addresses not all the open questions of AI training, but addresses a very important one. So it addresses the privacy and governance challenges in this context. And by doing that, it could create new opportunities. So probably federated learning won't replace the applications that we already have tackled successfully, but it might create new opportunities. So for example, it could enable precision medicine at large scale that was not possible before. It can naturally grow with the dataset size because this iterative process, you can add new data on the way, for example. One important part is it can enable novel research for rare diseases where it's simply not possible to create a dataset that you can centralize and make publicly available. So for rare diseases. And an important part is also the large-scale validation directly in the institutions. So this is partly also known as federated analysis, but in the moment where you have this kind of federation during training, you also have this validation process, right? So by validating the model locally on the institution data, on the real clinical data, you can make sure that the model performs actually quite well in the real clinical settings. And this on a large scale. On a side note there, so these are the most dominant pro arguments for federated learning, but some minor ones that are actually also important. So in the moment where you donate to a data lake, you usually don't erase the data locally. I, as a data donor, give some information to a data lake. In this moment, the data is already duplicated. So then you have a centralized data lake. In the moment where a researcher wants to work on it, usually it's downloaded locally with legal contracts and everything, but it's downloaded again. So you have tripled the size of the medical data. And in the moment where you reach— where you tackle areas where the datasets are immense, This is quite significant. And in a federated learning setup, you don't have this. So you don't duplicate the storage requirements every time you want to work on a different project there or want to, and different researchers want to work on that. And another feature, as I said, so federated learning gives some privacy guarantees by construction, but also enables to, to add additional privacy-preserving mechanisms. So just to give you an example, because this is quite abstract, and there are a couple of rare diseases around the world that we were not able to tackle for the entire time. We are all aware of one disease that is not that rare anymore and has been quite significant in the last couple of months. Or for the last year now. So for example, for COVID, this is an example where federated learning can really make an impact. Luckily, a single hospital does not have the sufficient data to create a model. Luckily, because this otherwise would mean that this single hospital has tens of thousands of patients with COVID However, unfortunately, globally we have definitely enough data points or a lot of COVID patients that have been hospitalized. So in this case, it's maybe not a rare disease anymore. It's a— I could all work together there to develop a model. And this is something that was done last year. So for several months, really bringing together 20 different sites across the world. So from US to Japan to Canada, Thailand, UK, I think was in there, Brazil, Korea. The hospitals did not speak or the institution did not directly speak to each other, but they all work together towards the same goal. So it's not a detection model for COVID because this you can simply do with a test. But what they developed was they call it Exam Model. So it's a model that predicts the oxygen need of a COVID patient. And the institutions did not talk directly to each other, but they worked collaboratively on the same AI model with the help of federated learning. But this is just one example where federated learning was it addresses a quite important challenge. And in the literature, you can also find other really interesting use cases. So for example, you can use federated learning for electronic health records to identify clinically similar patients and predicting the hospitalization due to cardiac events, for example. For medical imaging, there were several publications in terms of whole brain segmentation or tumor segmentation in MRI. Or there are also publications in the direction of fMRI classification. So there's a lot of traction right now in terms of research, academic research. There are several consortia that really push the limits and really identify what is needed to make federated learning successful. So for example, the Trustworthy Federated Data Analysis project at DKFZ. There are federated learning consortia that aim for clinical impact. So for example, the HealthChain project in France, or there are projects that focus on the industrial research and translation. So for example, the Melody project, where even competing pharma industries come together and with the help of federated learning reduce the search space of a potential drug candidate. Which is in the interest of everybody. So the impact of federated learning in the medical field can be quite significant. However, the successful implementation of federated learning represents a paradigm shift from centralized data lake with a significant impact of the various stakeholders in this environment. So for example, for clinicians, this could mean they can rely on AI-based algorithms as a second reader. So this is applicable or true for a lot of AI-based systems. However, AI-based systems that were developed based on federated learning fashion are potentially less biased towards a potential, towards a demographic group or to a bias. So there might be higher sensitivity to rare cases because they simply have seen a more complete picture of the data distribution. However, this would require from a clinician— so if he wants to participate in a federated learning effort, it's necessarily to comply with some federated learning agreements. So every clinician has this kind of own way to annotate data, to report data. And this reporting and this annotation or saving of the data needs to be quite standardized for federated learning setup. So standardization in this context is important and might have an impact for the current reporting habits. So what about patients? So of course they are the ones that would benefit from accurate and unbiased AI. And also patients in need of medical attention in remote areas could benefit from the same high-quality AI-based algorithms that would be available in large hospitals. So usually large hospitals have more data, and if they would develop their own algorithms there, they would have a major advantage there. But if you all work together, also small hospital can participate in the federated learning effort and make sure that the distribution there locally is captured in an AI algorithm. Another important aspect for federated learning in patients is it might become easier to be a data donor because you always have the possibility to revoke the data access at any point of time to your local data because you never ever give your data away. And for researchers and AI developers, I think this is the most important topic for DKFZ. It would be an immense impact there if you would make it available for researchers to develop on clinically relevant topics, even if the dataset can't be published, if we can really leverage the real in-house, the real clinical setups and the datasets there and work on this. So we could have potentially a really vast collection of real-world data that we could develop the new algorithms on. However, as a researcher, the development or the innovation on this in this kind of setup is very tricky because Usually in a federated learning setup, you don't share data, right? So you can't even look at the data, how, what are the differences there? So it's very difficult then in this setup to identify why our particular case, why it failed for a particular case. So this makes it very difficult in terms of research. Hospital and practices remain in full control and possession of the patient data. So they have the traceability of how the data is accessed and which algorithms were trained on that. So instead of giving away the data and selling it, they could contribute the data for research or monetize it in a federated development. However, in this setup, hospitals need some local infrastructure to enable this federated setup. And also for manufacturers, it's very interesting because they really can have a continuous improvement of their machine learning-based system directly in-house in all the different demographics. So if you would like to formalize federated learning, we had a couple of discussions there for our positioning paper because there's a small, narrow definition of federated learning and a broad definition. And we decided for the broad definition. So generally speaking, you could formalize it as follows. So you have a collaboration pool of K participants, hospitals, or whatever, and they want to work together on a model with several parameters. So the objective overall is to minimize a global loss, which is a weighted combination of local losses. And the important part is that the local loss is computed on private data x_k. So the x_k is never shared, only model updates are shared. And the combination of these updates can be very different. So most of— many of the approaches that we see right now use a weighted average, but there are several research directions going on in this direction. So there are many flavors of federated learning, and you can think of many different combinations But what they have in common are two characteristics, is that the datasets are distributed and remain local. So they're not centralized. And you have a collaborative model. So you work towards the same goal. So either the entire model is developed collaboratively or part of the model. So in the down right corner, you see the current approach that's based on a centralized data lake. So they have several data donors that contribute the data to a data lake. And at some point when a researcher wants to develop an algorithm or innovate on something based on this data, this data is either shared with this researcher or the researchers can access a certain infrastructure in this data lake. But the compute is very isolated. So they, the, participants don't communicate with each other and they usually don't collaborate. So in federated learning, the workflow is very, very different. And the most common one you see in the upper left corner. So this is the client-server approach where you have an iterative process that exposes the traditional AI training. So it starts with, so in the center you have the server, so the aggregation server that takes care of the communication and distribution and managing of the federated learning setup. So in the first step, you have agreed on a model that you want to train on. You initialize the model, you send it to the local clients, they do the local calculation on the local data, they send the model updates to the server, the server aggregates the information and sends again the updated model. So this is a loop until convergence. And this is the most common one and known one. However, it's not the only one. So just to make you aware, there are also, for example, peer-to-peer approaches which are independent of a centralized server. So in this scenario, for example, you still agree on a model or a project that you want to work on. You initialize the models, the initial model is distributed everywhere, then the compute happens again locally, but instead of sending it to one central location, you just send it around every everywhere. So you broadcast your model updates and the aggregation happens locally. So this can either be, for example, synchronized, this can also be asynchronized. And it goes rather into the, I think in English it's hive knowledge, so, or Schwarmintelligenz in German. So both approaches are considered to be federated learning. It really depends on the way you set up your federated learning. Topology. And there are different architectures that you can think of. So the server-client approach is rather this hubs and spokes direction. You have this peer-to-peer approach, but you can also think about hierarchical approaches. So let's assume, for example, this, the left node is Germany, then you have a collaboration in Netherlands, then there's another one in UK, and this is all centralized again in a separate node. So the topology can be very diverse and you can even mix it. So have a, can have a hybrid hierarchical approach. The compute plan itself can be considered separately of that. And just to give you an example, so if we consider federated learning as datasets are distributed and we all work together towards a collaborative model, you could also take, so for example, the peer-to-peer approach and have a sequential compute plan. So starting from one node, then going to the other, then going to the other So it's a cyclic approach. So in this very general definition of federated learning, you could consider this as federated learning too. However, as I said, the most common one that we see out in the research papers and in industry currently is the client-server approach where you distribute the model at one certain time to several clients and then you collect them all. So the distribution here is parallel. And just to give you an idea how this could look like. So this is an example, not the federated learning algorithm, but just to give you an example how this could look like. So on the left side, you see the server-side aggregation that takes place. So initialize a model, you send it to the clients, then you wait from the server side for the local updates, and then you do the aggregation. So also in this case here, you just have the delta in terms of parameters and you normalize this according to the local dataset size. On the right side is what could be happening on the client side. So it seems to be a little bit complicated because in this scenario, they also included, so investigators how to initialize the momentum and everything. But in the end, what happens on the client side is you have your local data D, You have your local learning rates, loss function, and everything. You receive the model, you initialize everything, you run a couple of iterations of local training, not until convergence, just a couple of local iterations. And then you either send the raw information, the raw updates to the server, or you can add some privacy-preserving mechanisms here. So privacy-preserving mechanisms could be, for example, adding noise, truncate the information, and so on. So this is from the algorithm perspective. However, if you really want to go to real-world implementation, you have to also think about other things. So if you take one step up and think about how this communication could happen, for example, for a client-server, there are several more processes happening in the background if you really want to go to a real-world scenario like the COVID example. So this is an example how you could enable the communication between clients. So for example, do you have a pool of clients? At a certain point, the client said, okay, I could contribute to a federated learning engagement right now. So I submit the intention to participate in a federated learning round. The server has the overview of all clients, sees that how many clients want to participate currently. Evaluates the client if he is eligible or not, if he's authorized or not, gives him a token in case he is or an error if he's not, sends the model and waits until the server receives a lot of updates. And for all the updates, again, has to evaluate if it's a valuable contribution or not. So there's a lot of, from the software architecture side, there's a lot of more happening than you would see in an algorithm site. So despite the advantages of federated learning, it does not solve all the issues that are inherent to learning on medical data. So a successful model training still depends on factors like data quality, bias, and standardization, like for all AI developments. So these issues have to be solved both for federated and for non-federated learning efforts. With appropriate measures. So careful study design, common protocols for data acquisition, structured reporting, sophisticated methods for discovering bias and hidden stratification, for example. But federated learning gives some privacy guarantees by construction, as I said, as the data is never exposed and the model updates are usually aggregated. So, but federated learning is different from traditional training insofar that the training process is exposed. So the training process itself is exposed to multiple parties. So you have this additional communication in between. So in this communication, it increases the risk of leakage via reverse engineering. For example, if adversaries can observe model changes over time or can observe specific model updates or manipulate the model. So there are different, depending on your consortium where you work in, if it's trusted one or not trusted one, you might want to have different privacy preserving mechanisms. So for example, limiting the granularity of updates or adding noise or differential privacy. Level of security that federated learning can offer really exceeds today's current commercially available machine learning models. So the nice part about federated learning is although this data, this training process is exposed, it allows you to add additional privacy measures. However, you have to think about every privacy measure comes at a different cost. So for example, if you map the information to a different mathematical space, like in homography, you can do this, this is quite safe but it comes with significant computational costs. So depending on the federation that you work in, depending on the level of trust that you have between each other, different privacy or countermeasures have to be considered. Another peculiarity for federated learning in the medical space is that our medical data is particularly diverse. So this is not only because of the variety of modalities and the dimensionality and characteristics, but also due to the specific protocols like the acquisition differences, the brand of the medical device, or even the preferences of the clinical doctor in terms of contrast, for example. So federated learning may help to address certain sources of bias by potentially increasing the diversity of data sources that you can see for a training round. But the inhomogeneous data distribution poses a challenge for federated learning algorithms. So usually we assume an identically and ident— so independently and identically distributed data distribution, so IID. And this you have usually in this huge consortia where you have millions of participants for federated learning. In our case, we usually only have a couple of participants. So this poses a challenge. However, initial results from research shows that it's still doable if you have certain countermeasures. And also in this COVID, this global real-world COVID experiment, we did not encounter major issues there. And as for all safety-critical applications, the reproducibility of a system is important for federated learning healthcare. So in contrast to centralized data, you run the training process on various different participants. And if you want to certify an AI algorithm and you really want to bring it to clinical practice, you have to show on which data it was trained on. So you have to keep track on which data was accessed, where, and how it was used. And unlike large-scale federated learning on edge devices, we are in the prime position in the medical field usually that we have better computational resources. And usually, so for example, with the phone, you can lose the connection. Usually a hospital does not walk away. So usually have a quite stable network connection. Which is great because in the medical field we have quite large models. So the models that are trained on phones are usually quite small. Our field, like if you really go to 3D convolutions, they can be very immense and moving these models from one side to the other can be quite significant. So you need significant compute locally and you have to have a stable connection to them. So one core requirement for a successful federated learning implementation is a common federated learning protocol on the other side. This I forgot to mention for the data heterogeneity. So you have to agree beforehand on your common goal and also on your metrics, because a global solution might not be the optimal local solution. So you have to consider that when you set up your federated learning protocol. And we see a lot of traction. I'm just checking the time if I'm still here. I have to hurry up. So we see a lot of adoption in the research field and also in the industry. And it gained a lot of traction, especially for the healthcare domain. So last year at MICAI, we had the first workshop on distributed and collaborative learning, which was quite successful. There are many, many research questions that can be tackled. So I encourage all people at DKFZ, if you're interested in the domain, look at it and work on it. I would love to see more research in this direction. And there are several tools out there that were really good. One of them is, for example, Clara FL. I won't go more detail there, but just to mention, this was, for example, used in this COVID study. So for the medical domain, there are several applications that are particularly interesting. As I said, federated learning probably won't replace all AI developments that we have currently. And why? Because you add an additional complexity to the training algorithm. However, for certain domains, it really makes sense. So for medical imaging, you work with sensitive data. This data can't necessarily be centralized. Genomics is a field that you probably don't want to share the full sequence usually on the dataset. Digital pathology is interesting because the datasets that usually have there are immense. So moving the dataset to a centralized location and then again and again might be cumbersome. So in this scenario, you might want to develop federated learning to bring together these datasets, but also drug discovery, as I said before, and other industries like smart manufacturing, we know it from self-driving cars, also energy and there are many, or finance, for example, there are many, many different fields where federated learning really starts getting an impact now. So some key takeaways. So federated learning is a new learning paradigm, super, super interesting, that enables collaborative and secure training. So several parties can work together. It's distributed and flexible. So clients, participants can join, die in between, or rejoin at any point of time. So it's not, you have a lot of flexibility in this sense. And it can enable innovative ideas that have not been possible before, and it can be a continuous improvement in terms of model development. And so with that, I hope I'm still a little bit in time. I would like to conclude and thank you for your attention.

Yeah, thank you very much, Nicola. This was a great, fantastic talk. And I think we have a lot of questions right now. So for further questions, you can use the chat function or raise your hand. And Lena will start with the first question, please.

Thank you very much, Nicola. You mentioned the term unbiased AI. And I think currently—

sorry—

I think currently we see a lot of problems when merging datasets in general, not only in the federated learning context. So I think one prominent example is the Kaggle challenges where they took, for COVID detection, where they took data from healthy infants and merged that with data from sick adults, basically, and then use this for training models, and then obviously there's a very high risk of the confounders biasing the whole result. So you actually don't learn the manifestation of the disease, but you learn to distinguish infants from adults, right? And I think, I mean, this is just a random example. I think we see this a lot now, and it's overlooked. So I guess my question is, when we leave all the data with the site and we don't have a lot of insights anymore? Don't we have to look much more into how we manage the metadata and how we can make sure that we don't have a selection bias, that we don't have confounders in the data, etc.? And I mean, do you have a strategy on this or any insights on that?

Very good question. I think there is no global answer for that. It really is application dependent. So one of the core components for successful AI development is that You pose the question the right way, as you said. And for this, you have to really create the dataset in a way that it's addressed, like it's a representation of the problem that you want to solve and how it would occur in a real clinical setup. So if you have a dataset that's only with infants and then you have a dataset that is completely different and you can't guarantee that the model actually detects the things that you want to detect, then you have a problem. So federated learning can be a help in that in terms of giving really a broader view on the overall distribution, making more data available. In some scenarios, you actually want to have a bias. So one of the good examples that we had in the discussion recently is for example, for self-driving cars. So there is a component that is common for all the scenarios globally. So you want to detect a sign. Signs are everywhere in the world. So you drive and you detect a sign and you want to identify what it is. However, sometimes you want to have a local bias because our signs in Germany are very different from the signs in US, for example. So depending on the application, you actually want to have a local bias on that, otherwise it does not make sense. So I would say the problem of creating So the question of posing the problem correctly for federated learning is the same as for general AI development. And it's not solved by the federated learning paradigm itself. What federated learning solves is the data access governance. It does not solve the bias. It can be a help in that in terms of making more data available, but it does not solve all the issues. I talked a lot, sorry.

Okay, then I will start with the first question from the chat. So it was a question to the beginning of your talk, and this was from Christian Niklas. He asked, from the very front-end perspective of this, how do we broker the benefit of this great joint effort to the local instances? Specifically data warehousing initiatives, clinicians? That was the first question. And the second question, are there common standards, strategies on how to deploy such models to different localities to compensate for diverse data models and structures?

Difficult question. So from a front-end perspective, it has to be as easy as possible. So in this federated learning effort, depending on who you work with, you either want to enable that this local person can also modify the model itself, but usually in the federated learning effort, what you— you don't want this. So you have to make it as easy as possible to make really a kind of very standardized training locally, that everything is already set, that you have this— the model that you receive, the learning rate, the losses, everything that a clinician does not care about that. He does not understand what it means, but he still wants to participate in this federated learning effort. So it has to be a very standardized way of training, not that flexible to make it successful.

And the second question was, are there common standards or strategies on how to apply such models to different localities?

Currently, I'm not aware of a standard for federated learning, to be honest. It's a very new learning paradigm and there is not a common standard for that or strategy. Depending on the federation, it's very, very different.

Okay, thanks for this. Then we have another comment from Tim Retsch. He thanks for the interesting talk and wants to state that in a federal, federated setting, the curate and label step remain at each hospital. The label step is is already quite error-prone in centralized settings. For example, consistency of annotations, different understanding of the labeling instructions. And he asks, how is the label step handled in the federated setting? And the second question is a step through ahead. How is the annotation quality evaluated for each distinct hospital?

That's a good question. And this is actually one of the difficulties of federated learning because you can't enforce it if it's not integrated in the training step itself. So annotation can vary a lot in a centralized setting and it varies a lot in a decentralized setting. So what we did in our global efforts usually is checking with them manually if everything is set in place correctly. So having a control experiment locally if everything is in the right format. So for example, annotation can be— if you annotate everything in ones or zeros, in the end, this makes a huge difference and can actually screw the overall results. So you really have to enforce both in the centralized setting as well as in the federated learning setup that the annotations are consistent. So the data quality and the annotation quality is a core requirement for successful AI training in a federated setup as well as non-federated setup. What you can think about is prior to a federated training, you can have a federated annotation. However, I am not aware of a very successful integration of this at this point. So it is an issue, definitely.

Okay, so we have another question. How effectively the individual model integrity check can be done before the model aggregation at server?

Depends on how you want to do this. So usually in the federated learning setup, what you have usually is you don't check for the quality of the model updates because it's very application dependent and it's very difficult to measure how good a model update is even in a centralized setting. So this check what I showed before in this high-level implementation is rather is this client eligible right now for this contribution, or do we have to exclude him for different reasons? So was he working on a previous model, for example? The usual aggregation step currently that happened for most of the approaches that I'm aware of is the simple sum of contributions without checking of like a contribution that is not positive for this scenario. There are approaches that are called robust model aggregation that check for outliers in terms of model updates. So everybody sends a model update and then there's outliers. So it could be that this model update that is an outlier was trained on annotation that is inverse, for example. There are approaches, however, it has to be considered with care because rare diseases, rare pathologies can be outliers and there can be very important information for model update. So it's still an open question what is the most reasonable approach for that. I would love to see some research in that.

Okay, thanks for your answer. Jonas Bohn also thanks for your talk and states that he sees the benefit of the federated learning. And he asked, but I guess one of the big problems are dealing with data heterogeneity, different modalities, for example, different parameters, and integrating this data from different institutions. What could be the solution if the institutions do not want to share their data? How could federated learning work in this case?

So in the end, so very good question. So 2 years ago, it was said that non-IID is a problem for federated learning. Now there's several approaches to counteract for that. So in terms of aggregation method, for example, there are some ways to counteract for that. So for example, the sum that you saw, it's normalized to the local dataset size. One of the approaches that we do, for example, is the data loading locally, for example, can be very different. The preprocessing locally can be very different for the clients. In the end, what is important is that you can make the data digestible for the model that you want to train. So even if the participating clients have very different modalities and very different setups locally, you can implement other preprocessing steps there, for example. And this is a major difference to a centralized training because they usually have have the same preprocessing steps. So you can create diversity in that and bring still this information to the centralized location. And it also depends on the direction that you want to solve. So if you want to solve something particularly for MRI, a segmentation algorithm for MRI, it doesn't make sense to bring in CT. However, if you want to develop a model that is able to handle everything, it should be also included in the training set. So modifications locally in terms of data preprocessing, if you really want to have a generalized model for everything.

Thank you. And now we have a last question from Tom Ricks. Does federated learning always happen in parallel, or are there also asynchronous versions where updates from hospitals arrive at different times on the aggregation server?

Absolutely. So there are two different main approaches. The one is synchronous, the other one is asynchronous. And in the real world scenario, you never really have synchronous approach or you really slow down the process. So for example, in this global COVID study that we did, Korea was— or like Japan was not at the same time as UK, that model update, because the processing locally was in a different time. So different approaches that you can have there. So either you have really make it synchronous. So you say, let's have 100 clients everywhere, but we wait until we have at least 80 submissions. And then we do the aggregation and ignore the other 20. Because for any reason, it can always be happening that you don't have the model updates directly in this time. Or you can really say, okay, we want to have all the contributions, we wait for all the submissions, but then it slows down So this asynchronous and synchronous, they are both there. And also, if you remember this compute plan, you can implement it in very different way. You can, so you can either do it all parallel or you can do it for certain parts in sequence, for example.

Okay, thank you. Then we have a question from Marco Nolden. You can ask directly if you want.

Yeah, sure. I mean, it's very late, but so I mean, it's so the whole thing is so great. And but I've seen that, I mean, explaining this aspect of, okay, the models are aggregated and they are not, they are safe to share and stuff. I mean, especially if you directly really interact with hospitals, there are sometimes like non-technical people are not so deep into this and The question is, I think that there will be still the challenge to explain this all to them and they are not deep learning experts and they say, okay, but this is like a 10 gigabyte file and it contains just numbers and how do I know what's in there? And so I would be interested what you think about this and if this is something you're also addressing right now, tools for introspection or explainability of models and things like that.

Definitely. So when we talk to clinicians, they don't understand what it means, a model update. So in this setup, we also work with institutions that don't necessarily develop deep learning themselves. So they don't even know what deep learning training means. What always helps is that saying that the data remains locally. So the data itself is never shared. So this is always a plus point. In the end, what is shared is statistical information. And usually clinicians can understand what statistical information means. So this is usually a language that they accept. And one other component is that you can always say, okay, I always explain the aggregation in terms of numbers. If you put numbers together, it's more difficult to identify the individual contributions of 5 5 10, but also 3 3 2 4 is also 10. So especially in this client-server approach where you don't know how many clients actually contributed to the model update, it's very difficult to identify the individual number. And the third part that is an argument for them is, okay, don't share the 3 and 5 with me, share with me 2.8, add noise. And then there you—

sometimes go down on this. Yeah, but the clinicians, I've— so from my experience, it was not the clinicians who are having concerns. It's really like then the people who are in charge of, for example, you have to technically at some point ship the model, transfer it from the clinical system. And at this point, you also interact with other people. And I mean, you can tell them you're doing this, like adding some differential privacy noise or whatever. But it's the question of, okay, but still you're downloading a 10-gigabyte file and you're sending it somewhere, and if you're taking data from our clinical system in this big amount and you're sending it somewhere and I can't even have a look at it. So it's just, just an observation. And I mean, I know there's TFDA also where we are contributing and things like that, but I think for the practical appliances will definitely still be something that has to be solved at some point because especially in Germany I've heard this a lot of times and it's definitely a major challenge.

Also has to be considered is you sometimes have to have an open port that allows sending and receiving, for example. Hospitals have to give compute resources for that and hospitals don't always have compute resources at the side for that. So it has some implications, but without it, it's not possible.

Yeah, sure. I mean, it's great. I love the approach.

No doubt about that. It's just— but it's a very important point, and that's why we had this positioning paper, because there's a lot of hype about federated learning, and this positioning paper really tried to highlight the benefits as well as the challenges. And this, what you mentioned, is definitely a challenge. For real implementation of federated learning.

Yeah, so thanks for this, Nicola, and also for a lot of questions which were also posted in the chat. We don't have time at the moment to answer all the questions, but if you have more questions, please send them to me to my email address and I would forward them to Nicola. Yeah, we are at the end of the data science seminar today. Thank you for listening and watching. The video will be available on our website later on. And yeah, thank you very much, Nicola. Thank you, all of you. And we'll see us in 2 weeks.

Goodbye, everyone. Thanks for having me. Bye-bye.