Scientist Stories: Stefanie Speidel, Robot-Assisted Surgery

Welcome to the Data Science Seminar. It's my great pleasure to welcome our speaker today, Stephanie Speidel, Head of Translational Surgical Oncology at the NCT Dresden, and she will talk about ways to quantify surgical expertise. As always, you will have the opportunity to ask plenty of questions by using the chat function after the talk. But before we start, I hand over to Lena Meyerhein for some, some introducing words.

Yeah, it's my great pleasure also to have Steffi here today. Steffi is a professor for translational surgical oncology at the National Center for Tumor Diseases, NCT, Dresden, our partner institute, since 2017 actually. She first received her PhD from KIT, Karlsruher Institute of Technology, in 2009, and this was actually in the research training group Intelligent Surgery. So Steffi actually worked on surgical data science and artificial intelligence long before it became a hype. I really want to explicitly mention this because nowadays everybody claims to do AI, but Steffi for me really is one of the pioneers in the field. Yeah, she also is general chair of IPCAI. This is the most important computer-assisted surgery conference and general chair of Endovis, which was recently acknowledged as the most important platform for challenges and data in the field of surgical data science. So if you want to talk about surgical data science, really Steffi is the go-to expert worldwide, I would say. And yeah, I'm very happy to welcome her today.

Yeah, thanks a lot for this very kind introduction. And I'm happy to have the opportunity to present our research today. I will share my screen now. I hope you can see everything. So I will show today a bit about our research regarding context-aware assistance. And the vision here is to quantify surgical experience. But first, I want to take you on a virtual tour at the NCT in Dresden. So you can see here a flight above the University Clinic campus in Dresden. And here is the new building at the NCT Dresden site. And similar to NCT Heidelberg, it combines patient therapy and research under one roof. And what I think is really unique in Dresden is the experimental research platform we have. So we have, for example, an experimental OR, you can see here, only for research with a lot of robotic robotic devices, sensors which are connected, where we pursue our research in close cooperation with surgical partners. And so my group is an interdisciplinary group. I'm a computer scientist by training. So I have a lot of computer scientists and electrical engineers, but also surgeons in my group. And we work on different aspects of computer and robot-assisted surgery. You will see some examples during the talk today. But first, I will start with some numbers that you might not be aware of. So according to a recent Lancet comment, postoperative complications are amongst the leading causes of death worldwide. And I believe that cognitive assistance function can bridge the gap between data science, robotics, and sensors, and to pave the way for next generation robotic surgery. So our goal is based on this to improve patient care and outcome. So, and this leads me to the term context-aware assistance. What what do we mean with it? You can see here the typical circle. So we start with perceiving sensor data in the OR. This is mostly laparoscopic video because it's easy to acquire, but we have also additional device data, for example. Then we do a knowledge-based interpretation and based on the context of surgery, we want to provide a real-time assistance because we are in an intraoperative setting. Real-time is an important requirement we have. So surgical performance in general is quite challenging, and it's not only about handling instruments and to have the technical skill, but there is also a lot of cognitive skills involved. So it's about information analysis, making fast decisions, doing the action in this very dynamic and complex progress. And as you have to possess different skills, for example, to make fast decisions, to have a certain kind of situation awareness, and it's not only the surgeon, but it's a whole team in the OR which needs to be taken into account. And so it's not a surprise that the outcome depends heavily on the surgical experience. And this is also shown in several studies. For example, here, a famous study where they have exemplarily two difficult cancer surgeries investigated, esophagectomy and pancreatic resection. And you see here, the more often this kind of surgery is performed in a specific center, the mortality decreases. So overall, the challenge here is how can we quantify this experience we have in the head of expert surgeons and make it accessible to machines in order to democratize surgical skills and knowledge. And this leads me to the Surgical Data Science Initiative, which Lena and Pierre and myself co-founded in 2016, and the goal is to improve the quality of interventional healthcare through capturing, organizing, analyzing, and modeling of data. So there were already two workshops, and as a result, we had, yeah, one paper that defined the term and the challenges, and the actual paper which is under review focuses on clinical translation. Because right now there are no success stories in surgical data science where we can really prove that using AI methods during surgery improves patient outcome. And I think this is the next step we have to take to really bring these methods into clinical routine. And let me show you here the future on that image. So we imagine the OR to be an interconnected environment where we can access different sensor data and combine it with domain knowledge, which for example is the expertise of the surgeon, case knowledge. So for example, procedures specific to that kind of surgery, additional data and patient individual data we have before and during surgery. Regarding the devices, we have the vision that the OR of the future will be a sensor environment. For example, we have different devices where we can acquire the data. And as sensors, I mean also a robotic system where we can have access to, for example, the kinematic data or a navigation system. So sensors are defined in several ways and it's important important that these are not single devices, but they are connected in this environment and we have access to it. And regarding domain knowledge, this is factual knowledge, for example, which includes workflows. Here in this example, you can see the workflow of a cholecystectomy with the different phases and triggering activities. But it Factual knowledge is also about clinical guidelines and hospital-specific standards. So this is, yeah, knowledge we have in a certain kind of structured way accessible. And then there is the practical knowledge, and this is the so-called experience of an expert. And the challenge is how can we quantify this experience? And one way to do this is is to have a case base to record lots of surgeries, have not only the video data but also additional data we can acquire during and before surgery, and to have a kind of case expert repository for a specific kind of surgery. But the challenge here is that this data needs to be annotated. And this is right now one of the major bottlenecks in surgical data science. I will show you a bit more about it later. So I have today, I have brought like 4 different examples regarding AI-assisted surgery. So the first example is in the context of data generation. And As I already said, a key requirement for cognitive systems is labeled data. And I think future advances in the field, they will heavily depend on efforts to create, annotate, and share data. The second example is in the context of data analysis. More specifically, I show here an example regarding soft tissue registration. Then I have an example in the context of situation-dependent assistance to enable an intuitive human-machine interaction and to augment surgical performance in the end. And another example is in the context of surgical training because surgery is considered as one of the most complex activities and If you have poor skill, it harms patients in the end. And the methods I show you today, they can also be applied to a surgical training scenario. So let's start with data generation. And this is a very important challenge right now in surgical data science. And you can see here 3 examples of annotating images or videos in laparoscopic surgery. So the first one is annotation of, of instruments. And I think this is an easy task. I could also annotate instruments or any student who is not a medical student can do that. But it's getting more challenging if you have to annotate events or as here in the middle, this is a bleeding event. So the question is, what is defined as a bleeding? And here you have to know the surgical intent because not every bleeding is a bleeding that you to, you need to take action on it. And the other example here is to define a cutting plane, and this requires a lot of experience. So you see, when it comes to annotation, we need experts, and this is exactly the expertise we want to quantify. And Medical experts, they don't have time in clinical routine to do such tasks. And so the question is, how can we bridge this? Because for our deep learning methods, we need large data sources also for evaluation and benchmarking such systems, but in reality, we don't have them. So one way to overcome this is to use simulated data. And you can see here on the top a simple laparoscopic simulation in liver surgery. But if you render an image based on that, you see it's not looking realistic. I mean, the advantage is that because it's based on the simulation, we exactly know the labels. So where are the instruments? Where are different tissue types? What action is being performed? And so on. So we used recent advances in unpaired image-to-image translation to generate realistic synthetic images that look like real laparoscopic ones, which can be used then for training our networks. And here I'll show briefly the idea. So we have simulated 3D scene and we use a simple rendering approach with a simple texture and light model. And this leads to this this image which does not look realistic. But then we use an unpaired cycle consistent image translation, for example, MUNIT, which enhances the realism. And you can see here the result on the right side. And we also incorporate an additional structure similarity loss and show that it helps to preserve the image content and the structure despite having large changes in camera viewpoint. So the data and code is available. So we have different datasets with over 100,000 images in different styles, but we also provide the labels. For example, if you work on semantic segmentation in laparoscopic surgery, or if you work on stereo reconstruction, you also get depth maps and so on. So it can be used, for example, for pre-training the networks. So these are single images, but I mean, in laparoscopic surgery we usually have videos. So the next challenge is to create synthetic videos. The problem is when we use this approach I just showed you, it's not view consistent and the texture changes over time because this approach is not designed for long-term consistency. So if an object, for example, leaves the scene or the field of view and a consistent rendering cannot be ensured when it returns because the previous frame contains no information regarding this object's appearance. So I will show you now a video and you see the background is flickering. We have vessels showing up and in the next frame they are gone. I'll try to restart the video. You will see later. Other examples. So we thought, how can we preserve this long-term view consistency? And because we use a simulation, we have the underlying geometry and the camera trajectories which can be used to ensure such a consistent translation between the frames. So we used a neural rendering approach to learn the representation of the scene textures in 3D space. And this neural renderer learns the global texture representation. So in this way, the information can be stored in a 3D texture space and can be used by the translation module from different viewpoints. So the model can learn the position of certain details, for example, vessels, and render them consistently. And you can see here that in this example, using the neural rendering, we preserve this long-term consistency. Let me show you the video again. Okay, let's take a more detailed look. So the core of the method is still unpaired. It's a cycle-consistent image translation, but the new model is this global learnable texture of the entire scenes. And to render the current view, we use a simple rendering and project the texture features into the image plane. And this serves as input to our translation model. And since the projection is differentiable, this can be learned end-to-end, and the model can store the information on texture in 3D space. So for example, the location of vessels or the color of the liver. And to enforce view consistency, we render a second view and then we warp the first view into the second and enforce thereby a pixel-wise consistency. An important thing to note is that the light source in laparoscopic surgery constantly moves, so we also provide a lighting invariant loss and this enforces a consistent U but also allows for varying brightnesses. And here no paired data is required. We only need simulated scenes and an unconstrained collection of real images. And here you can see some examples. So on the right, our approach on different scenes. So this is the approach I showed you we used for the single images. And you see that by using a neural rendering approach, the view consistency is preserved. In, in these examples. And fine details, for example, like vessels, are rendered consistently across different viewpoints, and they were not manually modeled in the simulated domain. Okay, then let's go to the second example. And here I show you a recent approach we implemented for soft tissue registration. So the idea in soft of tissue registration is that we have preoperative patient data, for example, CT data. We can generate a 3D model out of it. Here in this example, it's a 3D model of the liver. And you see, for example, in yellow, the tumor inside the liver. And, but during laparoscopic surgery, the surgeon looks at this image. So you see, you need an experience to match this preoperative data to this surgical scene. And normally the surgeon creates a mental model and if he or she is experienced, there won't be any problems. But if not, it's difficult to do this match. So we thought, how can we provide an augmented reality visualization by using these different data sources? And if you look now at the intraoperative scene, it's, yeah, from a computer vision perspective, quite challenging because it's extremely noisy. You have sparse correspondences because, I mean, you have the intraoperative scene and this needs to be matched to the preoperative model. And what is invisible are the boundaries. So the boundary conditions, the forces that are applied on the tissue and so on. And because in the end we need to predict the deformation to adapt our augmented reality visualization to the soft tissue deformation. And a classical way to do this is to use a finite element method to model the mechanical behavior of the tissue. But this is really difficult to parametrize and to parallelize. And also there is very sparse literature regarding the boundary conditions. So we thought about coming up with a data-driven biomechanical model. So is it possible to learn this tissue deformation information and also to learn to match the surfaces. And in the end, also to learn to deal with different input modalities. And the principle, the general idea is to use a CNN for the registration. So we have our intraoperative data, we reconstruct a surface based on the stereo images we I will show how it is done later. And then we have our preoperative model here in gray based on the CT images. And here we have the geometry of both data sources and we feed this into our volume-to-surface registration network. And what we get in the end is the displacement field we can use to register the pre- and intraoperative data. So important to note is that it works directly on the geometry. So we don't need to solve the correspondence problem between intra and preoperative images. And this is also a nice feature. It trains on synthetic data. So we use for training data organ-like meshes, not exactly a liver, but yeah, a lot of simulations with organ-like meshes. And the nice thing about it, it adapts directly to any new patient geometry, including the liver. So we don't have to retrain the network before surgery for a new patient. And so let's take a look at the synthetic training data. So we generate random meshes and simulated random deformations with random boundary conditions using a standard FEM simulator.. And we also extracted the partial surface. This is our intraoperative surface we reconstruct. This simulates this. And to pass this data to our network, we encode the geometry-assigned distance fields. And the displacement field is also extracted. And the distance fields are the input to our network, and the displacement field is the output, which the network should predict. And this is repeated for 80,000 measures, and this makes up our dataset. So we did a lot of experiments starting with simulation data and in silico, so where we showed that the prediction is usually plausible if you see enough surface. So at least we need 20% of the surface so that the registration is robust. But there are also exceptions when it does not work. For example, if the rotation is too large or if the surface is too small. Then we also did an inhuman CT breathing motion experiment. And here we have CT data as input. And we could reduce the displacement error to 5.7 millimeters. And we also did a qualitative evaluation. So to show that the method works in general in laparoscopic surgery, here we don't have the ground truth to compare with. And it works, but one open issue right now is still the large amount of noise and the changes in topology. For example, For example, in liver surgery, we have a gap between the left and the right lobe. And this is not visible in the preoperative model. It's only visible in the intraoperative scene. And not to, yeah. And if the surgeon is cutting the tissue, this is also a change in topology we take not into account right now. So let's go to the intraoperative model. And here, as I already said, we use the stereoscopic video, and this is straightforward approach where we solve the correspondence problem and then use a triangulation to reconstruct the surface in real time. I won't go into the details about that. I show you some examples, and we do not only reconstruct the surface of the scene, but also localize the camera within our scene. So this is a SLAM approach, simultaneous localization and mapping, which you can see here with intraoperative data. So we are right now close to the first in-human trials. Hopefully they will take place next month and we build up an intraoperative pipeline you can see here. This is still on a phantom, but it shows you the, the general steps of the pipeline. So first of all, we acquire the surface of the organ and localize the camera within the surface because we need the position of the camera in order to visualize in the end with augmented reality the risk and target structure. Then we perform a rigid alignment. So this is the first step, and this is done in an interactive manner. So we use here the knowledge of the surgeon, but it's a really easy-to-use tool, only 3 clicks, and it doesn't need to be accurate clicks, and we are able to have then a rigid alignment. And as you can see now, there is still a deformation, so the rigid alignment cannot compensate for this deformation. So we use our volume-to-surface network continuously now to perform a non-rigid alignment. And in the end, you can see here now what should be our end result is the visualization of risk and target structure on the right side at the phantom, but on the left side also on intraoperative data to show that it in general works also on real patient data. Okay, so the next topic I want to talk about is context awareness. And I said at the beginning, we want to show the right information at the right time. So we need to know in what phase is the surgery and what is the assistance need regarding that phase. And here you can see an example where we use the video data and additional device data we gathered in the sensor OR to predict or to recognize the actual phase we are in. You see here this video on the top in blue, you see the ground truth regarding the phases and on the bottom the recognized phase and it's quite quite accurate regarding face recognition. And the application here is the remaining time prediction. And this is, for example, useful for overall OR workflow management. And here we use a Bayesian neural network to account also for uncertainties with our prediction. But in the end, we want to predict complications and to warn the surgeons, for example, if we are in a phase where a possible complication can occur regarding the intraoperative course and the data. But for predicting complications, you need to have the data for it. And this is actually a major bottleneck. So we do not only want to recognize what's going on, we also want to anticipate what's coming next. For example, what events are going to happen or what instruments are used to enable an intuitive human-machine interaction. Anticipation of instruments, for example, can be used for instrument preparation if you have a scrub nurse robot, for example, or also semi-automation of robotic tasks. And here we propose a novel task of anticipation instrument usage from video, and we formulated it as a joint regression and classification objective. So predicting the time until the occurrence and whether or not an instrument will appear within a defined horizon. And we don't make assumptions about the timing of actions. And for training, we only require labels for events we want to anticipate, so in this case instruments. And for inference, we need only the raw video data. And here we use a Bayesian deep learning framework to also take uncertainty into account associated with future events. And we showed that we can identify identify events which trigger the usage of instruments through uncertainty quantification. So in this example, using the scissor, we could show that the network identifies the placing of a clip as trigger event for cutting. And you see here that for the prediction, the uncertainty decreases. Another example I want to show you is in the context of the Cobot project. And here we recognize the phase of surgery and depending on the phase we provide an assistance. And what kind of assistance can this be? For example, here I will pause the video so you have the chance to see it, is the prediction of the first where you put the first peritoneal incision. And yeah, here it's— and this is at the beginning of surgery. This is often quite challenging because you have no visual cues and a lot of experience is required. So we gather data from expert surgeons where they put this first incision and trained a network to predict this incision line. Another example is in another phase of surgery, it's also an important phase in rectal laparoscopic surgery, is the cutting in the right plane. This is so-called the TME plane. And here it's also really important to be in the right plane. Let me show you where you can see it a bit better. So for example, here you can see that the plane is visualized and where to cut. So this is also important information depending on the phase of surgery. So here, if I pause the video, you can see it better. And yeah, so our vision is to bring such systems into the clinic and to evaluate, for example, pitfalls when using AI systems and in the end also evaluate if the patient outcome is better when using such systems. And I think we are at the beginning, but we need to focus on this translation. So here another example in our experimental OR just to visualize how these different modules can be integrated and interact together. So here a laparoscope is mounted on a robot arm. We trained the laparoscope to have the right view depending on the phase of surgery and to enable a human-machine interaction. So this, the laparoscope guidance is automated.. And depending on the phase, for example, certain assistance is given, in this case, the visualization of risk and target structures. So this is just an example how these different models can work together. So in the end, I want to talk a bit about sensor-based surgical training. And here the aim is to improve surgical skill by using data-driven methods. So normally for training laparoscopic skills, you would use such a box trainer as can be seen here in that example, and an expert surgeon gives you feedback. But this is difficult and does not scale in clinical routine. And so our idea was here to analyze the performance based on multiple sensor signals we captured during training, not only from beginners, but also from experts, to quantify the expert experience and and use it to improve the learning curve. For example, by giving targeted feedback. And we created a large dataset which will be published soon for different laparoscopic training tasks. So for example, for peg transfer, as you can see here on the left, or for cutting a balloon which simulates a gallbladder. And we annotated the data. So this is from novices from courses at the University of Dresden for minimally invasive surgery courses where we recorded the students at different skill levels during the semester, but also expert surgeons. And based on that data, we can learn to estimate the performance. And these are standard scores we use, for example, OSETs, which is usually applied for assessment of surgical performance. Performance. And here is an example. This is not based on our dataset. This is based on the so-called ChixSource dataset. It's an open dataset for robotic assisted training. And here we used a 3D CNN to extract spatial-temporal features in order to predict the skill level and class. The problem with 3D CNNs is that they can process only a small number of videos. Frames. So the challenge is how can we use 3D CNNs although we have long videos for surgical training. And so to deal with these long sequences, we split the video into smaller snippets. So now we can use the 3D CNN and classify each snippet as novice, intermediate, or expert. Expert, and then in the end we can average the results and obtain overall classification score for the complete video, which we use then to estimate the skill class for the participant. So now the question is how to train the 3D-CNN. So basically you would put the video snippet through the 3D-CNN and compare the obtained class score with your ground truth and then backpropagate the error. However, we have only one label for the whole video and we don't have a label for every single snippet. And we also saw that this does not mean that every snippet is performed in a bad way because a beginner can also have in some snippets an expert performance. So there might be a certain overlap. And to get around this, we extended the 3D-CNN into a temporal segment network and now we look at several multiple snippets sampled throughout the video, and we use a Siamese architecture to classify all of the video snippets at once. And then you get an overall classification result in the end. So I want also to give an outlook of how we can have feedback because in the end we don't not only want to estimate the skill level but also to give targeted feedback. And here we are working together with the mechanical engineers at our excellence cluster, and they designed a haptic instrument to give haptic feedback. And in a training scenario right now, we give feedback to beginners if an instrument is out of view, because for beginners it can be really challenging to keep the instrument inside the camera view. And yeah, Here right now we evaluate different means of giving such haptic feedback for laparoscopic training. Another way could be to visualize the expert demonstration, for example, the trajectory an expert would use. So there are certain ways and how to give targeted feedback in the end. So In the end, I want to highlight some remaining challenges. I mean, there are a lot, still a lot of challenges regarding translation of AI methods into the clinic. And one challenge is data. So it's not only the access to data, but also annotated data. So where do we get enough labeled data that can be used for the method? Is the data representative for the task? And what about data security, patient protection? And so on. So yeah, this should be tackled by several complementary approaches. For example, the reduction of annotation effort via active learning or crowdsourcing, the generation of synthetic datasets or methods such as self-supervised learning, just to give you an example. So on the method side, we have weakly labeled data or unlabeled data. We need methods that can deal with that. We need methods that can ensure generalization. So models trained on data from Dresden, for example, they may not generalize to surgeries in Heidelberg. And so there is variability. So we need data from different centers. And then we have heterogeneous data sources we need to deal with. And yeah, another challenge are medical devices. I mean, I showed you the sensor, but we can only acquire the sensor signals in the OR if we have access to the devices and interfaces. And that is not easy because normally these systems are closed. And yeah, we need to have industry cooperation to get access to these devices. And in the end, It also has to be integrated into the surgical workflow, for example, via context awareness so that the system knows what's going on and enables an intuitive human-machine interaction. So I will conclude with the vision of the OR of the future, which is an interconnected sensor environment. And depending on the context, we can, for example, predict the remaining time or complications. We can enable context-aware visualization or robotic camera guidance. And in the end, I also want to advertise our sub-challenges at MICAI this year in endoscopic vision. If you work in this area or are interested, please have a look. We have 6 exciting sub-challenges in all different kinds of tasks with lots of data that might be interesting for you. So in the end, I want to thank my amazing group in Dresden and the PhDs and postdocs, I mean, for which I presented their work today, and also my collaboration partners and, yeah, my funding agencies. So thanks a lot for your attention.

Yeah, thank you very much for coming into our data science seminar today. It was a great pleasure for us for having you here. So before we start with the question and answer session, so, um, you can— all of the audience can post their questions into the chat, so please use this function, and Yeah, we will start with the first question. So, are there any approaches to visualize the intratumoral heterogeneity, for example genotype or phenotype visualization or PET-guided heterogeneity for surgically guided attempts?

I mean, we directly are not working on it and I'm not an expert regarding the state of the art in this field, but I think if you have, for example, intraoperative imaging that makes it possible to visualize it, it would be very helpful in surgery. But I mean, I'm not an expert in this application.

Okay, thank you. So then we have a question from Leo. Ayala, so can you directly speak please?

Hi, can you hear me? Yeah. Okay, so first of all, thanks for the presentation, really nice. I was wondering about the rendering, so the, the newer rendering that you showed. So the results look very nice, I think, but how would you go about into trying to simulate a scene that is also physically basically correct, let's say. So what, what would you need? So for example, would you need the bidirectional distribution functions for the tissue surfaces or yeah, how would you incorporate it in your pipeline?

So there's, this is actually the next step. I mean, I showed you video sequences, but I mean, these are short and there is no deformation integrated, so we need to find a way to incorporate such deformation. And since we have a simulation we use for it and we can extend it also to an FEM simulation, I think we are going in that direction to use a physically plausible simulation for the deformation and then to use it for the rendering. But I mean, in general, we also don't have instruments right now in the scene. And this is also a challenging part. How can we include incorporate different kinds of instruments? And in the end, how can we incorporate the interaction with the tissue?

So what about the interaction of light with the tissue? So like, for example, if you wanted to also simulate multispectral or hyperspectral images instead of RGB images, do you think that will be possible also?

Yeah, I mean, I know that you are working on such applications. So regarding Hyperspectral imaging. Yeah, I'm not an expert. I think you know more than me about that. But I mean, I think it would be really cool if we find a way to synthesize such images. And I think the ERC from Lena, she got, is tackling some of these challenges.

Right? Yes, yes.

Okay, thank you. Yeah, thank you very much. So we have another question from Sebastian Gruber: Which do you think leads to better clinical applications of AI, domain generalization or uncertainty quantification awareness?

Hmm, that's a tricky question. I mean, the applications we focus on are in laparoscopic surgery, and I showed you an example regarding rectal surgery and cholecystectomy and so on. But still, we need to have specific training data for this kind of applications. So I think it would be really cool if we could solve, if we have methods that generalize, and not only regarding the type of surgery, but also the different sites. I mean, we right now, we have a project between Dresden and Heidelberg where we acquire surgical data and still there is some variability involved even if they do the procedures quite similar. So let's imagine there are other centers where we could have access to it. So a generalization would be, I think, something that is really needed. But I mean, I also think uncertainty quantification is needed if we want to bring, if we want to make the methods explainable and to bring AI methods into the clinic, because right now this is a black box and there is a lot going on in the explainable AI research area. And if it's not explainable, I think it also will not be used in surgery or in medicine in general.

Okay, thank you very much for this answer. So there is another question. Is there a specific problem that international collaboration could focus on what would benefit most from a multicentric approach?

I did not understand the first part.

Is there a specific problem that international collaborations could focus on?

I mean, I think there are several problems where international collaborations could profit from. I mean, for example, now we have At NCT, there are 4 new sites and involving also surgical departments at the different sites. So I think it would be great if we could set up a data repository for surgical data we get from 6 different sites. I think this would be a massive advantage. Not only to record the data, but then also find ways to have smart annotation tools, for example, yeah, via active learning. So you only show the samples which are quite uncertain regarding the labeling to save time for the experts. So Yeah. And this is only a German-wide network and imagine it to be in an international setting. I mean, let's not focus on the regulatory boundaries and the security boundaries because they also need to be solved. But yeah, I think surgeries in the US are, for example, performed in different ways or in Asia. Compared to Germany. So there is also a lot of diversity included. So I think the first step would be to get the data and to annotate it so that it can be used for different applications.

Okay, thank you very much for this answer. So at the moment, if I go through the questions, we got it. I think nobody raises his hand, so okay, thank you very much. So for further questions, you can also address Steffi directly, and you can, yeah, directly visit our website, the Data Science Seminar website, for further information. And I say thank you again again, Steffi. Thank you for being our speaker today. Thank you to the audience. And the talk is recorded and will be available on our website later on. So yeah, thank you very much. Stay healthy, everyone, and goodbye. Bye.