Generative AI in Healthcare Series: Lessons in Applying Responsible AI

Foreign.

Let'S get started. Hello everyone. Welcome to the webinar on Gen in Healthcare. This is session number five, Lessons in Applying Responsible AI. My name is Satish Tariqonda. I'm a senior lecturer at Harvard Business School and I'm joined by my colleagues today. Var Shankar, Sabrina, she and Manoj Saxena from the Responsible AI Institute. And before we get started, I wanted to acknowledge a lot of background work which has gone on with the team at Harvard Business School as well as the Responsible AI Institute. And I want to thank them and most of all also thank my colleague Nikhil Bhajwani who's been instrumental in helping getting this going. With that said, I wanted to get started by giving you a little bit of brief background about what we have covered in the previous webinars and then we'll go into framework for implementation and assessment, also talk about some example implementations and give you some advice on how to get started. Please keep putting in the questions in the chat and we'll try to address them to the best of our abilities as we go along or towards the end of the webinar depending on the questions. And before we get started, I'd invite Manoj War and Sabrina to say word about like word or two about their backgrounds and then we'll get going. Great.

Thanks Satish and thanks for the opportunity. I'm Manoj Saxena. I founded Responsible AI Institute eight years ago as a nonprofit dootank, as I call it, to operationalize responsible AI. Before this I was IBM's first general manager for IBM. Watson took it from the Jeopardy game to a few hundred million in revenues. Prior to that I've built and sold a few startups, one of which was bought by bm. And now I have committed my life to operationalizing safe and responsible AI.

Yep. Thank you. My name is Var Shankar. I am the Executive Director of the Institute. I am a corporate lawyer by background and previously developed the AI Ethics course at Kaggle and have been at the Institute on for approximately three years now. And I'll pass it to Sabrina.

Thanks Bar. Hi, I'm Sabrina. I'm an AI Policy Analyst with the Institute. I joined the Institute back in November of last year and my background is in computer science with a focus on AI and machine learning and also in human rights. Thanks.

Thank you. Looking forward to our conversations today and we'll also be featuring some exclusive interviews with some industry experts as we go along in the example implementations. So with that said, going back to webinar one, we started with trying to answer two questions. If AI can help improve healthcare and save millions of lives. And if that is true, then how do you deploy AI in healthcare? Obviously the answer to that was to the first question was yes, it can. And when we speak about how we could deploy, one of the key elements which came up was about deploying it responsibly. When you deploy it at massive scale, not just from a regulatory or ethical perspective, but looking at it, you know that responsibly implementing it responsibly is a necessary function, mainly because you know the concerns arising out of the society about like the dangers of AI and all the implications of those. So we also reviewed a framework which looked at number of domains which are on the left hand side and looked at three use cases in the columns there around measure, decide and execute. And as we looked at all these domains, it was fairly clear that the level of adoption varies across all these domains, but is absolutely accelerating. And AI has been used in healthcare with various levels of maturity. But to scale it, obviously we had to use it responsibly. And what we also articulated was that each healthcare sector has its own specific consideration. As a result, we had Webinar two which focused on biopharma, Webinar three, which focused on digital health, Webinar four on the health systems providers, hospital systems, et cetera. Today's webinar, which is the capstone of all these, focuses on lessons in applying responsible AI with that contextual framework. I want to get started with the framework for implementation and assessment and Manoj Saxena will take over from you. Manoj, over to you.

Thanks Satish. And you said in the beginning, how do you apply AI in healthcare? And my two words are with a lot of excitement and with a lot of caution. And because this is a double edged sword, it has a lot of potential to do good, but also it has properties that needs to be managed carefully. And one of the things we have done at the Responsibility AI Institute, our mission, as I said, is a do tank, not a think tank. So we are there to create tools and processes that help companies operationalize responsible AI. We are a nonprofit and one of the things we have anchored our responsible AI operationalization framework is in the NIST framework. So the National Institute of Standards Technology from the US published the AI RMF AI Risk Management Framework. And we have been for the last seven years working with organizations on the left like NIST and ISO and oecd. And what you see in the middle, the pizza slice, is the seven dimensions around which an AI has to be steered so that the output of it is positive and beneficial to both the patient as well as the society. Artificial intelligence is actually a very poor term. There's nothing artificial in artificial intelligence. It's very real. It has real impact on diseases and people and cost of care and societies. So using these seven components we provide three sets of tools. We call it ABC assessments, benchmarks and certification. So our services for our members are come in and do an assessment like a JD Powers of AI. How safe is it, how valid is it? Will help you then benchmark against what good looks like. And for those that pass it, we certify both individuals as well as systems and processes on how good you are tracking to Responsible AI. If you go to the next slide, I'm going to share with you some examples of what the impact of these systems look like. Here's an example of a GPT style app that NHS England was piloting last year. So, you know, in England there's a lot of issues on getting doctor's time and scheduling. So imagine an app which works like ChatGPT that has all the documents on if you have appendicitis or Covid or a liver cancer, how do you take care of yourself and what medicines to use. On the left, this is from a member company called TrustWise of Responsible AI Institute is an LLM output with the API for guardrails. So on the left the question is what is the NHS recommended dosage where acetaminophen patient is such and such and the answer comes out properly. You know, for adults in this range, take this dosage on the right without the guardrails, you have an AI hallucinating which is coming up with fake stuff. These are token generation machines, these are not SQL queries. And it'll come out and say, well, it's one tablespoon a day, which is not even a dosage. So if you go to the next one we say, well this is a hallucination of type incorrect dosage. So if you go to the next one saying, okay, if I don't have acetaminophen, what do I do? Then you say, well you take ibuprofen or naproxen. On the right it says take a plain reliever like codeine. So that's again outright dangerous. So how do you put groundings as well as guardrails so that the output is something that you can trust and is beneficial? Is an example of one. So if you go to the next one, this is an example where the other challenges they have is of data leakage. These AI systems leak sensitive data. So on the left you have an issue like what medicine should I take if I'm a type 2 diabetes patient? It says consider Metformin. On the right it says take metformin like Tyler Smith who was born on this date, which is the data privacy breach. So these are different dimensions of the NIST framework that we are talking about, personified into actual harms that an AI system can do if you don't design it with a responsible AI framework in mind. Let's go to the next one. So if you scale this out, you know the impact of these things can be massive. So if you look at the 7 dimensions of gen AI concerns, you could have AI making up fictitious diseases and medical history in patients file. You could have dosage hallucinations like we have seen. We could have AI lacking traceability. So if I use it to do care and something happens to the patient, I'm getting sued three years later. Can I go back and say on June 12th at 4:10pm what model and what prompt and what insight was used to drive the decision and can I defend it in front of a jury? So that accountability piece is missing in these systems privacy. I talked about the leakage of data and last but not the least bias. How do you make sure that equal treatments are being offered? So taken together, this whole system of AI implementing AI in a responsible way is what our dedication is and what our focus is. And if you go to the next slide earlier on in the webinars, and I also want to take the moment to shout out and give thanks to the Harvard D Cube Institute for their collaboration with us over the last few months. And in the earlier one of the first sessions we did with the Digital Data Design Institute, Nikhil has spoken about this framework on the building blocks for responsible AI organization. Clearly it needs to start at the top. Like I said, there is nothing artificial about AI. The unfortunate thing that's happening with AI is being left just to technologist. I keep telling the boards, AI is too important to be left to the technologists. It needs to be seen at a board level as an important capability. Because today the way AI is being built unfortunately is engineers are building it and someone says that they're building a car. Someone says where's the safety on it? They said oh don't worry, I'll put a bumper on it before it goes to the paint shop. And they say that's safety. Right now people are waking up with frameworks like NIST and tools like what we are providing that actually they need to design responsible AI not as a bolt on but right in the beginning of the system. So they need to design the crumple zone, the airbags and brakes all the way at the beginning of the design of the AI, not at the end of the AI design. So this framework is a very powerful one in that you start putting down your AI usage principle on number one, number two, then you convert that into principles. This is where the responsible AI tools and frameworks comes in. That's the principles and framework that I just showed you, the pizza slice. And then finally once you have that nailed down you take every use case and you start implementing that use case from built in measurements and tactics and then you have work stream on how to govern it. So with that let me just turn it back over to you in terms of how you operationalize a concept into some real use cases in healthcare.

Okay, so the next one is on example implementations.

Great. Yes. Thanks Manoj. Thanks Satish. Indeed. Next we will be covering two case studies from the Automated Skin Cancer Detective space which will bring color to the key themes that we've been discussing and present details on what these building blocks and what implementation can look like. Next slide. Sorry, keep it here. Firstly, what is automated skin cancer detection? Automated skin cancer detection involves the use of a technology called skin lesion analyzers to detect cancerous lesions. Skin lesion analyzers can be integrated into a patient's diagnosis journey in two main ways, both of which we will discuss today and that you can see in orange. First is as a screening tool that supports a person's self examination of a suspicious skin lesion prior to seeking care. The tool can be delivered via smartphone app and smartphone images of lesions taken by the user are used as input for an AI powered analysis of the lesions for possible forms of cancer. The second is as an AI tool to aid clinicians in their examination. So these analyzers often use dermatoscope images taken by the clinician or by other professionals. There are many benefits to the use of skin lesion analyzers including making the patient experience more convenient, accessible or fast. So access to screening tools allows patients to avoid months long waits for initial screening appointments and the inconvenience or risks to in person visits such as during the pandemic or even for immunocompromised individuals. Screening can also augment dermatologists impact through triaging support such that more of their time is spent on positive cases. An AI copilot may increase accuracy and decrease the resource burden of unnecessary biopsies on the ecosystem and for patients. When used for training, it may also help improve clinicians ability to evaluate lesions. Oh thanks Andrea, that's a great question. We're moving on to that. Next slide please. So generative AI for automated skin care detection actually comes through as generative AI is greatly increasing the accuracy and power of skin lesion analyzers for cancer detection, mainly at the early stages of the model. So in one big case, synthetic image generation could be used to increase representation in data sets and otherwise improve the quality of training data. So as de identified and privacy preserving when created properly and less time consuming to generate than to create than collecting real world images. In some cases, synthetic images of skin lesions that are created by generative AI models can accelerate research and development in this field including the use of synthetic images for performance benchmarking. Generative AI can also be used to improve image segmentation of the lesion area which can improve overall performance. So this is particularly using generative esque frameworks such as generative adversarial networks like GANs. But there are risks to this of course, so one possible risk is the lack of explainability for models that use a type of general generative adversarial network approach or other generative AI approaches. Next slide please. So now we will cover two different case studies for automated skin cancer detection and we'll explore how two different institutions are approaching these same questions that we've been discussing with two different AI systems. The first will be explored through a prerecorded interview with Dr. Veronica Rotenberg from Memorial Sloan Kettering Cancer Center. MSKC is a world leader in patient care, research and educational programs for cancer and has been ranked in the top two best hospitals for cancer every year for the for more than 30 years as director of the Dermatology Imaging Informatics Group or DIIG within MSKCC, Dr. Rotenberg develops benchmarks and validates AI tools for dermatology including for skin cancer detection. One of the group's recent projects has been to evaluate an open source, non commercial and dermoscopy based top breaking algorithm for melanoma detection known as AD adae, which is intended to be used in clinical settings. So great. Yes. Next slide please. In the following interview, Dr. Rotenberg will cover these characteristics of trustworthy AI and provide insights into the risks and possible mitigations from the perspective of diig and these will be highlighted in the text in the upper left of the video that you'll see now.

We don't really know what is a good test for reliability for a skin lesion analyzer. We know you know lots of models might give the same answer every time to the same picture. But that might not be the relevant question for a skin lesion analyzer. It it might be for a lesion that has a 10% risk of melanoma. It will give you that 10% answer each time. But when we are trying to come up with these questions, we really need larger studies. And that's where I think there's a huge opportunity for advocacy to say we need to understand these better for each use case, and we need to do large collaborative studies to figure out the answer. It is true that the cost of doing the kind of studies that we want is also too high, but that is a systems problem. We don't have good systems in, for example, the skin lesion analyzer space, where we could just implement a system for the next 300 biopsies and analyze the results. So for me, it's not about trying to get these technologies going quickly implemented. We should, of course, get things that might benefit health implemented as quickly as we can, but we can't accept that the barrier to that is the expense and inefficiency of a trial. We should say, well, how do we fix the expense and inefficiency of a trial so that we can still do it well and have the answers sooner than we might in the current system? I think that that's what my lab is going to end up doing, is efficient validation, because there's no one doing it. And if you can validate an AI model in six months instead of three years, you know, the benefits to health are better. I mean, it's not just that, you know, companies will like it. It's like, if you have a good model, you are also helping the patients that would have otherwise had waited three years. We need to be able to evaluate the performance of an algorithm in its intended use case. We need to be able to analyze prospectively, when we're considering putting it in a new use case, how it's going to perform. And we're going to need to be able to interrogate potential biases related to the underlying training data and decisions that were made in model development. And sometimes that feels like, like we're asking for proprietary information, but we're not. We're asking for potential biases and potential decisions that will affect performance to be discussed and made public. When we think about skin lesion analyzers, for example, we can often think that certain skin cancers we know are more deadly than other skin cancers. So if you might miss a superficial basalt cell, for example, that's a Skin cancer that is much less likely to cause mortality than a melanoma. But if you think about missing a melanoma, for example, that feels really wrong and like something we have to avoid at all costs. But in these kind of systems, they define that phrase, you know, avoid that at all costs mathematically. So you have to assign a number to that if you're going to be able to move technology forward. And that is something that is super difficult. It's something that requires multidisciplinary buy in. So this is something that requires patient ethicist voices in addition to doctors. And it's very difficult to answer. It's not something that I have seen the answer to yet. There is actually really cool machine learning research devoted to this question. Question of how do we look at bias, especially in skin lesion analyzers? One of the biggest sources of bias is skin tone and the fact that darker skin tones are underrepresented in all the public data sets that we have access to for treating algorithms. The question is, how do we mitigate that bias? And there are approaches that aim to minimize the difference between groups that have the same skin tone, and so there is no difference in those groups. And there's also approaches that aim to minimize the difference between everything in your target category or in a specific class, regardless of skin tone. We don't know the answer. I think it's going to feel like a theme to you that essentially these models have to be tested in clay clinical scenarios, or at least contrived clinical scenarios, where we really think about, you know, what is the potential harm to this individual person if this algorithm makes this mistake in this way. And those are the kind of thought experiments that we have to do in order to answer those questions. We don't really.

Great. Thanks, Satish. So, moving on to our second case, we'll be hearing from Dr. Rakesh Joshi, lead data scientist at Skinopathy Inc. Skinopathy Inc. Is a company based in Canada. It was founded in the summer of 2020 with one simple create AI that can be used by physicians, nurses and patients to pre screen for skin cancers. Since then, Skinopathy's data and AI technology has expanded to other skin concerns such as burns and wounds, and across the dermatological ecosystem. So providing automated support for triage scheduling, treatment recommendations, clinic analytics, among others. Today we'll be focusing on their patient engagement platform and app, GetSkinHelp, that leverages skinopathy AI on any mobile device to provide remote screening for patients, connect patients and clinicians for scheduling options, and enable triaging of cases on the clinician end. It is also completely embedded with provincial health plans outside of Quebec, meaning that there are no out of pocket FE for the user. Next slide please. So in the following interview, Rakesh will discuss how skinopathy approaches, risks and implements mitigations related to the same following characteristics of trustworthy AI.

What would be characteristics of beings of such a platform? I think there are three things that we always talk about within our company. First of all is the consistency of results. For us it's not diagnosis, we only screen. But regardless of whether it's a diagnostic device or diagnostic software or a screening or triage, I think the consistency of results is the first point. Accuracy against a benchmark data set, that's always important. And I think the third point would be error transparency. If there are biases, errors within your system, you should be transparent about that. So for consistency and results, we want to make sure that the results of the AI system are reproducible regardless of the skin lesions. Since we're talking about skin lesion here, skin lesion type, skin tone type, as well as like the lighting of when that image was taken. Right. Let's say we're talking about a computer vision AI platform. So those would be the things we would want to look at when we are talking about consistency of results. The second thing is accuracy against benchmark. We at like we have our own data set, right? Proprietary data set or data sets with our collaborators, we train our models on them. But then how do we know how well it would perform against, you know, out there, Right. So we always do benchmarking. For example, for a skin lesion analyzer, we take the same model and make sure that it performs well against the open source data sources out there. So benchmarking is very important. The last one of course I mentioned was transparency, error transparency. So we always, what we do here at is we make sure that we publish our results right. For our skin analyzer, one of the current models, we just published a preprint paper where we put down all the limitations of our system, the biases that are there. We publish that on our website as well so that, you know, general user who might not go into the primary literature to read papers, at least they have that information. So I think if you have these, then you could say you have a pretty reliable system out there. Randomized clinical trials allows us to see whether an AI platform will be valid and reliable, first of all across a larger demographic. So that's the first thing we absolutely agree on, that we ourselves are trying to Be partners and larger RCTs. The other thing we realized when we are partnering up with, for example, we are partnering up with a hospital here who will be doing pilot of our skin analyzing. But one thing we realized was RCTs are great, they have to be done. But we would like an additional thing to be added to that is that they should be like, even if it's a three month pilot, that should be done wherever a skin analyzer is being deployed. Because what we realized working with this hospital here just outside Toronto, is that every clinic, every hospital, they have a different workflow. Of course it's a skin cancer clinic, but they have a different workflow. Some institutes are large, they have more doctors, it's more interdisciplinary where some are not. And the way that skin analyzer is used, it can affect accuracy, it can affect interpretation of the data. So yes, we are all for rcts. But we would like to add that even pilots, even if it's a short term, one month, two months deployment, those should also be almost mandatory for that clinic or that we are in the medical space, we know what the end result is going to be if we cut corners, right? So we know that we ourselves are patient, right? So for us, when we started on this journey, we knew that it's going to be a slow, painstaking process, but that's how it should be done. There's no cutting corners when you're talking about someone's health. We do need to figure out things piecemeal. We will need to recalibrate as we go along. For example, let me give you an example. So our, our skin analyzer, like the data, we collect them here in the GTA in Alberta. There is of course a skewed, skewed data set. Right. When you look at skin tone, it's usually type 2 to type 4, let's say whether we have a lot of data, but then type 5, 6, we don't have that many examples within our data set. Right now we are talking with folks in North Africa who want to bring this. So we said the first thing is, well, we need more data, right? Because we just, our system cannot, yes, it does have some level of accuracy, but we are not comfortable going there. So that's the first step we have to do. We have to make sure that if it's a geographic area or demographic that we are not very comfortable doing it. We need to do those steps. Even if it takes a couple of months extra, that's fine. Most of our data probably are of lesions where it was serious enough that the patients actually came to the hospitals, actually came to the clinics where, from where we source our data. So what happens for earlier, you know, earlier presentations of lesions, like it might look different from what it does later on. So we might be missing that. So we, for us, our biggest concern is balanced data set in all sense. I think that's what concerns us most and that's what kind of keeps me up at night trying to figure out ways where we can mitigate that. If a metadata like a patient detail helps our model, increases our model's accuracy, but does not reach the privacy statements that we have, the privacy framework we have, then otherwise we don't. And just to clarify, what we use right now are AI systems. It's mostly just the computer vision, but we also ask patients questions and that goes into a separate algorithm which is just a simple decision that's like just medical knowledge encoded within the platform. So yes, there is a trade off, definitely. But as long as we are not going more than that threshold we have said we would reach, I think it's okay. And again, you know, it's always a multi stakeholder decision, right? When we are, when we are saying, okay, let's add more metadata, more patient details within our algorithm. Regardless of what the algorithm is, we always talk with everyone. We talk with our patient groups, we talk with our advisors who are physicians and ethicists as well as our privacy officer. And then we decide on that like, okay, it's okay to do this or not to do this. That's one reason we are very, I guess, conservative on what we use compared to maybe some other companies on how they are doing. So the way we do it here is that we show the AI results to the patient in a very simplistic manner. Like we use plain language, not too much of technical language. We make sure very big printing that this is not a diagnosis. It's just the screening. If you see melanoma popping up, you know, don't get, you know, don't have anxiety attacks, right? But we know humans being humans, if I see something like that, you know I'm going to be freaking out. Right? So we have ways to mitigate that. Like we allow them to connect directly to a virtual, virtual consult, again, human in the group, so that the clinician or at least the nurse practitioner will see the same image and say, okay, you know what, you don't have to worry about it now or we will bump you up to the next priority level and haven't seen a specialist. Right. So we. So on the patient facing Side, we keep things very simple but very strict, right? We show them the results, but we make sure that they understand very well what these results mean. On the physician facing side, we show them everything. We show them the results of the AI, we show them the probabilities, the ranking, we show them saliency maps. Like why did the AI say this is a melanoma, not a melanoma? So all of that is going to be shown to the physician because at the end of the day we are just a clinical decision. We are a tool that aids the clinician. It's the clinician that's going to make the final verdict. Right? We just do the screening, we just triage the patient thing. Okay, you have 10 patients, these three might be the ones you want to look first. Look at first and that's it. So yeah, again it's very use case specific for us. That's the way we have done it because that mitigates the risk the most we have found. And you know, again, it was a multi stakeholder discussion when we decided on doing this. What's the best way to do it without, you know, freaking the patient or the user up, making the workflow smooth, even for the physician or the nurse practitioner who actually helps that person through that journey? We actually don't show the AI results to the clinicians right off. We only show them the image and we let them first say, okay, this is the clinical assessment. And then we show the AI and by just flipping that, you know, sequence, what we have realized is we do not bias the clinicians. Right? So it's after the clinician says, okay, from my experience, I'll say this person does not have a melanoma or does not have a pcc, scp, whatever, whatever you think. And then we provide them with the AI result. And then we say, okay, the AI is saying this because of this reason, the cdnc, map, et cetera. So just by flipping, by having that small change in sequence, we think that we are kind of mitigating, influencing the clinical decision.

Right.

What would be characteristics of being of such a platform? I think there are three things that we always talk about within our company, I think. First of all.

Great, thanks. So now that those are complete, I'll pass it off to Bar, who will discuss how you can get started based on what we have just covered.

Great, thanks Sabrina. So now that we've seen these two case studies from organizations at the intersection of AI and dermatology, we can talk about what lessons we can draw from the case studies and how our Viewers can get started with a responsible AI approach. So first is to build upon your existing knowledge of industry regulation and of the risks involved in specific use cases. So as we can see from both case studies, the practitioners here bring a wealth of knowledge, deep knowledge of their industries and understand their use cases and all the players involved. And that's where we all can start. Second, as Manoj spoke about at the beginning of our session, is to be thoughtful about the benefits and the pitfalls of adding AI and generative AI to a specific use case. So beyond understanding the industry and use case which is necessary, understanding the risks from particular generative AI systems and understanding the landscape of methodologies and tools to monitor and mitigate these risks. Third, which our experts kind of focus strongly on in the case studies is to define what the trustworthy characteristics mean to you and to make them measurable. So as an example is reliability defined by a metric for reproducibility or for sensitivity? And then getting a bit deeper, which metric or which set of metrics? Fourth is to invest heavily in testing, evaluation, verification and validation. And so both of the case Studies talked about RCTs and the skinopathy case study. Also we also covered kind of short pilots in the organ in the kind of organizations that actually use the tools. So it's, it's important to kind of make these investments so that you have visibility into and data on how a system or approach is performing on a given use case. Fifth is to make sure that others are checking your work. So one way that some organizations do this is to make a commitment to transparency so that your algorithms can be evaluated independently. So that's what you know, Diig the lab at MSK does. But there are also other ways to kind of bring in independent or arm's length parties to kind of provide a second set of eyes on your work. Can we get the next slide please? Great. So zooming out a little bit, let's kind of recap what this means at the organizational level. So you know, Satish and Manoj talked about at the top of the hour how each healthcare sector and use case has its own considerations. But at the same time there are common elements of a responsible AI approach. The NIST AI Risk Management Framework is a kind of recognized framework that outlines common terms, concepts and approaches that can inform responsible AI programs. And so that's why we, we at the Institute use the seven characteristics of the NIST AI RMF as kind of the pillars of our analysis. NIST also maintains a portal where organizations can contribute AI RMF profiles and share lessons with each other of what implementation of the framework looks like in their industry and use case. So to adopt a responsible AI program, organizations should develop AI principles, governance and policies that are interlinked with other organizational functions like risk, privacy and security, so that there's a clear understanding of the internal roles and responsibilities when it comes to kind of understanding the pitfalls of AI systems. Joining groups that can help you monitor new developments in AI policy and research, both in your industry and your use cases and more generally is a great way to kind of learn about how others are thinking about these problems. You know, especially given the pace of change in technology, in policy and in research, it's important for us kind of all to work together to make sure that we're mitigating risks and sharing best practices to do so appropriately. And finally thinking about how your organization can learn on an ongoing basis. So whether through it's, it's through courses like this one or kind of mandatory employee training, partnerships with research institutions, partnerships with nonprofit organizations, it's really important for everybody in your organization to be aware of these issues. So with that I'll pass it back to you. Satish for to get into our Q and A.

Thank you. Thank you. And please put the questions in the chat and we'll try to address them as they come along. And one of the questions which what role does culture play in applying AI in healthcare and how do we foster that data driven culture? Data AI driven culture, perhaps. Manoj, you want to take that question?

Yeah, yeah.

You know, I think as a starting point and what is culture?

Right?

Culture is nothing but a set of beliefs and actions. That's kind of what defines culture. And to me, when I made the point earlier that AI is too important to be left to the technologists, I was not being disrespectful to them. It's just saying that technologies need to be supported by leadership and a culture that realizes the power of these technologies. So in terms of culture, I think it's everything. It starts at the top where leaders have to believe that this is truly transformative. Imagine when email came about, your leadership had to look at it as a game changing business technology and they had to build a culture where people both recognized it and they started trying it and then implementing email systems. So AI is like an email systems times a million in terms of the business impact. So it's an incredibly important part and it needs to exist at four different levels. The culture one, it has to start at the leadership level where the C suite has to build an environment and propose a set of beliefs and actions by funding IT and investing in it. So that's one level of culture. The second level of culture is at the IT and business operations levels. So they have to start experimenting and trying, like you saw in those case studies, to show the art of the possible with AI. So you need to build an experimentation environment and a culture which promotes experimentation. Third, there needs to be culture around the notion of lifelong learning. So how do you make sure your employees and your company, like Var talked about education, how do you build a culture of lifelong learning around you? That's the third level. And fourth and final one is a culture of patient advocacy and member advocacy to make sure that you are coming at AI outcomes in and not going at IT data and models out. So those four pieces, culture at a leadership level, culture at an experimentation level by business, culture at a learning, building a knowledge and a lifelong learning team and then culture at a patient advocacy level. I think those are all the areas. And it's not just a data driven culture, it's an outcome driven culture. I think one of the mistakes we made is to say we are an AI first company. I think I would say we are an outcomes first company enabled by AI and then enabled by data. So that's sort of how I would phrase it.

Thanks Manoj. Just expanding, double clicking on that a little bit. Are there best practices you have seen on this topic from other sectors that. I mean clearly the Responsible AI Institute deals with a number of verticals, you know, finance, banking, you know, other related verticals. Are there anything that we could take from those vertical that you see either from culture perspective, implementation or governance and so on and so forth Perhaps.

There was actually a great study that was done recently about what are people that are deploying Gen AI successfully doing different and what came out of it satish where there are four big things that are different. Number one, that it starts with the leadership. Like I talked about, there's strong leadership support and a culture where people says let's go out and play with this stuff. Let's invest in training, let's invest in technology. So let's call it people and process culture. That's one thing they do different, starting with the leadership down. Number two is there is a portfolio approach that people are starting to look at this as a portfolio of use cases and not just little experiments. One of the CEOs of a large bank told me, you know, he used to use the term dots versus bubbles. He's like, everyone's coming to me with little dots of 1 million 2 million ROI. Where is my big bubble of a billion dollar ROI? So having a portfolio approach that differentiates the use cases to say what's dots and what's bubbles, that's the second thing. The third thing is putting in guidelines and policies in place, particularly responsible AI guidelines and processes and putting tools in place. That's the third thing they do different. And the fourth one is ecosystems really understanding that this is a complex, fast moving space and we need to be able to embrace learning from outside. Whether it's through institutes like the D Cubed at Harvard, whether it's Responsible AI Institute, whether it's startups, whether it's venture funded or whether it's large technology companies. So all these four pieces is what I've seen. And there is a great report from BCG on this, which is another member company of Responsible AI Institute that I will request the teams to look up for more color on it.

Okay, great.

You know, staying with you Manoj. One of the question which came about is in the chat is some commentary on the new AI law in the eu. Any thoughts on that? If you could. And then you know.

No, absolutely. I think, you know, AI needs to be regulated and shaped like the way nuclear power was and like the way chemicals were. So this is a very transformative technology that has to be regulated and efforts are going on. As of now there are over 800 regulations, guidelines and policies, 800 globally that are coming up on AI. But if you zoom out and look at it, there are three different techniques that countries are using to regulate AI. On one extreme you have countries like China and looks like India is joining that, where you have to register every algorithm with the government. Right. So that's a very tightly centralized control part on regulations. And then the other extreme of it is innovation friendly AI regulations which us, Canada and UK are taking. And then in the middle, which is a consumer friendly thing, is what EU AI act is doing. It's called the Brussels Effect.

Right.

So they're taking their buying power as multiple countries and they're mandating how AI is to be implemented. My view is in the long term, probably the winners in this are going to be more of the people who are taking an innovation friendly approach, but putting enough guidelines like we have with FDA and others eu, one of the things we're going to learn from them is, is I think a strong orientation of consumer protection and consumer law and I think the US will benefit from that. But if I were to look out 10 years and predict where these laws are going to help I would put number one, the US UK innovation approach, number two, frankly China and India approach. And third, I would put eu, which will be in terms of societal benefit. It's a little bit more conservative approach, but I think it's, it's how the, these, these laws are shaping up.

Great, thanks. There are a few other topics which are percolating in the chat there. So there was a question here, I think maybe war this from an how to get started perspective. There was a question around should every company have an executive champion for responsible AI and where, where should that come from? Is that the cto, cio, you know, P and L leader or some others? You know, probably if you can amplify on that and just speak a little bit about some thoughts around where that leadership should come from for some of the topics that Manoj articulated.

Great. Yeah, I'm happy to start and then I'll kind of ask other others to weigh in as well. So the Responsible AI Institute put out a document on best practices for AI governance structures that is kind of informed by a multi stakeholder effort. And what we kind of advocate is having an impartial, accountable and effective governance structure. So focusing on kind of what it does rather than how it looks. And so that's, I'd encourage our viewers to kind of download our best practices for AI governance structures. I think as a general view we see kind of two levels of responsible AI governance forming within leading organizations. The first is kind of the top level which is a high level board or steering committee that provides direction, mandates and resourcing. And the second is a more operational committee and the job of that committee is to develop the kind of tools and training and governance for the organization, to escalate issues for the higher level board or steering committee and to conduct reviews of specific AI systems or system types and kind of capture that knowledge. And so you know, just finally who, who is best positioned to lead these, these efforts? The chief data officer is often kind of given the additional hat of the chief data and AI officer. We also see in highly regulated industries that the kind of general counsel or top legal officer also taking on a lot of the kind of risk, risk related considerations, especially around the AI act and emerging regulations in the us. But what's really important is to bring all of those executives together in a high level steering committee to drive responsible AI efforts.

Thanks War. Manoj, there's a follow up question to your thoughts on the culture discussed which is, you know, obviously leadership support at health systems is often difficult because they are under tremendous Financial pressure, competing priorities and so forth. And from this attendee, the question is, you know, for institutions under financial pressure, what one recommendation would you make to adopt the culture discussed?

Yeah, I think the one recommendation would be that responsible AI is profitable AI. This is not esg, feel good type of stuff. If you do this well, you can actually save money and invest more. And one of the recommendations I made to boards in healthcare companies actually that are financially constrained is to say, put your projects in two buckets. Call it sort of mundane versus moonshots. Mundane projects are things that you can take to release operational productivity in clinicians work or payments or claims processing. Through that, you can save enough money through simple automation because a little bit of AI can go a long way. So the mundane projects can release enough cash that you can use then to fund your moonshots. So the starting point, what's the powerful thing about generative AI and AI is that the use case here is all about cost savings to begin with. So implement it in simple projects, save the money, and then invest it into the transformational projects.

Thanks, Manoj. So the articulation is start with cost, maybe focus on revenue things before you get to more predictive pieces. How about does quality compliance, any of that get lost in the shuffle if you start on the cost side? Or is that like afterwards you bring, you know, any efficiencies in compliance?

There are a lot of simple use cases where compliance doesn't come into play. For example, the ability of generative AI to summarize notes after a doctor a patient's visit and be able to generate a draft email to a nurse, to the pharmacy benefits management people, as well as to the doctor as more of, hey, here's my suggestion on what I just heard. So next visit, when it happens, make sure you're scheduling her for these tests before they come in. So going beyond alerts and notifications to personalized emails and recommendations as the next best action, those things are all internal processes. You don't need a lot of compliance around it. These are used as first draft guidance that the human can then close the loop. So the notion of human LED automation I think is a very important piece in all of this. Because AI to me is not artificial intelligence. AI to me is augmented intelligence. That's where the most value is.

Great, thanks Manoj. We have a few more minutes, so a couple more questions which were. I'm just monitoring the chat piece here, which is looking at size of data sets. I think one of the attendees pointed out about like, you know, in rare diseases and things like that where you using extremely small data sets to train your algorithms. Any thoughts on those? And I think there was a follow up question on longitudinal data which we can address. But size of data sets to train, if they are really small, what are the implications or how do you handle those situations? And I'll open it up to all of you.

I think Sabrina has actually answered it. Well, that's one of the most powerful things that Genai synthetic data. So Sabrina, you want to take that?

Sure, I'd love to. Agreed. I do think that with the advent where generative AI is going with its general purpose foundation, that capability really allows for us to ideally address these gaps through few shot, zero shot learning. But of course there are risks here, especially in the follow up question where you know, comorbidity, morbidities, when you're actually really focusing on identifying causation versus correlation, how those two, how to make sure you're making the right decision for something AI. AI, you know, isn't too great at that. And I think one of the things that came out of the AI summit, one of the, one of the events one person from this space talked about how what is the best indicator that someone is at high risk of passing when they enter the er and the joke is that it's a stack, it's a stack of papers, a long, long medical history. Now is that useful information, is that going to help you identify a particular thing, a particular treatment? Perhaps not. And so when we're thinking about having generated I identify a cause, identify a very particular diagnosis, you need to be careful about, you know, these types of assumptions that we might not even be able to see because it's baked into, into a model and not something that we can discuss.

Synthetic data generation is one of the most powerful capabilities of generative AI and that could be used in rare diseases where there is sparse data sets to be able to create a lot of data sets that could be used for training. So kind of turning AI on itself to say now go generate safe questions and unsafe questions on which I'm going to learn I think is a very powerful use.

Thank you. Thank you. Manoj War and Sabrina. I think we are coming up on the end of the webinar a very fascinating discussion on variety of topics. I want to close out, you know, first of all again for thanking all of you for your time and for also the attendees for your questions and participating in the attending this webinar. But there are three announcements. I wanted to make all the webinars and insights pages from this series, in addition to like, you know, really good bonus content is can be found at the DQube website.

You know, if you can if you.

See the link in the chat that can take you to the DQ website at Howard Business School. And we also have many other materials and events related to responsible innovation, application of ethical AI and a lot of topics around surrounding these areas and including we also have an upcoming summit program for AI entrepreneurs on defining and pitching your AI startup. Please look that up. And finally, please be sure to sign up for the D Cube newsletter and follow DCUBE and our responsible AI on social media to stay in the loop about these upcoming offerings. And with that, thank you all and that we come to the conclusion of the webinar. Thank you very much.

That ends the.