#159 Building Trustworthy AI with Beena Ammanath, Global Head of the Deloitte AI Institute

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI Transformation Journey on datacamp.com I'm excited about the fact about how much attention AI regulation is getting now, so I'm hopeful that we'll see more progress on regulations. Of course it takes time, but hopefully this will be a time when most organizations start imposing some level of self regulations and from a trustworthy AI aspect, right? So I'm very optimistic about where we are headed with AI as long as we consider the downsides and address it.

Hello everyone. Welcome to Data Framed. I'm Adele, data evangelist and educator at datacamp, and if you're new here, Data Framed is a weekly podcast in which we explore how individuals and organizations can succeed with data and AI. Throughout the past year, we've seen AI go from a nice to have for many organizations today to a must have. Almost every boardroom today is talking about its generative AI strategy and as a result, there's never been more pressure on the data team to deliver with AI. However, as the pressure to deliver with AI grows, the need to build safe and trustworthy experiences has never been more important. So how do we balance between innovation and building these trustworthy experiences? How do you make responsible AI practical? Who should we get into the room when discussing the trust angle of AI use cases? Here to answer these questions is Bina Amanath. Bina leads Trustworthy AI and Technology Trust Ethics at Deloitte. She is the author of Trustworthy AI, a book that can help businesses navigate trust and ethics in AI. Bina has extensive global experience in AI and digital transformation spanning across E commerce, finance, marketing, telecom, retail, software services and industrial domains, and a lot more. In our conversation today, we delve into the core principles of trustworthy AI and the interplay of ethics and AI in various industries, how to make trustworthy AI practical, who are the primary stakeholders for ensuring trustworthy AI, the importance of AI literacy when promoting responsible and trustworthy AI, and a lot more. If you enjoyed this episode, make sure to let us know in the comments on social or more and now onto today's episode. Beena Amanath, it's great to have you on the show.

Thank you for having me.

Adele, you are the global head of the Deloitte AI Institute, a technology trust leader and author of the book Trustworthy AI. So maybe first setting the stage, especially in the industry, I'd love to get some definitions out of the way. We see a lot of different terms within the responsible AI ethics sphere. We have AI ethics, responsible AI, trustworthy AI. So maybe walk us through your understanding of these terms, where they intersect, where they diverge, and how you settled on the term trustworthy AI for your book.

Yeah, Adele, that's a great question to start with. Just in terms of level setting. And I wanted to approach it very much from an enterprise lens to solve for AI ethics or all the challenges that we hear about in the headlines. I think you have to look at it purely from an enterprise lens. And at the end of the day, what matters in an enterprise perspective, if you are building an AI solution or AI product, you want your product or solution to be trustworthy. And that includes, in my mind, the dimensions of ethics, responsibility and accountability, transparency, all these different dimensions, because it's trust encompasses a number of different dimensions. At the end of the day, if your users trust your product, then they're going to use it, then they're going to adopt it, and your product is going to succeed. So I think trust for me is a good way to define and it also enables us to measure when the product gets rolled out.

Okay, that's really great and I really want to deep dive with you on what you mentioned. Here are the dimensions of trustworthy AI in your book. You mention or you identify six distinct dimensions to trustworthy AI. Love it if you could share these dimensions first at a high level, and more importantly, walk me through the process of identifying these dimensions as you were writing this book.

Yeah, I think we'll start with the big one. That always comes up in every ethics conversation or any headline on AI. Many times it comes up is around bias and fairness. And in my experience and Adele, I'm sure your audience has seen my prior experience as well, is that I've worked in a variety of industries. Right. And yes, fairness and bias is absolutely crucial, but it may not be applicable in certain use cases. So the best way to think about these dimensions is to identify which of these dimensions is relevant for the use case that you're working on or for the AI product that you're building, and then identify the dimensions that are relevant and then define the metrics to measure and track it. So let's start with fairness and bias. If you are, in most cases, if you're not using human data, or if you're not directly influencing human behavior, then fairness and bias may not come into the picture. What do I mean by that? Anything that is directly consumer facing, whether it is personalized, marketing, patient diagnosis, bias is a crucial factor and you absolutely have to be able to address it. But if you are looking at, for example, say, predicting a machine failure, predicting a jet engine failure, then bias may not be as relevant. It is more IoT data, it is more machine data. It is about looking at service records, it's about looking at the black box data and then predicting whether that jet engine is going to fail and when it'll fail. Right. So I think it is very important to use these dimensions that I'm going to walk through and identify which ones are relevant. And once you identify the dimension, I'm hitting a little bit on the process part that you mentioned. Once you identify the dimension, it's important to define what's the axis accepted level, Especially for something like bias. We know that it is impossible to have completely unbiased algorithms. It just doesn't work. So then what's the accepted level of bias? Say you're using AI, say facial recognition technology, say you're using it in a factory floor in a manufacturing plant to prevent worker accidents. If you see somebody's eyes are drooping or they look like they might be falling asleep while on their job, you might want to do some preventive action to prevent the accident. But that same technology, facial recognition, can be used in a law enforcement scenario to tag somebody as a criminal. And the accepted level of bias there is absolutely zero. Because if you flag somebody as a criminal wrongfully, then their life gets disrupted in a way that's unrecoverable. So there's no zero tolerance for bias. But in that factory scenario, the question to ask is it helping us prevent accidents? Has it changed the metrics? Have the number of accidents reduced by 10%, 90%? What's the accepted level? Yes. It may not be catching every possible scenario, but at the same time, do you want to use this algorithm? Think of an extreme example. We know facial recognition technology is being used by certain NGOs to identify human trafficking victims and kidnapping victims, and pretty much in a very law enforcement type of setting where it's being used at traffic stoplights and so on. Now there, the question to ask is, what's the accepted level of bias? Is it helping us rescue 60% more victims than we could if we didn't use technology. Right. Is that okay? Do we understand it's bias, but is that still an acceptable metric? So for each of these dimensions that we'll discuss today, I think it's important to identify whether it's relevant for your specific product, project, use case and what's the access accepted level of that dimension. What's the accepted level of bias in this specific case?

Yeah, no, I love the way you focus here on how you apply that trade off thinking depending on the use case. Especially when it comes to bias and fairness. I find it's a very pragmatic view on how to approach these different dimensions and maybe thinking about bias and fairness. How have you seen the landscape of solutions for leaders looking to build out use cases that leverage human data? How should we think about the trade off when we start leveraging use cases that use human data in this regards?

That's a great point. You're going into that next level of detail, right? Like who makes that decision? What's the accepted level of bias? And that's where you need to bring in the key stakeholders across the organization. It is not the data scientist, it is not just the data science team or the AI team or even the technology team where the project is being built. You have to absolutely make sure when you are defining those metrics, right, the accepted level of bias. It is crucial to make sure that you have business leadership involved. At the end of the day, it is not a technology decision because whatever decision is made comes with inherent risk to your organization. So in that factory floor scenario, right, where you're using potentially biased facial recognition technology to prevent accidents, it's not just the IT team that's developing and deploying it. It's the quality and risk management team. It's a brand prot team, it's your legal and compliance team. It is the business owner, the GM for the business that manages that factory. I think it is a quick meeting, but where you have that discussion and make an informed decision on this is the benefit. But these are the risks that come with it. And what are we okay with moving forward? I think it is absolutely crucial to bring in the stakeholders beyond just the data or AI team to make that informed decision so that the business has bought in and is aware about the risks that come with it.

That's really great and I'm excited to unpack with you. Even furthermore, tactical ways teams right now can start operationalizing trustworthy AI. But I want to go into the dimensions a bit more In a lot of ways. You mentioned a great example of predicting jet engine failure as not an example of a use case where bias and fairness is going to be very foundational or important, but that is a use case where another dimension is very important, which is robustness reliability. Because if there is a sacrifice made on robustness and reliability here, that could mean something, that could mean the difference between life and death in a lot of ways. So I'd love to learn from your perspective what makes an AI system robust and reliable.

I hope, Adel, you've read my book because you're literally quoting me, so that's great.

I am. I have read the book.

And that's where again, that term trust is important for me. Right. As opposed to just fairness. Bias squarely fall under the ethical implications. But for your AI product to be trustworthy, it has to be reliable. And in the world that we live in today, where we hear a lot about hallucination. Right? But if you take a step back, what is hallucination? It is a reliability issue. It is software that's not consistently producing reliable results. So it is a reliability issue, which we cutely call it hallucination today. But for me, reliability means where you are able to configure consistently provide accurate results that match the metrics that you've defined. What do I mean by that? The reliability metrics can be different. And let's use hallucination, because that's in the common vernacular today, right. Is if you are using AI for doing personalized targeted ads, your reliability can be a bit off, it can be a little bit more flexible. Okay. You serve the wrong ad to the wrong Persona. Okay. But if you are using that algorithm for some kind of patient diagnosis, it needs to be 100% accurate, even if there is a human in the loop. So you take the reliability dimension and you're able to look at it from that use case lens. See, in your world, Adele, right. Your education, if you recommend the wrong course to the wrong student, what's the accepted level of reliability? Is that a life and death scenario? Probably not. But if you are recommending the wrong drug drug to a patient. Right. If you're using AI, which I was just seeing recently, a result where a lot of consumers are beginning to use AI to look at treatments or understand medical jargon. So I think it really comes down to that use case, the reliability metrics. And it is in the simplest way, thanks to Gen AI, I can say reliability is all about how do you tackle it?

Yeah, we're definitely going to expand on how trustworthy AI has evolved given the advent of generative AI here as well. But given the that you just mentioned, generative AI, I think one thing that I've also been thinking about when it comes to generative AI is the concept of transparency, which is another dimension that you have in the book laid out. I remember five years ago, we used to talk a lot about the importance of opening up black box models and trying to make models interpretable. And now we live in an era of large language models where we have models with billions if not trillions of parameters as these models become stronger and more sophisticated. So maybe how do you think about the concept of transparency and the trade offs leaders need to make now between transparency and performance as these models become more and more performant?

Yeah, yeah. I think transparency, there are two parts to it. Transparency is showing a little bit on how that model arrived at a certain result or a certain outcome. But it's also making sure that the stakeholders understand how the model operates and the risks that come with it. The kind of training data that has been used, what are the gaps, what has it not been trained on? So transparency is being able to provide that level of visibility from a dimension perspective. But in terms of metrics and how do you weigh the opportunity versus the risk? I think again, it comes down to the use case. If you're using transparency, if your Maps algorithm recommends a certain pathway to get from point A to point B, you probably don't really care. How did it arrive at that optimal path? Okay, it might, you know, in which sometimes my app does it, which it might take you through a longer path. Right. But if you're a doctor going back to that patient care, right, where your algorithm is recommending a certain treatment pathway, then as a doctor, it is absolutely crucial to understand what were the factors that were used for the algorithm to make that decision. Or we've seen cases where, you know, credit cards were denied. Right. I think they're like driving that level of transparency is crucial. So it comes down to the application of the AI. It comes down to what's the level of transparency that's needed. Sometimes it's okay for the AI to be a black box. I do not need to know why my music app is recommending me a certain song. Right. I really don't need to know the black box. I do not need that information. Right. But if my doctor is recommending a certain treatment plan based on the AI recommendation, then I as a consumer want to know and the doctor needs to know as well to make sure that it's the right path. Right. Because there's a lot more impact if that level of transparency is not available. And at the end of the day, if the end consumer doesn't feel that you as a business, as an enterprise are not being transparent, then they're not going to trust whatever AI solutions that you put out. So I think transparency is an absolutely crucial pillar for building trust in your AI.

That's awesome. And one thing across our discussion so far that is a common theme, a common thread, is the importance of looking at it by a use case by use case basis. The importance of understanding the level of risk across different use cases. So maybe how do you think about a framework for categorizing risk for different AI applications? I know this is maybe a bit of an annoying question in the sense that there are different use cases across different industries. What is the kind of a framework that you would apply when it comes to categorizing applications of AI vis a vis their risk level or potential risk level?

Yeah, look, there are going to be regulations that come into play and will help us define this more in a standard way. But as of today, there is a balance of self regulation that needs to happen. Right. And it is the organization that is building or using the AI that has to define the tolerable level of risk for their enterprise. And that's why it's absolutely important to include your chief legal or compliance officer, chief risk officer in these conversations. Right. I don't think it's a one size fits all. And even within an organization. Right. We've talked a little bit about patient diagnosis, but if you're using AI for hospital bed allocation and you're using AI to do it in an automated way, is that less risky? If your marketing department is using AI for providing targeted marketing. Right. Would that be considered less risky? And I think that's where having that AI steering committee or a group of stakeholders that come together and have that discussion. Right. Taking it beyond your IT or AI team to bring in the business leaders and having that discussion and defining the level of risk tolerance for your own organization. I think that's the number one thing that needs to be done to define the level of risk. There is no one size fits all. Even at an organizational level, it will be based on what the AI marketing tool might be doing versus what the AI tool is doing in the factory floor. Very different. But the stakeholders have to weigh in. And I have a pillar in the book Adele called Accountability. And I think that one is where it kind of forces that discussion because accountability is about as you are building the project or as you are building the product or designing it. Defining upfront, who is accountable for it when things go wrong? Because we've seen scenarios in the last three, four years when the algorithm does something wrong. The CEO might have to go face the law enforcement. Sometimes the data scientist gets fired, right? There is no fixed protocol for it. So defining accountability upfront is actually serves two purposes. One is it forces that risk discussion. It forces the discussion with bringing all the stakeholders together. And secondly, it also almost puts a level of urgency to have that discussion and mitigate them. Because I fundamentally believe that all of us, technologists especially, are not setting out to do harm. It is just that in our processes, in our project management tools, there are no checks for these risks. Right? For too long, we focus just on the ROI and the positive value creation of AI. And look, I'm a technologist by training. It is very easy to get enamored by the cool things that AI can do. But when doing it at an enterprise level, I think bringing in that risk lens is crucial. And whether it's just adding a simple check into your project management tool, saying, have the risk for trust and ethics being considered, and forcing that conversation, and even if you spend just 5% of your entire project planning time brainstorming it, if you identify those risks, I fundamentally believe that you will do something to fix it. The discussion doesn't happen today. The conversation doesn't happen today because our processes are not set up to force that discussion. So that's why we are not thinking about those risks. But thankfully now we are at that point where there is more urgency around operationalizing for these risks. And especially after regulations come in, it will be a mandate. But I think there's a lot companies can do today to start operationalizing.

You mentioned here, the accountability kind of segues into my next question quite well here because leaders are trying now, like leaders are listening to this podcast, they're trying to think about, okay, how do I operationalize these conversations within my organization? So in your opinion, who should take the lead in terms of organizing different stakeholders to discuss the potential risk of AI use cases? And more importantly, who should the stakeholders be? You mentioned quite a few different profiles from legal risk, business, functional stakeholders, and what should that checklist look like?

Yeah, no, that's a great point. And that's actually part of my role at Deloitte, so I can talk to it from my own experience. Experience, right. Leading a change of this scale at a large organization at Deloitte and It comes from years of prior experience building data analytics and AI products and taking them to market. I've seen those gaps. So I think if a company is farther ahead in their journey, you probably need an explicit AI ethics officer or a leader like I lead our technology trust ethics focus on which includes the trust and ethical component. Right. And a few things that I've seen working across organizations because there are a number of industries where they are in the mode of self regulating and there are best practices evolving and being shared. Right. So one of the key factors, the first step should be to gather this steering committee, this group of stakeholders which is cross business and cross function. Meaning if you have different business lines, the CEO or a representative of the CEO should be part of this committee. Then the functions, marketing, finance, legal and compliance risk, they should also have a seat. At the end of the day, it is bringing in your cross business and cross functional committee of senior most leaders to be able to have these discussions, define those metrics and move forward. That's number one. Number two is doing an AI ethics level fluency, right? I'm assuming the organization has some level of AI fluency, but if you don't, you know, having a base level AI fluency training where every employee in the organization understands the basics of AI. Right? And maybe they take a data camp course to do that, but understand the basics of AI. But one big component should be around the ethical principles, the trust principles that the organization believes in and how can employees get engaged. So everybody is on the same page, they're talking the same language. When the question on fairness comes up, they know how the organization internally tracks it and then who to reach out to. So that leads to the third step. So first is having a cross business, cross functional leadership group. Second, a base level AI training, AI ethics training. And third is changing the processes. If you're using a project management tool, whatever it might be, make sure there is a check the risks we check for when we start a new project. There is the financial risk, brand reputation risk. Add a risk around trust and ethics. If the AI is involved, right. And make sure that is being, getting filled out. So being able to change your processes. And then if an employee has a concern, where do they call? Right. And it is super important that every employee in the organization has the knowledge, understands the processes and is using it. The reason is it's not just your AI team, it's not just your data science team that should be thinking or looking at ethics. There is probably an intern in your marketing team right now evaluating an AI Tool for personalized marketing that intern should know the junior Most employees should know what questions to ask, whether it is just asking more detail training data sets were used, right? They should be empowered. They should have enough information to ask the right questions so that they do that frontline mitigation of any issues that might come up later. And then there should be a process where they need further information. They reach out to the central team who can then help guide. So I think this is these three are immediate steps that any organization that using AI or building AI can use. And I think it's important to point out this is not a challenge just for big tech or companies that are building AI. Even if your company is just using AI, applying it, different applications have different ethical impacts. So there are impacts that will come into it. So it has to be on both the creation side, but also on the applied side.

That's great. And there's one thing that I wanted to also tease out from this particular conversation is when exactly this conversation should start. One thing I've been reading about quite a bit is this concept of shifting left in the context of data is considering reliability, the implications of data and AI systems way earlier in the development cycle. So maybe at what time in the development process as AI use cases are being developed, should this conversation start?

I will answer it in the context of a pillar I have under trustworthy AI called responsible AI. And responsible one is the most, I would say the most philosophical and vague one. But that is where you pause and ask the question, is this the right thing to do? Should we be doing this? Is this the right thing for broader humanity, society for the future? And I think of the classic Jurassic park scenario, right? Remember the quote, just because your scientists could, they did without pausing and thinking about it. I think this is where we have this very powerful technology and if you are planning to use it a certain way, there are long term impacts. So I think being responsible citizens, right, were never part of an engineer or scientist mandate. But now more than ever, I think it is an absolute mandate. Even when you get that first idea of building a certain, whether it's a large language model or whether it is using AI in a certain way, like should I be doing it right? It's a good. I think it should be when you get that first idea, pause and ask should we even go down this path? And look, I've seen scenarios where. And again, I'll switch now more from the philosophical to the enterprise lens, right? Where see, look, I fundamentally believe that all technologists come with a good Mindset, wanting to do good, wanting to change the world for the better. I'm an optimist that way. But as you can probably tell, I'm very pragmatic as well. Right. Because we are not trained to think about the long term implications. We just see the immediate, oh, there is new revenue opportunity or we see a cost saving opportunity. And I'll tell you about a great couple of scenarios. One was this was an enterprise with extreme good intent, wanted to make sure they could retain their topmost employees. And this was in one of my prior roles, very early days of AI where oh, let's combine employee social media activity with their data and see if they're unhappy and just focus on the top most employees and we will make sure they're not looking to leave. Right. Instead of trying to have them leave and then try to retain, let's be proactive, let's use AI. And we're like, wait, but do the employees know that this kind of match, it comes with good intent? Look, when social media started, it was all about connecting and bringing communities together. Great intent, great idea. Right? But if you had paused and thought, but how are we going to make money off this? And what does targeted advertising, what kind of social impact could it have on teenager minds? Right. I think there would have been guardrails put into place. And that's where my optimism comes in. Right. If we can think ahead a little bit and make sure our process are set up in a way where you are thinking and being able to identify some of the ways it could go wrong, I am very confident that we would actually address it. Right. So I think when the initial idea comes, pause and think. And I think it needs to be at that very beginning stage and of course and throughout. Right. This is not something that stops at the idea phase or even design and development. It goes right into your mlops because the way machine learning operates, it is changing. Right. So as model drift happens on the value creation, model drift is also happening on the ethical risk side. Right. So you have to continuously monitor. So it starts right at the conception of the idea till the model gets retired.

Yeah. The road to hell is paved with good intentions, as they say. And given that you're discussing here this, I think a brilliant example of social media. I think one of the reasons why as well the social media, the ecosystem evolved the way it was is also because of competitive pressure. Right. And now we see a lot of competitive pressure as well with generative AI, a lot of organizations within our hype cycle at, at the moment where we are. Everyone's thinking about generative AI use cases and how to deploy them and how to operationalize generative AI. And there's a delicate balance between being first on a use case and trustworthy AI and the importance of creating AI systems that in the long term also benefit humanity. So when deploying AI technologies right now, generative AI use cases and technologies, how should organizations approach this balance? What is a good place to start, in your opinion?

I think having this risk conversation early on, proactively thinking of the ways it could go wrong, is a great starting point for any AI conversation. I think it should not be an afterthought. At the end of the day, it's not like you'll be able to eliminate all the risk. And the idea is to make it an informed decision. And we've heard too long the notion of unintended consequences. And there will always be unintended consequences. But can we reduce that bar down? Right. Can we make sure that we are proactively thinking about the ways it could go wrong and then reducing the number of unintended consequences? We have to move away from the phase of massive scale unintended consequences where there is no rollback option. So I think the idea is not to completely eliminate risk, but make it informed decisions, document it, have those discussions, bring in the experts to talk about it, make sure you're getting the guidance, and not hide under the excuse of unintended consequence.

That's great. And one thing that we discussed and kind of we alluded to throughout the discussion is the importance of AI fluency or literacy and that kind of base level. And I think that base level of fluency enables a common AI language or data language within the organization that promotes that conversation and that cohesion around responsibility and risk. So maybe switching gears a bit and talking about the importance of AI fluency, something that we care about quite a lot here on datacamp, what do you think is the ABCs of trustworthy AI within the organization?

So in terms of dimensions, I think we've covered most of it. And there are two more that I would refer to, like privacy in the context of AI. Right. We have quite few laws around data privacy, data protection, but when that same data is used by AI, how do you untrain a model that was was trained on a certain data set? Right. The nuances that AI brings to the table from our data privacy concept is important to consider. And then the last one is the safe and secure right. AI can actually expose vulnerabilities within your system. And we've seen Scenarios where the chatbot goes rogue or racist. Right? So I think because the way AI operates, it's not your traditional software, where you built it deployed and it consistently produce the same results, it's constantly evolving and changing. The logic is changing. Right. So how do you make sure that you're protecting from security hacks, especially if you are using it for say your jet engine maintenance or you are looking at manufacturing plants? We live in the world where we are surrounded by IoT everywhere, right. If you do not think about AI and the security vulnerabilities it exposes, I think you're walking on the edge over there. So I think thinking about security and safety is important. We've also heard of safety in the context of the decisions that AI makes, right? And I think safety in AI there is a physical safety. We've heard the examples of when it's a self driving car, who should it hit, which pedestrian, that scenario we've all heard about. But it's also about safety from a mental health and psychological perspective, right? What kind of human behavior is going to be induced by using this sort of AI? Right. What's going to be the impact on mental health? Is it going to increase teen suicide rates by using the software that we are building? Right. So I think it is important to think about safety broadly and holistically beyond just physical safety, also on the mental, psychological safety as well, to be able to make sure that your AI product or solution is really trustworthy. The last one I'll touch on Adele, is explainability. And I think it's a tricky one because it's easy to explain an algorithm and how it works. But I think for companies that are serious about trustworthy AI, it has to be explained in a way that your end stakeholder understand, meaning you explain how your algorithm works very differently to your IT or data science team versus to your CEO and her board. Right? Versus your end customer. So I think extra explainability works only if it's understood by the end user. So it cannot be just this one big massive technical document and publishing it and saying we've made it explainable, right? I think you have to get down to that nuance level, especially where transparency is crucial. You have to be able to explain in a way in the language that the end user understands. And here is where I see something like Genai playing a huge role. Think about it, right? Gen AI is extremely powerful and there are many, many good things it can do, right? But you know, think of the explainability being able to explain in different languages depending on Your user base or the audience. I think that's something that can be done with generative AI. Right. So all of these, if the intent is there, if your stakeholder leadership group is prioritizing this, I think technology itself can be actually used to solve some of this.

Yeah, I couldn't agree more. And I very excited personally about the potential generative AI has in unlocking what we think about as data literacy by design. Right. The ability to inherently within the tool provide that useful context. And maybe as we're discussing about the potential of generative AI here, I'd love to also talk to you about where you think we're headed within the next 12 months within this space. So what do you think will be the stories that define trustworthy AI over the upcoming quarters? How do you think generative AI will evolve? And yeah, I'd love to share your thinking here.

So Adele, I'll just start with when generative AI burst into the scene. It's actually not new. Right. Large language models is something that the tech community has been working on for a while. I think we were even looking at it 12 years ago. So I think it just burst into the public mindset in the past few years. And I got a question like, oh, are you going to add a chapter on generative AI to the trustworthy app book? And I said no, everything that's there is still applicable. The foundations are still the same, you might call it by different names. But generative AI has still the same anchoring principles of trustworthy AI. So what I see evolving over the next 12 months, there are going to be a burst of use cases. In fact, we put out a AI dossier of 60 generative AI use cases across different industries. Right. So we do see there are going to be several high impact, low risk use cases that come out over the next 12 months. I think there's also going to be much more awareness on what are the risk factors and best practices that come out from different industries of what works, what doesn't work and where it can be used effectively. And where do we need to put in additional guardrails. Right. I think there will be a lot more real world applications of generative AI. I'm also excited about the fact about how much attention AI regulation is getting now. So I'm hopeful that we'll see more progress on regulations. Of course it takes time, but hopefully this will be a time when most organizations start imposing some level of self regulations and from a trustworthy AI aspect. Right. So I'm very optimistic about where we are headed with AI as long as we consider the downsides and address it.

Yeah. And maybe here you're touching upon regulation you mentioned. How do you see the regulatory landscape evolving over time as AI becomes more powerful?

I think the conversations have been happening for a long time. The challenges and as we've seen in our conversation today, Adele, is that it is not a one size fits all. It cannot be just one AI regulation. There will be overarching principles and overarching ideas on what we align with, whether at a national level or a global level. But at the end of the day it is going to depend on the use case. So I see regulations coming in from two, two broad categories. One is for existing industries, right. The pre Internet era industries like healthcare or bank, financial systems, energy and manufacturing. Right. Where there have been regulations, it'll be more about extending out those regulations to make sure the AI impact is weaved in to that regulation. Right. Or there might be additional regulations that come in a fewer ones. But then there is this other category which is primarily the industries that have emerged in the last few decades, right. Where there have not been as specific regulations, whether it is broader technology platforms or social media. Those are completely new industries and we probably will see more regulations over there given where they are. But bottom line, I don't think it's going to be one regulation. It's going to be very different depending on the industry and the use cases itself.

Okay, that's really great and I think that's a great place to end on Bina, it was amazing having you on the show. Really appreciated your insights. Maybe before we wrap up, do you have any final call to action or closing words to share with our audience?

I think based on your audience and the work that they're doing, I think you hold a tremendous power in your hands with the technology that you're working on. It's time to step up and be responsible citizens about it.

That's awesome. Thank you so much, Meena, for coming on the podcast.

Thank you, thank you for having me. Ada.