#173 Building Trustworthy AI with Alexandra Ebert, Chief Trust Officer at MOSTLY AI

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming.

It can be to keep up with.

The rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com.

Many organizations go with okay again, more regulation. This is going to hurt our business, but particularly in the context of responsible AI. We have seen so many examples where organizations were simply wasting money because they haven't taken care of fairness or explainability right from the start of the project.

Welcome to Data Framed. This is Ritchie. There are thousands of customer facing use cases of AI, from product recommendations to fraud detection to chatbots and beyond.

When these work well, they can bring.

Tremendous benefits for both your customers and your organization. Unfortunately, there are a lot of ways that things can go wrong and bad AI can lead to poor decisions, discrimination.

And a loss of trust.

Today we're going to think about some of the risks involved in AI, like the fairness and bias and privacy problems. We'll also explore the use of synthetic data as a solution to some of the issues around data privacy and as a way to democratize access to data. Our guest is Alexandra Ebert, the Chief Trust Officer at Synthetic Data platform mostly AI, where she advises people on how to create responsible AI. She was recently named in the Forbes 30 under 30 and 100 Brilliant Women in AI Ethics 2024 lists for her contributions to AI ethics. Alexandra is also the Chair of the IEEE Synthetic Data Expert Group and host of the Data Democratization podcast. In short, she knows an awful lot about helping build trust in AI and I'm keen to hear her insight.

Hi there Alexandra. Thank you for joining me on the show.

Thank you for having me, Richie. Very much looking forward to being on dataframes.

Brilliant. Yeah, glad to have you here. So, to begin with, I want to talk about why trust is important. So can you just give me some examples when AI has been used and then trust has been breached?

There are a lot of examples. So maybe the first one that comes to mind was back then when Apple decided to publish a credit card together with Goldman Sachs. And then there were some accusations that this card was actually discriminating against women. And granting the specific case, a wife less credit than the husband, even though she had more financial assets. And in a kind of legal process this was then deflected. But in the end this was something that impacted the trust users had in the AI system that was used. Or also Google who used the labeling technology to label photos that users of Android took. And then suddenly some users figured out that their black skinned friends were labeled as gorillas, which is something that they couldn't resolve for plenty of years. And therefore also he impacted the trust many had in brands like Google of not being able to solve something like that.

It does feel like there are far too many examples of this where AI seemed like a good idea and then something's gone wrong. It's caused like even the best is just like a public relations disaster, but worse, it's caused harm to the users or someone else. So are there any patterns to this? Like why do people lose trust in AI? What are the common themes here?

The examples that I just named, we had discrimination. So one pattern is definitely biases and discrimination of AI. Another one, privacy infringements, if users suddenly figure out that some data was used to train an AI system that wasn't allowed to use. Or explainability aspects when you as a user, for example, are denied credit and you can't get sufficient explanation of why this decision was made, which rids you of the chance to challenge the decision made by an AI.

Okay, so it's like it's getting wrong answers or being unfair in some way or there's some lack of transparency which is causing a loss of trust. So it just seems like there are a lot of sort of these important points that we're going to have to get to in more detail later in the episode, I think.

Absolutely. And maybe also to add to that, I would say how people lose trust already has the prerequisites that people trust AI. But I think today we're in this strange stage where one bunch of the population has this overinflated expectations and belief that AI can do everything and the other one is rather mistrusting of AI. So I think it's also still a process of building up trust in AI, where AI deserves this trust, and then cautioning in the areas of where AI maybe shouldn't yet be used or just with the necessary precautions.

Okay. Oh, it's like, yeah, type one and type two areas like you're trusting it when you shouldn't or you're not trusting it when you should. Yeah, interesting. Okay, cool. So I think related to this, what are the costs or the consequences when people do lose trust in AI.

So I'm going to answer this in two aspects because I work a lot on the regulatory side advising regulators. And I would say on this macro level, we see that plenty of nations or economic unions like the European Union have this objective of becoming a global leader in artificial intelligence. But this is something that will absolutely not happen without people trusting AI. Because without trust, there's not enough adoption and not enough organizations developing, building or using AI. And this is why we also see on a regulatory side, so many initiatives happening, like the European AI act, like the blueprint for responsible AI from the White House and so on. Because responsible AI or trustworthy AI, these two terms oftentimes used synonymously, are seen as the basis to ensure that this trust is here and that we can reach this goals of becoming leaders and having more widespread adoption. So I think this is the answer on the macro level, if we bring this down to the business side of things. Of course, if people are not trusting AI, this could mean that you as an organization have problems with bad press or even legal repercussions if something goes wrong with your AI or on the employee side, that you will never, ever manage to get AI from this PoC and piloting an innovation stage to actually having fully fledged AI in production. Because if people are skeptical and not trusting the way how you use AI, then I'm sure there will be big blockages on the people side. And I think we all know that bringing AI to an organization requires a lot of change management, a lot of people involvement and approval. And here again, trust is just simply essential.

That's actually a really fascinating point that you think regulation can increase adoption. Because I think a lot of the times companies go, oh well, you know, regulation is just something we comply with, we don't want more of it. But actually, if you have bit of regulation to encourage trust, that's going to help people be more confident. They're going to adopt AI and everyone's happy.

You're completely right. Many organizations go with, okay, again, more regulation. This is going to hurt our business. But particularly in the context of responsible AI. We have seen so many examples where organizations were simply wasting money because they haven't taken care of fairness or explainability right from the start of the project. One example was Amazon, who was thinking of using AI to help them make their hiring decisions, whom to invite for an interview. And here it took them two years until they figured out, okay, this model is actually discriminating against female candidates. We can't bring this into production if they had thought about this right from the beginning and figured out ways to mitigate this discrimination, then not as much money would have been wasted and there would have been potentially much quicker bringing out the product.

That's interesting. We had a recent episode where this same thing with Amazon was discussed, and there the guest was sort of saying, okay, well, Amazon did quite well because they actually found out there was a problem and then they shut the system down. But you're saying that actually they should probably have thought about this from the start and built trust into the decision making from the beginning.

Absolutely. I think with everything in the realms of responsible AI, trustworthy AI, ethical AI, it really pays off. And it's also necessary to think about these things right from the beginning before you even decide is AI suitable, suitable for this problem or not. So you really have to have it in there from the beginning, throughout the entire life cycle.

So I'd like to talk a bit about accuracy and what happens when AI gets the wrong answer. Because it's true, sometimes AI does come up with a result that's either unexpected or just downright wrong. So can you talk a bit about what are the consequences of when you get that wrong answer?

That's a good question, because it depends on in which context. Particularly, I would even say that you rarely get an answer from an AI, where also from a fairness context, it's accurate and fair in all respects, because particularly with fairness, we have these different mathematical definitions and you can't satisfy all of them at the same time. So that's one thing to keep in mind. But particularly when you think about getting a wrong answer, I think it's important to first ask the question, in which area, for which purpose do I want to use the AI system? Is it an area where I can live with the wrong answer? And if the answer is yes, then you can proceed with putting AI into this context. If the answer is no, then don't use it. Also, I think it's very important that organizations educate the users of their AI systems about what to expect. Let's take ChatGPT as an example. If I'm, as a user who isn't well acquainted with the inner workings of ChatGPT, assume that this is some type of crystal ball, then potentially the outcome for the organization and also for the user will be quite surprising because the user didn't expect that. ChatGPT doesn't always tell you the truth. So I think it's really important to question in which context you can use an AI system and also whether the people using this system have the necessary understanding of what the limitations of a system are.

It's always a shame when it's like, oh, it's really context dependent. You actually have to think about what you're doing in order to come to a sense. Of course, of course.

It would be more satisfying if I could give you the three easy areas and steps, what to do.

Yeah. So you said something interesting, that there are different metrics for fairness and you can't satisfy them all at once. Can you maybe elaborate on that? Tell me what some of the different metrics for fairness are and when you might consider one over the other.

So from a mathematical side, there are plenty of different definitions. And I think for this episode it's not as important to go through the detailed metrics, but more give the audience a more general understanding what the challenge is. And the problem with fairness is it is not a black and white thing. It's not either fair or not. But if you ask five people, you will get seven different understandings of what counts as fair. So what I oftentimes like to use as an example is my imaginary nephew and my imaginary niece. Let's assume he is six years old and my niece would be three years old. And if I have six pieces of chocolate, he might argue, well, I'm the bigger one. I should get four pieces. My little sister only should get two. You could say that's fair. To some point. She might say, well, I did the dishes yesterday while he was playing computer games. I should get 4. He should get 2. Also a valid point. And then there would be others who say, well, three pieces for you, three pieces for him. That's fair. So you see, there are already many different things and you can't satisfy all of them at once. And therefore, it's super, super important to keep this in mind, particularly because as a data scientist, you will always stand in front of this challenge to satisfy two different general concepts of fairness. Because if we look into anti discrimination laws, what we find there is on the one hand, the principle of trying and aiming to treat everybody equally, not discriminating, for example, based of gender. But then there's also this acknowledgement that historically speaking, we had this history of prejudices, biases, discriminating against people on gender, ethnicity, sexual orientation, and so on. And here, to not perpetuate these biases, it sometimes important to make an effort of what is called positive discrimination, for example, giving preferential treatment to people from the African American ethnicity or to females or something like that. And as you can see, you can't have equal treatment of everybody and course correcting historical injustices and therefore it's sometimes really tricky. And this is also why I oftentimes make this point that this decision shouldn't rest on the shoulders of data scientists alone. It's really a super important, not only societal problem, but also problem for an organization. And therefore I think it's important that not only the data scientists, but also legal experts, even ethnicities, social scientists come together to decide how can we approach this in a given scenario.

Okay, so it just sounds like it's going to be a real sort of cross team effort in order to get the right measure of fairness and then implement that as well. So that sounds like it might be something that's quite difficult to communicate to users and make sure that all your customers have sort of trust in what you're doing here. How do you message this fairness to your customers and explain what you're doing?

That's a great point. So on a more general level, we've seen many organizations coming out with their own ethical AI, responsible AI principles, however they're going to call them here. Of course it's super high level and I think in nearly everyone you will see we take fairness and non discrimination as a crucially important point. Therefore I think it's also important to add up to this and show how you're actually doing this by giving a sort of explanation. Also, in the context of AI transparency, give whenever you use an AI system that users are exposed to a general explanation of what the purpose of the system is, which data points will be used to come to a decision, which training data was used, how you counteracted potential biases is how you observe this. And then of course how to make this in an easily digestible question or in a. And then of course how to make this in an easily digestible way is one of the open research challenges where unfortunately again, I don't have the three points answer that everybody is looking for.

Okay. But it just sounds like just having some sort of system in place to try and measure your level of fairness and being able to at least show the methodology or, or report on some things to users, that's going to go a long way to making them trust that you are at least trying to be fair with your AI. Does that sound right?

Two points here. So I would argue it's not only about measuring because then data scientists oftentimes like to go to the mathematical side of things, look for some metrics to use and then make the check mark that the system is fair. But unfortunately it doesn't work like that, because if you haven't looked into how certain data was collected, you could get the most perfect score on your fairness test, but it still would be heavily discriminating. To give you an example, there's this famous ProPublica case where algorithms were used to help judges come to a decision who should get early released from probation versus who has to stay in. And there scientists found that there was discrimination happening against black skinned individuals. And if you looked at the data that was used to train the system, the algorithm was behaving correctly, it satisfied a specific mathematical fairness definition, and it also didn't do anything unfair. If you looked at the data source, the problem came before the data was initially collected because it happened to be that police was predominantly present in predominantly black neighborhoods. Which of course led to the result reflected in your data that more black skinned individuals commit a crime after first being held in probation. And therefore it was a correct conclusion that it would make sense to keep them in. But of course, if you consider that if more police would have been present in predominantly white neighborhoods, they might have seen the same rate of crime happening, then this would change the picture. So it's never ever enough to just measure fairness and just rely on a mathematical way to do so. But you really have to think hard on how was the data collected, how am I using this AI system, Which parameters am I using in my model, how am I modeling things? Because all of these things can actually affect the outcome. So as we've discussed, quite a tricky problem.

Absolutely. And data collection is one of those things where it sort of in principle seems easy. It's like, oh yeah, you just go out and collect some data and then often by the time it reaches a data analyst or a data scientist, they kind of assume, well, okay, the data is probably fine, this is what I've got to work with, and then the rest of the analysis is fine. But you've actually got a problem right at the start. So I guess related to this. So we talked a bit about how sometimes AI can give you the wrong answer. And are there any ways to sort of mitigate problems with this? Like how do you communicate this to your customers? Say, okay, well, you know, sometimes you get the wrong answer and how do you deal with it?

Can you give me a specific example where you're interested in this answer? Because I think if you keep it on this high level, then I again can only give you a high level answer.

Okay, Medical diagnostics is a good example. So quite often you use AI to assist doctors in coming up with a result. So given that AI can sometimes give the wrong answer, how might the doctor go about communicating this? What might you do to keep trust in the AI system but still have your patient happy?

Yeah, makes sense. So I'm neither a medical expert nor a psychologist to know what's the best way to deliver messages. Even though I've heard that particularly in a marketing context, people actually appreciate getting bad news from the AI and positive news from the human being. Not sure if this can be translated to a medical context though I would say if it's still okay to use AI in this context, which for example would be that even though there might be some wrong answers, overall you would get correct answers or you would perform better than if you only had the doctor without AI assistance making decisions about, I don't know, whether you have cancer or something like that. If that's the case, and if overall it makes sense to use AI here because you can can more positively impact more patients, then I would still communicate that the AI system not always is 100% wrong. And I hope that the specific hospital also doesn't let the AI on its own do the doctor's job, but that it's just used as an assistance tool. And therefore this could go about and explain how AI is one piece of the puzzle that helps the doctor to come to its conclusion. So I could imagine that something like that would be appreciated.

Okay, so it is possible to still retain some trust even if you are getting wrong answers some of the time. Just as long as you're a bit clear on exactly what's going on and explain how AI is not like everything, there is also a human involved.

Exactly. So I think that's one point. And also we oftentimes have this conversation with self driving cars where this question is, okay, what if the self driving car has to make the decision between running over the granny or the group of school kids, which kind of decision to take. That's of course a hard problem. But I think in this context even more interesting question is how does it look on a more overall level? So we have, I don't know the exact number, but we have a significant percentage of people dying in traffic each year. If we can reduce this number to let's say only 10% of this number, with switching to self driving cars, is it for us on a societal level okay to use AI and to use self driving cars, or do we still insist that everything has to be human, including the human damages that we see in traffic? And, and again, this brings us to this point that AI is touching us in so many different levels of our day to day life that it can't be data scientists alone making these decisions. It's important that also on the regulatory side they will come up with how to navigate this current challenges. Because as of now, we still have scenarios where anti discrimination laws and privacy laws sometimes force data scientists to build AI that's for example, discriminatory but legally compliant, or to build something that's not legally compliant in conflict with privacy and anti discrimination laws, but does a better job on the fairness side. And I think this is something that we also need to clean up on a societal level, on a regulatory level.

Okay, yeah. So maybe in some sense it's about benchmarking and saying, well, you're not going to get a perfect AI system, but how well does it do relative to humans? And so certainly with the self driving car example, if the car can drive better than a human, crashes less often, then probably that's good enough compared to it crashing absolutely never for some societies.

Others might have a different opinion. So of course I'm not a judge on this, but I think that's one way to think about it. Yes.

Okay. All right, I'd like to move on and talk a bit about privacy with AI. Can you tell me, like, what are the most common privacy concerns related to AI?

I'm going to answer this on the business perspective. So for businesses when they want to use AI, the most common privacy concerns first are not having access to data because. Because it's not possible to access them in compliance with gdpr, ccpa, other privacy laws, or it's the time to data because they have to go through this cumbersome case by case processes of anonymizing data. And then it just takes what we hear from the organizations we at. Mostly I work with, if they're lucky, a few weeks, but much more often two months, five months, six months, sometimes even eight months if they want to share externally. So these are the two big privacy problems. And then of course, in case they use some data unlawfully, if this comes out, then of course not only the legal repercussions, but also the reputational damage that comes with that.

And with generative AI in particular, I think there are sort of some new worries around the use of data, particularly like people's privacy being violated or in fact like the sensitive business data being sent to the model and then suddenly it being generated in someone else's conversation. Can you tell me how real of a concern this is?

I would say the biggest privacy concern with generative AI and particularly large language models like GPT and ChatGPT. The biggest privacy concern many legal professionals and privacy pros and organizations have right now is their employees actually using ChatGPT and mindlessly typing in some privacy sensitive data of their customers, because that's definitely a privacy breach. On the other side of the spectrum, if I as an individual am concerned about generative AI infringing my privacy, I would, I would say it's a problem. But in the current discussion, it's much more heavily debated whether it's okay that LLMs like GPT scrape the entire Internet, including all the copyright infringements that come with that. Of course, if you have your personal data also publicly available, it quite probably was included in some training data for generative AI. How big of a concern that is on a general level, of course, hard to tell, but there definitely have been some scary examples. So, for example, a US professor who was accused by ChatGPT to sexually harass people, which wasn't true, but since some information was picked from the left, some information from the right, and also his real name, then this resulted in some accusations that nobody wants to have out there about themselves. So that's definitely a problem. And then also in the context of hallucinations, if I get a name from ChatGPT that happens to exist in real life, and then something bad that this person did, for example, it's hard for me to tell whether this actually happened. Particularly with is ChatGPT being so deceptive, giving you, I don't know, fake URLs to New York Times articles, even sources to papers, and something like that. And this brings us back to this point, why it's so important to educate every user that GPT has some of these problems and that generative AI, large language models tend to hallucinate some information and that you have to take it with a grain of salt.

Absolutely. So if you're looking for facts, then probably the generative AI, it's not. You've got to be very careful in terms of using it.

Maybe at least this type of generative AI, there are others, but with large language models like that, in this context, you shouldn't use them as your crystal ball.

Okay, and can you just talk me through, like, what are the most sensitive types of data when it comes to AI? What do you need to be worried about with respect to privacy?

I think a common misunderstanding when we talk about sensitive personal data is that many people have this old concept of personal data in their mind, which is, okay, first name, last name, home address, Social Security number, something like that. But in today's world, where we have big data, where we have behavioral data assets, scientists actually concluded that there's no not personal data anymore. And it's really surprising how easy it is to reidentify customers, employees based on just tiny bits and pieces of data. And therefore it's important to protect your entire data asset. So to give you a more tangible example, one study for example, looked into credit card data and everybody, I would say, has at least a few dozen, if not a few hundreds of credit card transactions per year. And this study found that only three out of these hundreds of transactions per individual are sufficient to re identify over 80% of individuals. But the surprising thing is you don't even need the entire information about this transaction. You just need the date and the merchant. So something like yesterday, Starbucks, the week before Walmart, and two weeks ago McDonald's. These tiny bits and pieces are sufficient to reidentify over 80% of customers. And I think this is something that many people are not even aware of. And why they still rely on legacy anonymization technologies like masking, obfuscation and the like, which by now are proven to not work for these big behavioral data assets anymore.

That is fascinating and also terrifying. I mean the fact that you said that you can identify people just from like a Walmart or a McDonald's purchase, it's not even like, oh, they bought some obscure thing, it's just like whatever.

No, it's really very basic information. And I could give you the same examples for healthcare, for telecommunication, mobility data, even demographic data. Because today we have these much more unique digital fingerprints and much more high dimensional data, which is something that legacy anonymization techniques which always stick to the original data and just try to delete, distort, mask certain elements of this data, simply can't protect anymore.

So in that case you're saying a lot of the standard techniques don't work. What can we do to mitigate these privacy concerns?

Great question again, and this is why I'm so passionate about synthetic data, which is also something that created with generative AI, but in a completely different context than what many people now are aware of with ChatGPT and the like. So AI generated synthetic data is basically anonymization technology that was developed to solve particularly this problem how to protect people's privacy, how to create something that's impossible to reverse engineer, that's impossible to re identify, while at the same time not destroying the utility of a data set. Because you as a data savvy person, but I'm just highlighting it also for everybody else listening today can already guess that if you have this rich amount of financial transactions and everything you as a data scientist get is Walmart, McDonald's, Starbucks and three dates. Of course, the analytical value, the utility of this data set is heavily diminished. And the interesting thing with synthetic data is I can basically have the cake and eat it too. So what happens here is that we work a lot with financial services providers, insurance organizations, that they use a software like ours, a synthetic data generator, to let generative AI train on their existing customer data sets, their existing financial transaction insurance claims. And the generative AI can automatically understand the patterns, the correlation, the structure of the data set. So basically how an organization's customers act and behave. And then in a completely separate step, once this training, this learning process is completed, you can use the synthetic data generator to create new synthetic customers and their synthetic financial transactions, their synthetic insurance claims or their healthcare records. And from a statistical point of view, the real production like data and the synthetic data will be nearly indistinguishable. You just a ballpark estimate or a very rough number to paint a picture. You can basically retain 99% of information. You lose the extreme outliers. So for example, we in Austria don't have as many billionaires as we have in the United States. If one of our Austrian customer banks synthesizes their customer base and they only have two billionaires, they might not be included, but they will still get a much more granular picture of fully anonymous synthetic data. And that's so interesting not only because, because it finally allows you to use highly accurate and representative data for machine learning purposes, to share with startups, to upload to the cloud and use for machine learning projects. But you can also be much more inclusive and much more diverse in what you develop, because you don't only get this average chain or John Doe that you get with legacy anonymization, where you have to strip so much away, aggregate a lot and so on, but you really get granular level individuals down to the very edges of your spectrum of customers. And this also allows you to understand how does this subpopulation of our customers behave, how do they act and behave, and how can I develop products and services that cater much better to their needs and not only to the average customer.

Okay, so this sounds really useful in that you've got some. Well, I want to pretend data is not the right word, but the synthetic data is giving you some of the same properties of your original data set. But it does have this sort of privacy benefit.

It.

I'm worried that if you start making up data, is it going to cause those sort of accuracy and fairness problems that we discussed before?

No, and partially. So synthetic data, well, partially was related to the fairness. Let's start with the accuracy I mentioned. You can't get 100%. This simply wouldn't be possible from a privacy point of view. But you get super, super close to your production data. And this is also why our customers find, when they first test, test synthetic data, and they do this mainly by training a machine learning model on the production data and then a machine learning model on the synthetic data that was created from this production data, and they find that the performance is on par or that they have a deviation from area under the curve of, I don't know, 0.2 or something like that. So it's really a super interesting technology that is as good as their production data. Particularly for those organizations who are in a position of saying, okay, either we don't do anything because we can't get access to data, or we use synthetic data which is so close to the real that we can use it as a replacement. So from an accuracy point of view, there are not really any concerns with synthetic data, because particularly from machine learning, I'm not interested in this one billionaire, but I want to find granular yet generalizable patterns. And this is what still sticks or still stays in the synthetic data. The fairness question, however, with the general process of creating synthetic data, you want to have everything from the statistics, everything from the information in there as in the original. So if your data set originally was biased or discriminating or something like that, you will still find this in the synthetic data set. But that's oftentimes a good thing because in most organizations, really just tiny group of privileged people can even access the production grade data. So it's really hard for them, particularly if you don't have the knowledge to spot any potential bias problems. But once the data is synthetic, you can share it with a much broader group of people, which on the one hand brings in diversity, but also allows you to even share it externally for, for example, with experts on AI fairness, because many organizations don't yet have this experience in house and get their point of view on whether this data is suitable, whether you need to collect additional ones or how you could mitigate this. So in this sense it can actually help with fairness. But you will still have the biases in there that you had originally in there. So you still need to mitigate them once you have detected them. And then there's also this fun thing called fear synthetic data, which helps you to mitigate biases in a data center. Think of a data set where you don't have a gender pay gap anymore, or where you have much more ethnic diversity in there. This works to a certain extent and in some cases can help with fairness, but it's definitely not the solution to end all fairness problems that we have with AI fairness.

It does sound like we're going back to this idea that there are lots of different metrics for fairness and you need to do a bit of controlling and thinking about exactly how you want to get to those targets or how are you going to measure what success looks like.

Exactly, exactly. So if I want to have a better understanding of how many females I have in different management positions in my company, having a fair synthetic data set where everything is 50, 50 or something like that, of course, would give me a wrong picture. So for analytical purposes, it might doesn't make sense to change the real world. But in some areas, you don't want AI to perpetuate historical biases. And for example, building a model where you want AI to suggest which salary you should offer a new candidate there, it could make sense to also tweak the data and make it fairer to improve fairness in downstream models. But it's something that's rather new, and we, for example, have one customer who started experimenting with this, Humana. I also have a data democratization podcast out on this, which we could maybe link in the show notes, where they're looking into fair synthetic data to help them to be much more inclusive in their quest to be more proactive with healthcare and have more ethnic diversity in a data set to make sure that algorithms then allocate health resources more fairly and diversity.

This is fascinating stuff. And it sounds like because you're using generative AI to create the synthetic data, it feels like it's a productive use of hallucinating in some sense.

I wouldn't say that it's hallucinating in the same sense as we see with ChatGPT, because there we sometimes don't even have a clue where this information is coming from. With synthetic data, you're super close to the original. With sphere synthetic data, you're still informed by the patterns in your original data set. So you couldn't use this process if there wasn't any example to learn from. Let's think of a medical study that was only performed on the male body there. It wouldn't make sense to let the AI dream up and hallucinate stuff. But if you have just a few high earning females, for example, but the AI could learn on other high earning individuals, then it's much better in giving you realistic examples that could have happened, but just didn't happen to be in your data set. That so it's much closer to the room of real possibilities and not as all over the place with some other hallucinations that we see.

So let's nerd out for a moment. So this idea of creating synthetic data does sound very close to the idea of imputation where you sort of fill in the blanks in missing data. Are these two techniques related at all?

You can actually use synthetic data to impute data much more effectively than with more common approaches. But no, I wouldn't say that they're similar because I mentioned earlier that legacy anonymization techniques always stick to the original data and just try to strike through, distort, mask certain elements of the data. With synthetic data, on the other hand, you learn the entire data set. So with legacy anonymization, let's say if I had 200 columns in the beginning, I would only have three columns, five columns in the end because I had to delete so much information. With synthetic data, all the information is there and I learn all the information in its entirety to then present you with another dataset where again have all the 200 columns populated. So it's not necessarily filling out missing gaps. You could of course use it if there are some gaps, some values that you need to impute to also fill them out with a much better educated guess than more simple imputation approaches. But in general you need to have quality data to get quality synthetic data. And it's not so much the purpose of imputing information, but rather protecting privacy while retaining the value.

Before we move on from synthetic data, one last question on this topic. It seems like the main benefit is round the privacy of data and you get some sort of benefits around you can control for different types of fairness. Are there any other benefits of using synthetic data?

Actually, plenty. I mean, you're right, privacy is oftentimes one of the reasons why synthetic data is discovered. But it's also this time to data element that I mentioned, which is a huge concern for many organizations today because particularly smaller organizations are quite fast in innovating and the larger organizations, particularly in banking, financial services, are afraid of lagging behind. So getting access to data in a matter of one day or one week, in contrast to having to wait six months as a data scientist, is definitely quite interesting. So it's the time to data, it's the collaboration aspect. Many of our customers use synthetic data to kind of kickstart collaborations with startups or with other partners in the AI ecosystem because it's just so challenging to get data out there. It's also more innovation because in many organizations we still see that data is siloed and only a small group of people can access this resource. But also you with datacamp, with your mission to democratize data skills, see that if more people are empowered with data and this could also be synthetic data in an organization, then this can have massive impact on the capacity of an organization to innovate and do something that they haven't yet considered before. So this is also why more information, more organizations use synthetic data not on a case by case basis to for example, say we have a credit scoring algorithm that's not performing as good as it could be with synthetic data. We can feed in high quality data and we can improve this, but that really provide internal synthetic data hubs or marketplaces to any employee, from the intern to the CEO, so that much more people can become data driven and can really understand who are the customers of our organizations and how could it translate this to innovation. So these are a few of the the points how synthetic data is used today and then an emerging areas, the whole part of synthetic data for augmentation purposes, rebalancing data, fair synthetic data, using it to impute and in the future also to simulate certain things. So it's really quite growing area, but already today quite interesting, not only for privacy.

That's absolutely fascinating the idea that by getting rid of some of the privacy concerns, you've actually improved productivity because more people get access to it and you're going to be able to get results just, just much faster. So you really are improving performance there.

Brilliant.

So I'd like to talk a bit about transparency and explainability of AI, because I think sometimes AI is seen as a black box, but it's not sort of clear like how AI made the decision. Do you have any ideas on how you can make AI more explainable or more transparent to users?

I think there are plenty of different tools out there to improve explainability of AI, but what I see very often with data scientists approaching the problem is how can they build in explainability that helps them to debug a model. But I think explainability also in the context of regulators is really seen as something that should not only serve the creators of AI, but also those affected by it, maybe auditors and so on. So I think one very good point to start is what is actually the information that a user would need to understand to be able to, for example, challenge a decision made by your AI and then can work out from that. And as many people find, it doesn't have to be that detailed, it doesn't have to be that granular. Which is why this excuse of hey, this AI is a black box, we don't know how it is actually operating, oftentimes can't be hold up anymore because what users actually expect is much more high level. But it's an open area of research. So again, not the three points, how to actually achieve explainability.

Definitely it does seem to be an ongoing problem is trying to figure this out. But it does sound like if you can show people what data's going in, that's at least going to make it a little bit more transparent about what's happening. And perhaps this is where synthetic data is going to help you out.

It's on the one hand what data is going in and for what AI is used in the first place, but also which data points or the feature importance are the most influential on a given outcome, and so on. I think this also brings us a little bit to distinction between transparency and explainability, which if you ask different experts you will get plenty of different answers because again, this is something researchers haven't concluded about how to actually define it. But transparency, at least the understanding that I like to perpetuate is more on this general level. How is the system working? For many initially it's just being transparent that AI is used, that I'm affected by AI at the moment. But of course you shouldn't stop there. You should give a general explanation of what's the purpose of the system, which data is used, which data is processed, what's the feature importance, how are decisions being made, which training data was used, how did I mitigate biases? So it's also some overlap to fairness versus the explainability being seen more as a reasoning for the individual decision that the individual output that a model created so that I as an affected person can understand how was this decision about me being denied credit was made versus the overall system explanations, which wouldn't help me to argue and make my case in this context. Since you mentioned synthetic data. Yes, synthetic data can help with explainability, particularly because we as human beings can't reason upon the model to just based on the code of it. We need to have specific examples. And particularly if I as an external auditor, for example, or even within an organization as a Different unit. That's job it is to make sure and assess that the systems that are put out are trustworthy, explainable. I would need plenty of different granular examples to see how does my model behave in different scenarios, how does it behave with particular sensitive outliers? And here synthetic data is so handy because there's no privacy concern, so there's no issue to share it with another department and not even an issue to share it with an external auditor. And yet it becomes possible to argue on an individual level how specific decisions are being made.

In this case, you could make up your billionaire and they've got their three purchases of a yacht and a Lamborghini and whatever else, and then that's okay because you've not revealed the privacy of like that of any real individual.

Exactly, exactly. So oftentimes this is also one finding when it comes to AI fairness. Microsoft Research conducted a survey, I think back in 2021 with leading AI fairness practitioners. Number one problem they had in counteracting AI bias was actually knowing whether the model was biased or not. Because again, anti discrimination laws, privacy laws oftentimes prohibit that you use sensitive attributes. So they have to operate in the blind and don't see if there are some adverse effects happening for specific ethnicity groups or. So with synthetic data you can not only make these sensitive attributes available from your existing customer base, but you could also use it. And there we come back to this fear synthetic data idea to create more diverse examples that could have happened in the real world. And you with your human expertise can assess this, but that happen to not be in your customer base, not in your training data set and therefore something the algorithm, algorithm wasn't exposed to. And I think this becomes quite interesting, particularly with this whole AI assurance ecosystems that are currently created because there are so many organizations that should provide this auditory and certification services and many of them are currently looking into synthetic data to bridge this privacy gap between getting access to a customer's data and making their assessment and their decision.

And so once you start getting into this regulatory aspect, it really is quite important. Important that you're both keeping data private, but also like showing the regulator some stuff and explaining things to them to see what's going on.

Absolutely.

Okay, so I'd like to talk a little bit about the processes involved in ensuring trust in AI. So are there any particular processes that you think are important in making sure that any AI you're building is going to be trustable?

I think the processor side is tremendously important. Many organizations do just think about their AI Principles set out with the list of, okay, we want to have fair AI, explainable AI, transparent, robust, privacy, friendly AI, and then forget about it. So to make sure that this is something that's actually translated into practice, you need to incorporate this in your existing business processes. You need to set up the governance structures, you need to educate people on how to actually translate these high level principles into the day to day data science and engineering work. It's also something that involves the broad group of people. You will, as we already discussed before, run into problems where not individuals, not data scientists can decide, okay, how to move forward, but where it would be helpful to have a group of AI, responsible AI experts, legal experts and so on and so forth to come to this decision. And then one other element which many organizations tend to forget, it's not only about building your own AI, but you definitely also need to have some guidelines and processes for the procurement of AI systems, because this is an area that's oftentimes left out in programs. But as we all know, there's so many AI systems being developed and many more systems will have AI components in the future. And therefore it's important to also think about procurement and which questions to ask new vendors, et cetera.

So it does sound like there's going to be a lot of teams involved in this. Can you maybe sort of iterate, like what are the different roles involved in creating AI or making sure that AI is trustable and just how they interact with each other.

So if you want to set your organization up for success, as with many other initiatives, C level support is something that's tremendously helpful. I also mentioned, of course that you want to have a more holistic view and many different perspectives, not only from different professional departments, but also from a diversity point of view, young, old, different ethnicities, different cultures. This is definitely something that will benefit. So many organizations build up their responsible AI centers of excellence, for example, or even some committees that have to decide about more challenging or high stakes scenarios. Of course it wouldn't work without the CDO and many technical folks working in this department. It wouldn't work without the legal specialists. And I think it's in general something that should touch every employee in an organization, but just with a different depth of information. But we all know that so many people will use AI based systems. And even if I'm just using ChatGPT and not building GPT like systems, I would benefit from some responsible AI information to know how I can use this in a trustworthy manner that doesn't hurt my organization. Or my customer base.

So since you mentioned that this is going to touch basically everyone in the organization, what sort of skills do you think everyone needs related to trust in AI? Like what should everyone know?

So I think AI and data literacy skills at large definitely help. But more specifically diving into all these elements of responsible or ethical AI, so having a general, general understanding of what the challenges with fairness are, why this is not such a black and white issue, but really tricky explainability, privacy and so on. And I think most importantly this general understanding of that many of these aspects can't be tackled at one stage of the AI development or deployment life cycle, but really have to be considered throughout the process. So I think that's the kind of entry level information. But in general I would wish for people to look more into AI ethics. Even though it might doesn't sound that interesting at the beginning.

I feel like AI ethics is sort of thing. It's good to have like a late night argument with people about. So it's probably, yeah, it's worth having a bit of a learn about it, if only for that. Okay. And on the more technical level, like once you got past the sort of the basics, particularly for people who are in data roles or machine learning roles, what sort of skills do you need there to make good use of trustable AI?

I would say more in depth ethics if you really want to build it yourself. But also again, bringing this back on the meta level, I think we need much more responsible AI talent. But with the speed of the development that is ongoing and also with this ambition of many nations economic unions to have significantly more organizations using AI, we need to have AI governance and responsible AI assurance ecosystems because otherwise this is not going to work out. And I think in this puzzle also the general purpose AI systems of the big tech companies will have an important role to play because I would assume that in the coming years, years, I don't know, 80, 90% of all AI systems will have some of these building blocks from the big tech companies in there. And they are currently some of the best equipped institutions to not only make sure that their building blocks are adhering to responsible AI principles, but also developing new tools that make assessing and achieving AI fairness, explainability a breeze for the end users of the systems or the developers of the system. So I think that's one area where I really hope that we will see much more coming from.

Okay, yeah, that would be nice if there were tools to sort of help you out with maybe some of these ethical issues and fairness issues, things like that.

Exactly. And we have many of these tools today, but it's still very early stage, and many still think, okay, if I use this specific tool, then I could put the check mark under the fairness questions. It's much more complicated than that. And really having some more comprehensive holistic tools that help you to develop it in a. A more stringent manner, I think this is something that's still needed. But we still have a vast open area of research, so we don't yet have all the answers to that. But I hope that more of these will come in the coming years.

Okay. Yeah, it does sound like maybe the mathematical side of this is the easy part. And the tricky bit is really thinking about, like, well, how does ferns apply in this particular context?

Yeah, absolutely.

Okay. All right, so before we wrap up, is there anything that you're particularly excited about?

Most industry, Anything I'm particularly excited about. I mean, in general, I love the world of synthetic data, not only because it helps you to reconcile data utility and using data, democratizing data with the privacy side of things, but also for this impact on responsible AI. And what I'm personally excited about in the next few months is definitely the ongoing work with different regulators being allowed to advise in some responsible AI contexts, and also all the keynote speaking and podcasting that I get to do because it always exposes. Exposes me to so many brilliant minds. And this is something that I really appreciate.

Fantastic. Do you have any final advice for people who want to improve their skills or learn more about trust in AI?

Go to Data Camp.

That's the best answer. I like that.

Not sponsored. No, definitely. Great way. How you approve, how you approach data literacy and of course, educate yourself on AI ethics in particular. And hopefully there be will be plenty of courses on this topic on Data Camp, including synthetic data in the future.

Absolutely. All right, on that note, we'll wrap up. Thank you very much for your time, Alexandra.

Thank you for having me, Richie. It was a blast.