EP 141: How To Understand and Fix Biased AI

This is the Everyday AI show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business and everyday life.

Why are AI models so biased? It's pretty bad when, right? Like whether we're talking about ChatGPT or AI image generators or just about any large language model, we see so many biases and prejudices come out in these models. And today we're going to talk about maybe how to understand and fix biased AI. I'm really excited for today's conversation. It's going to be a good one. It's actually a first for everyday AI, which I'm excited about. So, hey, welcome to Everyday AI. My name is Jordan Molson. I'm the host and this is your daily live stream podcast, free daily newsletter helping everyday people like you and me learn and leverage generative AI. So let's today learn about why AI models are biased. But before we do, a quick reminder if you're joining us live. Thank you. If you're listening on the podcast, make sure, as always, check your show notes. We're always going to leave links to some other great resources where you can read more about today's episode and listen to related episodes as well. So before we get in to today's show, let's go over, as we do every day, the AI news. All right. So is AI coming to your hand? Yeah, kind of.

So.

Startup company Humane is set to do is set to debut the highly anticipated Humane AI Pin. So it's a new AI powered smart device that is set to launch today for a price of $699 and a monthly subscription fee of $24.00. So it is a screenless device that aims to go kind of beyond the typical smartphone and by providing a lot of AI features such as translation services and music streaming. So the way this works is you kind of wear it on your shirt and it literally beams the information to the palm of your hand as the display. I'm not sure if I want it because I have kind of like fat, weird hands, so I'm not sure if I want to stare at them all day. That's just me. All right, we have Chip news. Yay. We love Chip News on everyday AI. So Nvidia may be providing chips to China. According to a recent report from the Financial Times, this is newsworthy because there were some recent restrictions on certain chip exports to China. But some leaked documents that the Financial Times is reporting on are showing that Nvidia has developed three new chips aimed at growing demand for the AI Technology in China, but also at the same time complying with these new U.S. export controls. So kind of making both parties happy. All right, last but not least, in our AI News for today, animated films are about to get a lot cheaper thanks to AI so. So recently, the DreamWorks co founder, Jeff Katzenberg said that AI will help cut the cost of animated films by 90%. So be prepared for a slew of hopefully higher quality and even more animated films coming your way, especially as there's these ongoing disputes and feuds with. With the actors unions and the screenwriting guilds and all these things. So probably a lot more animated movies coming our way. And hey, I'm a grownish adult, and I'd say bring it on. I mean, like, have you guys seen Coco? You know? Thanks, AI hopefully we'll see a lot more of that. All right, we didn't come here to talk about animated films. Actually, the probably exact opposite. We came here to talk about how to understand and fix biased AI And I'm extremely excited to bring on our guest for today. But before I do, if you're tuning in, why. Why do you think AI Is biased? Get your questions in now. It's not often we have someone that can talk about biases in AI So I want you all who are tuning in live to get your questions answered. So make sure you get them in and also at the same time. Now help me. Welcome to the show. Let's bring him in. There we go. We have Nick Schmidt, who is the founder and c. CTO Of Solas AI Nick, thank you for joining the show.

Thank you very much, Jordan. I'm looking forward to the conversation.

It's going to be a good one. I'm excited. Real quick, tell us just a little bit about what Solas AI Does.

What Solas AI does is the idea is that algorithms can be biased. They can cause discrimination and unfairness. We want to find out if that's happening and then fix it. What the software does is that it goes into an algorithm and it says, is there evidence of discrimination? And that can be defined in a number of ways. But is there evidence of a problem? If there's not, that's great. But if there is, it goes on to the next step of trying to break open the black box of the algorithm and understand what's driving the model's predictive quality as well as what's driving discrimination. And from there, people can start to make decisions about what whether to include certain things in the model or how to mitigate it. The problem Once you have that idea, then you can start building alternative models that are actually less discriminatory. And that's the big part of it, is that we ultimately have software that's designed to search for and find fairer AI.

Yeah, and I'll say this, there's probably no shortage of work for you all because AI models are, in my personal experience, extremely biased. So I mean, let's start with maybe the why are these models that we use? And without getting too technical, right. So most generative AI systems that we use, whether they're ChatGPT or Google Bard or Midjourney, they're all trained, right? So they're all trained. So why are these models even biased in the first place?

So unfortunately, there's so many reasons for this. And it can really happen at any point in the modeling process. It can happen with the data that you're putting into it that can reflect historical or present discrimination. It can be around the modeling process where you are building a model that says, looking at whether or not you're likely to default on a loan, and the data set you've got is built in, it has discrimination in the outcomes. People who got loans were discriminated against. And so once you take that algorithm and actually go out and put it into production, it can anticipate that someone is going to be discriminated against and discriminated against them in, in anticipation of that. And then finally there's actually using the algorithms. There's ways that you can use them that are discriminatory. Building an algorithm for one purpose and then using it for another is potentially highly discriminatory. And there was one example that I think is really good of a algorithm that predicted health care costs. And it's totally reasonable for an insurance company or someone else to want to understand what someone is going likely to spend over the next year or two years or whatever it is. That way they can have their financial models be appropriate. But some brilliant people figured out that, hey, we could use this algorithm that predicts future costs to predict health outcomes. The problem is that after the Tuskegee experiment, that was up to the 1970s where African American men were intentionally injected with syphilis by doctors in the United States, when that came out in the public, African American visits to primary care physicians dropped by 26%. What that means is that trend has continued, by the way. And what that means is that health care spending for African Americans is much lower than that of whites or non blacks. And so if you're using prior spending to predict future health care outcomes, you're ultimately going to be really underestimating how sick black people are. And so you're going to say that they're not nearly as sick as they are and not give them the treatment they need.

You know, it's, that's a great example that you bring up because it does seem like, you know, and I've even kind of seen this firsthand on more, you know, widely available models, but it seems like there is always a discrimination against certain populations of people. Like as an example, if you ask for, you know, every single week, we put out about a recap of AI news and we use AI image generators. So if we're asking for a tech CEO and it seems like it always, almost always gives you a mid-40s white guy, right? So maybe, why is there always this maybe bias or trends that keep showing up in different models against certain populations?

In that example, a lot of the problem is the input data. I mean, if you look at tech CEOs in the US right now, there's a lot of white men in their 40s, most likely. And so when you train the data or train the model on that data, that's what you're going to get out. And what that means is if you want to change those results. And the important thing about that is that by changing those results, you potentially change the future. You could make a more equitable future if you want to change those. You have to really understand what the algorithm is doing and make interventions in it that will make a fairer output.

Yeah, it's a good point. I mean, how then can AI models find that right balance, right. Of kind of maybe showcasing cultural norms, but at the same time being inclusive, equitable, and truly showing diversity that exists. And maybe not just thinking of image generators. Right. I know that's a very small use case, but even in, I think if you're using a large language model to write content and you have it write you a story, I think you'll probably see a lot of those same trends and storylines. So how do you find or how can models find that balance?

So there's actually a legal framework already that I think is really good for this in the conversation, particularly academic conversation around fair. AI. People are very binary about it. It's either you shouldn't have to do anything, the data's the data, the model's the model, just deal with it, or it's, you have to make everything completely equitable and fair. And while that would be nice, it's oftentimes not realistic. And there's A legal framework within the US that's actually quite good for defining the boundaries. And the idea of it, background of it is that, and it's called the three step burden shifting process. And the idea of it is you start and you say, is the algorithm causing discrimination? And if it's intentional discrimination, then you have to do something about it. But if it's unintentional discrimination, then you move on to a next step which is does the model have a valid business justification? If you are an artist using generative AI, you probably have, you may have a valid justification. If I'm trying to write a memo, I'm not a very good writer, so I have a valid justification for using generative AI to make my, my memos actually look good. But the, you know, credit model, for example, you know, which is perhaps a more realistic one, a bank has a justification for building a good credit model and if they're just giving out loans randomly, they're going to lose money or go bankrupt. So that's the second stage. And then the third stage, and this really gets into what you were talking about, is the, the company has the responsibility to see if it can generate a fair model that still meets their business needs. And so this breaks away from that dichotomy of we shouldn't have to do anything to we have to completely throw out the algorithm or make everything entirely equitable to just can we do better and how much can we do better? And I think that that framework is really what people should try to apply.

Yeah, it's some great tactical advice there for sure. So quick, quick question here and I love this. May Britt, thank you for your question. And if you do have other questions for Nick, make sure to get them in. So May Brit asking, I wonder if it only has bias because AI uses essentially everything from the Internet, right. And those sources have bias and AI has a bias by making assumptions. Super curious how it works. So is that maybe why it is because these models are trained on essentially the entire existence of the Internet and more. Right. Is that why these biases play out in the end? Because the Internet is an ugly, biased place.

It is definitely part of it and is probably the main driver of it. You could even take a step back and say it's not so much the Internet's fault as a piece of technology, but it's our fault as human beings for what we put on the Internet. And I think that actually is a really important point, is that we live in biased societies. So regardless of where you're getting the data from those biases are likely to get put into the data. That's absolutely. So that's probably the primary source of bias, but it's not the whole story. You can also have bias creep in because of model design not using the right model, not using a model that's equally accurate on multiple groups. You can also have bias come in through the usage of the model, like the example I gave earlier. So it's really important. Well, you should focus on the data going in because that's probably the main place you really have to look through the entire pipeline to know where the problem is coming from.

You know, a great let's. Hey, while we're at it, Dr. Muthana here with a comment kind of a question as well, so saying he's looking forward to learn the best strategies and practices to keep AI bias and risk to a bare minimum. Because this is a great point because you know, kind of what, what you all are working on, Nick, is you're working on big picture. Right? But for individual users, I mean, dep, obviously it depends on the, on the model that we use or the large language model or whatever. But what are some maybe best practices for users or can users, you know, end users, individuals do anything to help avoid this bias?

Yeah, absolutely. And I think that the first step is actually asking yourself if your model is fair for everyone and what that means, is the data that goes into it, is it reasonable data? Is it predictive of the outcome? You know, are you using month of birth year to predict whether or not you're going to default on a loan? People do things like that all the time. They say, oh, the data is predictive, I'll put an end. But that doesn't, that doesn't make sense. And so asking yourself, are the data that are going into it okay, or as good as you can get, who are you excluding from the model? And are you putting in data that's sort of collectively punishing groups of people? So for example, zip code data, because there's so much segregation in the United States, if you put information about where someone lives in a model, you're going to get biased results. So asking those questions is the first step. And building a good model is the first step. Once you've got an idea that you've got fair for everyone, then you can start to think about fairness for a given group.

Yeah, and I think that's a good point because there's obviously different use cases for AI and bias in AI goes back to many decades. Right? Like, like AI is not new. I think, but for the average person, generative AI is, is very new, right? The accessibility, the affordability and the use cases are, are all kind of all swelling up together at the same point. But maybe even to ask that a little follow up on that more specifically. Let's just say for the average person, right, if they go into chat GPT, if, if, if, if they go into mid journey, you know, they don't necessarily have any control over how the model was trained. But are there any best practices or strategies to avoid by just, you know, the everyday person using the most popular kind of large language models? Is there anything we can do to avoid that? Or do we have to literally spell it out in the world's longest prompt like hey, I do not want ABC through, through 100.

So I think in the, in a use case of, you know, I'm, I'm going in and trying to write a memo that's not offensive and not, you know, evil or get other kinds of information, the thing you have to rely on is that a lot of machine learning researchers and AI people don't realize is that you're still smarter than your algorithm and you're still smarter and you know more and you generalize better. And so if you are getting stuff out of a model, getting stuff out of generative AI, use a critical eye and ask yourself, you know, what the world looks like, you know a lot about bias. I mean we live through this every day. We see this. If you're a critical thinker, you probably know that. And so that's, you know, in a simple use case like, like just someone using ChatGPT, it's really about not just trusting the output. And when you get into the question of actually building the model or deploying the model, that's where you can start to really get into kind of the quantitative solutions.

Are you still running in circles trying to figure out how to actually grow your business with AI? Maybe your company has been tinkering with large language models for a year or more, but can't really get traction to find ROI on Genai. Hey, this is Jordan Wilson, host of this very podcast. Companies like Adobe, Microsoft and Nvidia have partnered with us because they trust our expertise in educating the masses around generative AI to get ahead. And some of the most innovative companies in the country hire us to help with their AI strategy and to train hundreds of their employees on how to use Gen AI. So whether you're looking for ChatGPT training for thousands or just need help building your front end AI strategy, you can partner with us, too. Just like some of the biggest companies.

In the world do.

Go to your everyday AI.com partner to get in contact with our team, or you can just click on the partner section of our website. We'll help you stop running in those AI circles and help get your team ahead and build a straight path to ROI on genius.

You know, I'm. I'm curious, Nick, and if you're, if you're just joining us midway through, thank you. Welcome. We have Nick Schmidt, the founder and CTO from solas AI, talking biases and in models. My question is, like, maybe we kind of skip to the end, but we also talk about the trend of where things are going, because it seems like model development, large language models, generative AI is being developed at a breakneck pace.

Right?

Like, even early on, there's, you know, thousands of, you know, CEOs and very recognizable people that signed a letter and everyone said, pause AI development for six months, and instead we tripled down. Right. You know, big news. OpenAI just released their newest update to their model, GPT4 Turbo. It's supposed to be, you know, better, faster, bigger, better at decision making, all these things. So if the big companies are just developing faster and faster, and, I mean, I'm sure they're doing things to address their own model biases, but how can you actually fix this, right, when the companies probably have so much pressure and feel they just have to get out more products, bigger models, more parameters, like, how can you possibly fix these when it's just everyone's just sprinting with their.

Head down, well, to make a shameless marketing plug, buy or license solas AI software more seriously, I mean, that's true, but more seriously, what I think is really important is that we've designed software, and we're certainly not the only company out there that have designed software that can be used quickly to mitigate these risks. And so I think that there has been and certainly was true in the past. Mitigating these risks was very difficult and time consuming. We're actually using AI to fix AI, and it's much faster as a result. And so you can still innovate and make fairer models. And I would say that you can actually make better models if you're innovating and considering fairness.

So that was actually on my mind, Nick, because. So I'm glad you brought that up. Right. It's kind of ironic, I guess, that you're using AI to fix AI. I guess my question is, why aren't the big companies doing this? Would it Slow their process down? Would it slow their development down? You know, why aren't they maybe creating a bigger internal focus on fixing this problem before it ever goes out in the wild? Because once it does go out, it takes companies like yours many months or multiple years to really start to even make a dent. Right, because everyone's using these models. So maybe why aren't these big companies just fixing the problem internally first?

So I think that they're trying often, but there's also, there's a big split between companies that are, are and have been doing something for a very long time and ones that are just getting started. So actually in the financial sector, the big banks have been working on this and they've been working with us in my consulting role for 25 years trying to make sure that algorithms are fair. And so there really is already an established practice within, within financial services and to some degree employment. The tech companies kind of came in and said, we're smarter than you guys, we'll figure it out on our own. And you know, it's kind of worked out how it's worked out. And I think that what's going to happen is there's going to be a lot of regulation that is going to actually move companies more and more in the direction of what financial services companies are doing. And so, so yes, this is a problem. And, and why isn't it being done? Why isn't anything being done about it? Well, there's a little bit, but they don't realize that there's already a good story solution out there and I think that'll change. Yeah.

So another great question from our audience and thank you everyone for getting these questions in. Monica asking, can model outputs change over time with users asking for different or more diverse outputs? What a great question, Nick. Is that a thing? Can the outputs change?

It depends on if I understand the question correctly and it depends on the type of model. Most of the work I do is in financial services and healthcare and other industries where the models are not dynamic, so they're not continually updating. You build a model, you put it into production for a while and then you rebuild it later on. And that, that kind of breaks that process of reinforce feedback on an immediate scale. So in the work that I do, no, it generally doesn't happen. But with some of the large language models, my understanding is that there is a lot of reinforcement and back and forth that happens.

Yeah, that's, that's a great point. And also, you know, not, not to put Nick on the spot on all these things but he's not. You know, I think most of us use the chat GPTs of the world and the anthropic clods and mid journey. And that's not necessarily where you focus all of your time because you're working on helping model fairness across the board and not just solely large language models. But it's a good question that Monica brings up because I'm sure users in mass are having that conversation with the ChatGPT or, or with a Google bard saying like, no, this is biased. Please reflect a more diverse output. But I believe over time that it would help them train their models to make them more diverse and less biased.

Yeah. Although there's actually a different way to frame the question that Monica has that I think is important and really good and also shows the real benefit of moving to algorithmic decisioning, which is that when you start modeling, you're usually starting with a. Modeling a human process. So when companies first start using underwriting models, they typically will model what their human underwriters have done. And if there are biases in the decisions that human underwriters make, then those biases immediately get transplanted into the algorithm. But what can happen is, over time, because the algorithm is not intrinsically biased, it can see that those biases are not predictive. You know, it's giving a loan to an African American and they're repaying it, even though it, you know, it assigned a higher probability of default to them or whatever. Well, so over time, what can happen is that those biases can actually be pulled out, out of the system. And so in that way, over time, the use of an algorithm can make things fair.

So, Nick, we've talked about a lot. We've talked a little bit about how models work. We've talked about some ongoing issues with biases and stereotypes that show up over and over in these models. So maybe as we wrap up today's episode, so let's just go big picture here. Like, if you had to say in a very, you know, direct way, how can we fix biased AI? Because we've, we've attacked it from all different angles. But what's the takeaway? How can, like, aside from. Yeah, like, you know, we can use Solace or Solace or products like this, but aside from that, how can we fix the biased AI?

Make a start on it. Don't get overwhelmed with all of the options. Think about what your model is doing and make a decision and test it. See whether or not it's biased, and then choose a metric and go with it. And then fix it, try to fix it. And that's the first step. And once you have that process in place, you can start refining it. Don't let the inability to completely define the problem keep you from doing something about the problem.

Such great advice. And I think that, hey, even if our listeners and our viewers, even if you're not working on a model, I think that what we talked about today is so important because it does take users, it takes consumers, the people paying, you know, to, to continue to have this conversation, to let you know, the, the big tech companies know, yes, this is a problem. Yes, this is something that we care about. We care about fairness in our models. And, and thank you, Nick, so much for coming on the show and helping us dive into this issue. We very much appreciate your time.

Thank you very much, Jordan.

And hey, as a reminder, we did cover a lot, as we always do, so make sure you go to your everydayai.com Sign up for the free daily newsletter. We're gonna have a lot more information from Nick and what we talked about today, as well as just about everything else that's always going on in the generative AI world and more. So make sure you do that and make sure you join us tomorrow and every day for more. Every day AI. Thanks, y'.

All.

And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit your everydayai.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.