EP 546: AI Bias Exposed: Real-World Strategies to Keep LLMs Honest

This is the Everyday AI show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business and everyday life. Do you ever just blindly copy and paste what a large language model gives you? Right. I get it. We're all overworked, we're stressed. There's so many things you're, you know, your manager is demanding more now that you're using AI, but that can actually be very dangerous, right? Just blindly trusting what a large language model like Chat GPT or Gemini or Copilot or Claude spits out. And one of the biggest reasons, and I think a reason that sometimes AI fails, is because of bias. Right? Essentially, large language models are a reflection of the Internet, they're a reflection of society, and there's a lot of things wrong. And sometimes these models aren't the absolute truth, sometimes they're very flawless. So we're going to be talking about that more in depth today, as well as what you can do about it and how to keep an eye for different types of bias. Biases. Right. I guess that's how it's said that can show up in your large language models. All right, I'm excited for today's conversation. I hope you are, too. Welcome to Everyday AI. Maybe it's your first time here. If so, where you been for the last three years? We do this every single day. My name is Jordan Wilson and this is your daily live stream, podcast and free daily newsletter helping everyday people like you and me not just learn AI, but how we can leverage it to grow our companies and to grow our careers. I want you to be the smartest person in AI in your department, at your company. So if that's what you're trying to do, you're going to want to go to our website. That's your everydayai.com. we're going to be recapping today's conversation as well as really recapping everything else you need in the world of AI. And we do that every single day in our free daily newsletter. So if you want more insights from today's show, make sure you go sign up. Let's talk cognitive bias, because I think so many people are just blindly following what comes out of a model, not even knowing some of the dangers that you might that that entails. So please help me welcome to the show. I'm excited for today's conversation. I hope you are too. So we have with us Anatoly Shillman, the CEO and co founder of Cog Bias AI. Thank you. So much for joining the Everyday AI show.

Thanks Jonah for having me.

All right, I'm excited for this one live stream audience. Thanks for tuning in. Big Bogey and Michelle and Marie and Jamie and Vincent and everyone else. If you have questions, get them in now. But let's start at the top. Anatoly, so what, what is cog bias? What is it that you all do?

Well, we built a platform that detects and mitigates cognitive bias and communications. We what we what started off as our own internal project to help us ask better questions during customer discovery is now to turned into a full platform. We're able to take people's questions for customer discovery, marketing research, NPS scores, assessments, anything and give them a breakdown of the biases they may be facing within the questions they have. And also on top of everything else, we rephrase and give them better suggestions on how to do it better. The same now applies for their emails. So if you have that difficult email to write, say gotta break some bad news or an angry mail, you know how they say wait 24 hours to write an email. But our client, you can actually do write that email. And our system based on the basically the actual context you give it will rewrite it for you in a better way and also tell you what was problematic about your original email. And we found that a lot of folks such as salespeople, obviously marketing research, UX UI product managers have been using our product. And it's funny, one of the things we've been discovering that people are constantly creating new methods of using it. One of the things that's coming out very soon, as you mentioned, AI agents is that we are actually able to audit AI agent conversations and detect the biases that they have and make reports for companies to make the changes necessary to make them better.

So I want to kind of skip to the end here and this is kind of how I started off the show, right, Because I think so many people just blindly either copy and paste what comes out of a large language model or they just inherently trust it as, as being accurate and factual and you know, bias free. Why is that a mistake?

Well, the big up is because just like, you know, AI is like people, it was built by people, just like Google was built by people. You know, before we had the whole AI explosion and people went on Google, well, it's on Google, it must be true was a very constant refrain that people gave. And that's just absolutely not correct. And I think one of the things that you'll discover in AI more and more Often we call people are calling hallucination. It's really more of a form of BS. There's actually a very popular paper called ChatGPT is BS and it was written simply from the perspective that, you know, AI is really more like your know it all friend. You know, everybody has one in their circle. They tell you all these incredible things and most people are like, well, they know everything. They expect, accept it as fact. But what the AI has to do is they have to answer your question. So even if they can't find the answers, they'll make the answer up. Obviously many lessons in that. Most recently, one of the bigger law firms in America had a scandal where the AI that was being used for them to do casework with actually created precedent cases on its own. So it created a whole universe of the cases that never existed. It's a consistent theme over and over again. What we end up with is people are so enthusiastic about this new leap in innovation that they forget that just like anything else, you should have an attitude of trust but verify. I recently did a TEDx event and the conversation there was, you know, the future we make and the future that is AI. And what was most interesting about it is the initial when I asked people how many of you are enthusiastic and excited about AI? You know, like the whole room raised their hands. It was very exciting. But as soon as I started talking about some of the things that have been observed and it started asking people individually, like, well, tell me your real thoughts on AI. It was always a trust but verify attitude. And I think the issue is a lot of time people think, well, you know, obviously look, it's made by OpenAI, it's made by Microsoft, it's made by Google. It has to be good. But they're just like, they're not infallible. They can make the same mistakes. And because they're built by engineers, they have the biases that those engineers have. So they have the same human extensions of our personalities. So the way they gather the information, the way they disseminate it is reflective of humanity.

So I think maybe let's break this down piece by piece or, you know, we'll, we'll go chain of thought on this episode title here, right? But you know what, what is cognitive bias? Right? I think all people kind of understand what it is. But what is actual cognitive bias?

The best way to kind of think about cognitive bias? You know, there's a really long scientific definition that I will not bore you with. It's honestly irrational beliefs based on our perception that's the best way to kind of extremely simplistic, mind you. So please, none of the psychologists in the crowd yell at me, but I'm just trying to make sure that it's something easy to understand. And what I say about irrational is that we don't think clearly when we have certain beliefs, right? We try because our ability to have cognitive bias is what gets us through the day. Ultimately, we make a choice every morning when we get up to get dressed, a certain way to do our hair, a certain way to drive a certain type of car and everything else because of the way either we want to be perceived or we perceive ourselves or the feeling that it gives us. These are all biases. The key point of cognitive biases, they're not bad. They're just part of our humanity. So in some cases, you know, most, some of the most well known biases out there, confirmation bias, framing bias, availability heuristic, those are things that help us and hurt us whenever the situation calls for it. You know, sometimes availability heuristic is reach for the first thing that is closest available to us to solve the problem that we have. So in some cases it's a hammer to nail to nail a nail to the wall. Other times it's going to be a flat object because that's the closest thing to us. It's kind of the same day. We operate with a lot of the things that we do. So choices from a perspective of, hey, I need to get in, I have to send out a survey to my customers. Let's ask ChatGPT for the top 10 questions about car buying. ChatGPT spits out the questions and bam. All of a sudden you get through a whole process where it becomes, you know, here's the questions and you're like, well, they sound good to me. They're perfect. There's no breakdown, there's no analysis, there's no belief.

So I want to break down two key words I heard you say there. So, you know, you said irrational beliefs based on perception. So beliefs and perceptions, right? Because these are things that most people probably don't think go into large language models, right? Beliefs and perceptions. Those aren't fact based, those aren't scientifically researched. How does that happen? And how can people be on the lookout for when that does come through a large language model?

Well, I went through it a simple way, right? The first thing is it happens because humans are the ones who make it. So even if AI makes another AI, it's based on the original programming of the human human. So you're just Going to have a new permutation of the same biases or an evolution of some new biases based on the old biases. The key thing to understand is like, you know, for instance, an engineer will program an AI and say, I want you to put the information out this way and I want you to put. When they ask for a list, this is how the list will be based on this thinking. And when you're pulling from news sources or media sources, this is the first 500 you're going to look at before you look at anything else to solve the problem. Is it because of their personal beliefs? Is it because of their perception of what's reputable versus what's not as a new source? We don't know. Right. And the same applies on the other end as it kind of goes through the process of coming up with answers. If it can't find it in those 500, and I'm just making up that number, I don't really know what the real secret sauce is in those cases. All of a sudden it becomes a situation. Well, they're gonna. If they can't find in those 500, it may think the other ones are less reputable. So instead it'll come up with its own. An answer it. Or it will add its own little spin to it. And because it's AI, you think, well, it's a computer that answered. It must be correct.

Yeah, yeah, that's, that's the worst thing you can do with a large language model, right? Is this like, oh, it's a computer, it has to be. Right. But so many of the things that we ask large language models, there's nuanced. Right. It's not binary. We're asking it for strategy to make decisions. We're not necessarily always asking it to count the number of Rs in Strawberry or, you know, the capital of, of Illinois. Right. But maybe if you could, could you walk us through just what are the types of biases and maybe just briefly. I know, confirmation bias, right? Maybe. Could you walk us through briefly two or three of the most common types of biases that show up in large language models and what they mean?

Yeah, obviously confirmation bias is probably the most well known one. It's confirming its own initial beliefs. And quite often what it'll do is the way best to consider is not from the point of the AI, it's from the point of view how biases really impact us is our perception of what is being said to us, shown to us, et cetera. So AI is going to respond to us in a specific way and bias us in that way. So in some cases we'll ask it a certain question, it'll respond back and it'll trigger our confirmation bias because it's going to be confirming our facts. So if we ask an AI question that has an obvious answer, it's going to spit it back out at us in a specific way. Just make it, you know, prettier, more or less or more sophisticated, or expand on it more. So confirmation is a big one. Framing bias. We frame something in a specific way to get a specific answer back. So if we say just make it up, Mercedes Benz is the fastest car and the gross car for the money based on the luxury, blah, blah, blah, and then we're going to ask questions about it. Now the AI is going to be responding back in the same way just like humans would, because again, the AI is not here to argue with you. I know we've seen those comical stories where AI starts arguing facts with you, but that's not really the reality of how it operates. And then obviously I talked about availability heuristic, which is, which is one of the most interesting ones because like I said, it's the lowest hanging fruit, basic way of saying that people reach for the first thing that's available to them that might seem like it solves the problem.

Yep, you, you bring up a fascinating, you know, point here that I want to dive a little bit deeper in. Right. So when, when models are, you know, essentially mirroring our, our own beliefs. Right. But I think what's important to call out is, you know, a system prompt. Right. All large language models have system prompts. And one thing you said, there is most of them, they are designed to be a helpful assistant. Right. So even if there's not an answer, they kind of want to be helpful. And I think that's why sometimes you get halfway answers or, you know, things that maybe you, you look at and you're like, is this right? Well, sometimes it doesn't always matter because it's ultimately trying to be helpful. But you know, I want to ask you, how does the conversation in the context of, of a large language model when we're, you know, whether it's co pilot or chat, GPT or whatever, how is that going to influence it? Like actually how we're prompting it and, and you know, what the outputs we get in terms of bias?

Well, it's, it's a huge, it's actually a big, big factor if you think about it from a perspective, say you for a specific element or a specific answer. And then you say, well now I want you to write it, but pretend you're a 20 year experienced engineer and write it in that term, but write it nicer. So now there's the bias element of what is truly a 20 year engineer. How do you write nicer? What is nicer? Sheer definition elements and how the biases are perceived from then becomes a hot mess. And quite often that's where the prompting kind of falls apart. And that's why for a while people are like, well, we don't have to know how to do prompting anymore because AI is so smart. I'm like, unfortunately we do. Because the one thing that AI claims to do that it actually doesn't do is it doesn't really understand. Well, it understands in a very, initially, in a very bare bones thing. I heard a very great quote yesterday at an event where they said this, they said, at this point in time, AI is the worst it's ever going to be. And that's a, it's a true statement. Right now we're at the very beginning, at the very earliest stages. So quite often people have expectations of a flying ship when we're probably somewhere closer to a horse drawn carriage by AI standards. We'll get there. But the problem is, as the more things get sophisticated, the more complex they'll get from a perspective. How do we ask a question that it's perceived by the AI in the right way? Because we'll say write it nicer. So it'll change a few words, it'll sound nicer to us. But if the context is still something that has harmful biases in it to what our objective is, it was not really helpful to us. No, it was just an answer given because they're right at night. So fine, I'll put some puffery around it. I made it nicer. And so our prompting doesn't necessarily help it be better at its job. Our prompting just helps again, confirmation bias helps us confirm that, helps it confirm that we want something nicer. So it will change the stone to its perception of nicer, but not necessarily solve the actual problem.

This podcast is supported by Google. Hey everyone, David here, one of the product leads for Google gemini. Check out VO3, our state of the art AI video generation model in the Gemini app which lets you create high quality 8 second videos with native audio generation. Try it with a Google AI Pro plan or get the highest access with the ultra plan. Sign up at Gemini Google to get started and show us what you create. Maybe talk about the root of this, right? Because you know I kind of, I, I, I kind of reference that, you know, large language models are a reflection of the Internet and that's a reflection of, of humanity and.

Right.

And that's why there's sometimes stereotypes and, and biases, you know, to begin with. But walk us through, how are models, like, actually reflecting these biases in the long run? So maybe can you just walk us through training data and like, where do some of the, you know, issues in terms of cognitive BIA get inserted into this whole equation when it comes to training data?

Well, I mean, right. We kind of talked about the idea when they even begin the telling the AI, this is how you're going to parse out information. This is how you're going to pull it apart. This is how you, based on these requests, this is how you're going to gather together. Then we have to start thinking again once those issues are inserted and once the analysis start thinking, what's a reputable information source of information? And then it starts trying to pull that data out. We still have to consider the element that it's very much the equivalent of drinking from a fire hose. I can't even imagine the sheer amount of petaflops and God knows what other measurements we could apply of information that are constantly flowing through that it has to parse through to delineate whether or not the one specific minuscule thing that we're asking has the answer for it, you know, and it still returns it in seconds. And so that stuff is where you really start falling apart on train data, because it's not, you know, there was recently a big thing, but deep seek, you know, how they trained it for 5.6 million. Obviously it's not true, but it was.

A cute episode on that, but thank you for calling that out.

Yeah, it was, it was a cute, it was a cute number though. Right. And, but the big thing about it is that kind of brought to the forefront what does training really mean? Right. And when we think about training a small model, it's literally us humans sitting down and working through the labeling elements of specific data points and how the AI should treat those elements based on the labels we assign to it. In the case of a massive model with massive, massive, massive amounts of data, it labels it itself. So it's only thought to label things. And what's the issue when you attach AI to label things? It you wish it was more like all colors of the rainbow, being able to see all points. It can, it's much more limited. It doesn't have that neural depth that we have right now. So instead what it does is it's more or less if and then rules. A lot of those are applied. And I'm simplifying way too much. That's not really how AI is, but for the basis of understanding, it's really how it kind of perceives information. Does this answer the question? Yes, no, next. Does this answer information? Yes. No, next. And it goes through that whole routine and when it gets to a certain point where was like, well, this partially kind of answers the information. If you extrapolate this and then kind of, you know, smooth it out, which is your BS factor, that's the information. Yeah, so. So in every hallucination or BS point that AI makes, there's always elements of the truth in it, which makes it so convincing. And that's the biggest thing to consider, like during the training, because when you consider what's the training, you know, they do spend months and months, substantially years, training models. But it's not like they're hand labeling stuff. What they're really doing is just overseeing this enormous model, trying to label everything and then doing audits and checking and checking again and rechecking and seeing like, whoa, that's way wrong, you know, and if it's big enough, they catch it. The problem is sheer amounts of data is just not possible right now to catch everything. Will it ever be? I can't possibly answer. I don't think they can either. And that's why, even though you see these new evolutions, and we saw these new evolutions in agents, literally almost every week a new tool comes out and it feels like it's the next tool and the next tool, the consistent theme is the same. They're not actually creating better depth, they're creating better response time, maybe a lower latency, cheaper. They're sometimes making a better conversational piece to it, but the info being put out is still very much the same. And for us, when we actually look at the idea, how it measures cognitive biases versus the scientific models, we have consistency of ChatGPT and cloud and a couple of the others were around 30 to 40% versus what we do. And the reason being is because again, sheer amounts of data and how you label it and how the scientific application actually applies to that, to a specific word or specific nuance in the sentence is not the forte. So when you see like where AI is floating, we have the general AIs. And I think you probably might have talked about this already in the past, the explosion of narrow AIs that are going to be good in Specific elements and that's going to be their main motif.

Are you still running in circles trying to figure out how to actually grow your business with AI? Maybe your company has been tinkering with large language models for a year or more, but can't really get traction to find ROI on genai. Hey, this is Jordan Wilson, host of this very podcast. Companies like Adobe, Microsoft and Nvidia have partnered with us because they trust our expertise in educating the masses around generative AI.

Get ahead.

And some of the most innovative companies in the country hire us to help with their AI strategy and to train hundreds of their employees on how to use gen AI. So whether you're looking for chat, GPT training for thousands or just need help building your front end AI strategy, you can partner with us too. Just like some of the biggest companies in the world do. Go to your everydayai.com partner to get in contact with our team or you can just click on the partner section of our website will help you stop running in those AI circles and help get your team ahead and build a straight path to ROI on gen AI. So a great, a great question here from our audience and you kind of mentioned about the future of AI, right? Like everything going, you know, agentic or multi, you know, multi agent environments. But you know, obviously now we have these models that, you know, think, these models that reason, these models that, you know, take their time. So good question. Question here from Cecilia asking, you know, what do you do to detect cognitive bias and encourage maybe slow thinking versus the fast thinking when we want AI to be fast. Yeah. So like I'll even add on to her question, you know, do reasoning models, you know, that take their time to think? Is that also a process that maybe we're going to see less bias?

We may potentially. But again, it still falls back on the developers. Right. It's a, unfortunately it's like a wheel. As soon as you insert humanity into the wheel, we have our biases, all of us do. That's not a bad thing. Like I said, it's just unfortunately how we perceive information and other things will dictate some of the more harmful biases that pop up. Which is quite often when you hear about stereotypes and stuff. It's not that the engineer wrote, yes, men are better than women or some other thing. No, it's literally their biases and how information is parsed. What gets priority over what causes the model to extrapolate into the next piece, which is, well, this is how I'm going to perceive things. And while they have hundreds upon Hundreds upon hundreds of engineers constantly looking and auditing and checking. It's just, again, such a vast amount of information and such a vast amount of variance that's not possible. To Cecilia's question, what you're dealing with. There's a great book by Daniel Kahneman called Thinking Fast and Slow, which kind of is. I don't want to say he's the father of cognitive, modern cognitive bias, behavioral economics. He is, though. Right. And so the best way to kind of consider it, there will be a space for these models that are more analytical and focused on things. It's the same thing as we have other things that do certain more complex things. I always liken it to the idea, you know, people have an Apple watch and then people have a manual watch that still has gears in it, and they prefer that element because they feel to them it's more reliable. Because it doesn't run out of battery. Yeah, it just moves because you move.

Yeah, I think. I think that's a great example. And, you know, we'll definitely put that in the newsletter. Just kind of about the, you know, system one versus System two thinking. I think it's important, you know, when we think about, you know, using AI. Another good question here from Douglas asking, are there some models that have more inherent bias than others because of how they were trained?

Honestly, that's very subjective. Unfortunately, all of them have bias, and there's no such thing as harmful bias. It's simply their perception of certain information points, how you ask questions. Right. Quite often what we do is, what we notice with a lot of our users is they actually will generate stuff on, like, chatgpt dovetail, name a system that generates questions, and then they'll run them through our system, and then that will be the result that they use for their actual product. And the reason why they do that is sometimes they're fine with the questions. They just want to understand, how am I going to be biasing my audience? Because no matter what you ask, you're always biasing them some way. But the question becomes, am I advising them just to answer the question truthfully? If I'm doing marketing research or if I'm doing sales and I want to prompt them to action, am I doing that effectively? So those are the things you're really pushing on. But right now, for the most part, like, for instance, you know, I think if you look at size of model, large language models have a lot more inherent biases than smaller models. And that's just sheer. By the sheer amount of data they consume. And also the sheer amount of hands that touch the model. At the Same point, narrow AIs totally have a bias in them. And the key element that we try to do, you know, I've always been a big proponent of having a very diverse team look at the model and label it and look at everything else and how it perceives data. And that's the reason why, because if you have a large variety of people look through it, you're not going to have the perspective on how to parse data from one or two individuals or 10 individuals. You're going to be able to do these things in a much more almost objective way. To a degree. It was the proper word when we developed our model, for instance. And again, narrow AI, right, very different. We started with 17,000 questions and we looked at their sentence structure, and from them we're able to extrapolate to 450,000 questions. Somewhere in that neighborhood, probably even more. It's one of the numbers I heard. And the idea is now we're able to label those 17,000, give the model the nod of these are the scientific facts of science, those cognitive biases. This is how you're going to react to this, based on the scientific definition, not our perceptions, but what science defines. Now, does that mean science doesn't have biases? No, but science based on the best thinking we have in existence for our society, this is what it is. And as it evolves, we evolve. This is how the bigger models do too, because as they keep coming out with new versions, it is our hope that they keep updating that particular element of how they model things. But the question I have, and I think everybody has, is with such a fast evolution cycle, I mean, we're talking sometimes 30 days between new releases, not with less. Obviously, a lot of that stuff is maybe quick fixes, but you always have to wonder what's going on in the background. Are they actually fixing domain issues or are they just adding to them by creating more features?

Yeah, yeah. And I think sometimes you spend time circumventing some of those shortcomings. And then new model comes out and it's like, okay, what about all that work that we put in to build bridges around, over or through some of these inherent problems. But we've talked about a lot in today's conversation from everything from bias and training data and bias in the humans that are building it, the types of cognitive bias, which is super helpful. But as we wrap up here, I think I'm going to toss this over to big bogey here on YouTube. I think this is a great way to wrap the show. So he's asking, how do you remove unwanted bias when that bias may be deeply entangled with essential data? And I'll even say, hey, aside from using your platform, how should companies be tackling this? Because it's a huge issue.

Well, I think the element has to be, is the value of the data and the value of the output of the data. Right. Because quite often if you're looking at specific data where the bias may, for instance, affect the key decision the company is making, getting outside help will be essential because, and obviously my platform can help with a lot of the question stuff and the other stuff, but when you're looking at the bigger pieces, like there's consultancies like Forcipio out in California whose entire stick is to look at cognitive bias and how it impacts decisions. Because the elements that he's talking about, essential data, may have cognitive biases deeply entangled in it. But ultimately our perception of that data is what causes the biases that will impact our decision making. The data will bias us in a way, but we have to make the choice of how we receive it. And if we're already aware that we may have a problem with it, that's when we have to seek outside counsel on it it and be able to almost bring a pair of eyes to oversee our process and to figure out if we need to have a more objective, a subjective process or objective process to how we make choices. Because ultimately, again, humans, right, trust, just like trusting a machine. Well, it's not a great idea, especially when the data is complex, complex or maybe to a degree, maybe very, very much human related and something that an AI could not possibly comprehend the same way then it becomes a situation. You have to make your best choices. There's a lot of good classes, a lot of good reading, a lot of good curriculum. I've always been a huge proponent of companies doing behavioral science and behavioral economics training for the very reason, not because it can replace tools like mine, but it can enhance people's ability to be able to spot the issue and then seek the solution versus, you know, finding out after the fact. Oh my God. Our sales calls and our sales meetings and the way we extrapolated this data from these sales numbers was completely wrong. The customer didn't really want this. They just felt they had no choice until they found a better solution.

So much good advice there. I, I, I think, Anatoli, thank you so much for taking time out of your day to join the show. You helped us, I think, make much better sense out of a very complex and a very important topic that we all need to understand. So thank you much. Thank you so much for your time and insights.

Thank you so much for having me. And I really appreciate the podcast, honestly enjoy it quite a bit.

That's great. Hey, I do too, but you know, it'd be weird if I didn't. So hey, if you enjoy the podcast too, if you heard something here from Anatolian to like, wait, what was that? Don't worry, we're going to be recapping it all in our free daily newsletter. So if you haven't already, please go to your everydayai.com Sign up for that free daily newsletter. If this was helpful, tell someone about it. Please subscribe to the channel. Leave us a rating. Tell your friend. Tell your mom. Tell your neighbor. Tell your mom's friend's neighbor. But more than anything, make sure to join us tomorrow and every day for more Everyday AI. Thanks y'. All.

Thank you.

And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit your everydayai.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.