Fighting bias in AI (and in hiring)

Bandwidth for changelog is provided by Fastly. Learn more at fastly.com we move fast and fix things here at Changelog because of Rollbar. Check them out@rollbar.com and we're hosted on Linode servers. Head to linode.com changelog this episode is brought to you by DigitalOcean. They now have CPU optimized droplets with dedicated hyper threads from Best in class Intel CPUs for all your machine learning and batch processing needs. You can easily spin up their one click machine learning and AI application. This gives you immediate access to Python 3R, Jupyter Notebook, TensorFlow, SciKit and PyTorch. Use our special link to get a 100 credit for DigitalOcean and try today for free. Head to.co changelog once again.co changelog. Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show@changelog.com community. Follow us on Twitter. We're PracticalAI FM. And now onto the show.

Welcome to another Practical AI Chris, I know that you've had a number of jobs throughout your career. Was the hiring process always super smooth for you?

Anything but I've been hired more than a few times and I've hired lots of people over the years and no, for me at least, way, way more art than science. So I'm looking forward to maybe learning something here.

Yeah. We've got Lindsey Zulaga with us. Welcome, Lindsey. Hi.

Nice to be here.

Yeah.

Did I get the name right?

Yeah, you did. Good.

Okay, perfect. Well, I'm excited to have you on the show today. I know me as well, kind of with Chris. I've had some awkward experiences in the h hiring process. I've done well at interviewing. I've crashed in the interviewing process. I've done well and bad at assessments and coding things. And. And we're just excited to have you because we're going to be talking today about your work with AI and hiring and also bias in AI. So super great to have you here. It'd be great if we could just hear a little bit about your background. I know you started out in academia and then eventually moved into industry, so give us a little bit of your story. Sure.

I studied physics, so I did my undergrad here in Utah at the University of Utah, and then I did a master's and PhD at Rice University in Houston, Texas, and a postdoc in Germany as well. So during that time I was in the field of nanophotonics. I was studying and doing experiments on how nanoparticles interact with light. So building laser setups and kind of pretty different world than what I'm in now. When I went into graduate school, I really wanted to work with my hands. I thought I didn't want to sit at a computer all day. But to my surprise, what I actually enjoyed the most about my work was writing code to analyze data. So when I did transition into industry, data science ended up being a really good fit. Kind of relies on a lot of those similar problem solving skills that I learned. Obviously having a strong math background is useful and you know, analyzing, analyzing data, writing code to analyze data. So it was kind of a good fit. Right place at the right time. My transition and you guys talked about job interviewing. I'll say I've written a blog post about this. But my transition from academia to industry was a lot more difficult than I expected. I was doing well in academia, so I kind of thought it'd be easy for me to transition into industry and really kind of was naive about the importance of connections. You know, I had a cv, not a resume, but a CV with publications on it and things that people in the industry don't really care about. So I came into the whole industry job world a little naive and ended up applying for a lot of jobs online and going through this process and many people have probably been through it, where you apply for a job through what's called an applicant tracking system and an ats and you enter all your information in, you upload your resume and then you have to re enter all the information in and then all your information kind of gets parsed into plain text and you finally submit, you know, and you've, you've spent all this time kind of trying to personalize your cover letter and you submit and you just never hear anything again. So it's kind of this black hole.

I hate those systems. Whether, whether I'm an applicant or a hiring manager, either way, they're terrible.

Yeah, I hate how you format your resume perfectly and you get it all flashy looking and then you go through the system and then you realize that you just have to like put it in as plain text or something. And all of that work is for not exactly.

And there's a lot of gaming in the system, which I didn't know and maybe I would have benefited from knowing this at the time, but there's actually like, there's a website, can't remember what it's Called, but you can go and see. You put in a job posting, and then you put in your cover letter or your resume, and it will tell you the likelihood of getting past these ATS filters. And it's really just a lot of times it's just like a keyword match. And I always felt like that was a little weird to just like, put the exact keywords that are in the job posting in my application. But it turns out that does help you get past these filters. A lot of times these filters are pretty simple. They're looking for certain keywords, or they're looking for certain school you went to or gpa, which is silly because we found a lot of times that doesn't really tie to job performance very strongly at all. But the bottom line is, you know, companies just have to so many applicants, they need some way of filtering through people. And I went away from the experience feeling like something's wrong with this system. When me and so many people that I know that had PhDs and once they got a job, they did really well. They were just passed up by so many companies. The companies are losing out as well. That story is kind of my motivation of why I care about hiring and kind of fixing this broken system.

Now you're director of Data Science at HireVue, is that correct?

Yes.

Yeah. And what does HireVue do? Is it one of these systems or what do you work on and what does the company do?

Yeah, we're not an applicant tracking system, although we do a lot of times, have to work with them, integrate into them. We are a video interviewing company. So our philosophy is that a resume and a cover letter are not a very good representation of a person. So, you know, we started years ago with a video interviewing platform, and our most popular product is called an on demand interview, which is asynchronous. Our customers are companies that create interviews. They can have any different types of questions. They can have professional football player ask the question. They can do all sorts of interesting things. They can send the same interview out to many different people, and the people record themselves answering questions on their own time. And then the companies can review those interviews on their own time. So that's a very popular product, and that was kind of our main product for a long time because it kind of replaces this resume, phone screening, initial stage of the funnel, because we've all experienced looking at resumes and they all look the same, and it's really hard to differentiate people. But once you hear them talk about what they're interested in and how they Communicate, you can get a better feel for who they are. So we had a lot of success with that product, but we still had this issue of volume. Like I said before, like, companies are just getting so many applicants that it's impossible for them to actually look at all of them. And the way it ends up going today is that a lot of people are just randomly ignored. So we started building our AI product a few years back where we said, we have all this rich data from job interviews, and if our customers can tell us who ended up being good at a job and who is bad at a job, and that what that is depends totally on the job. So we have some performance metrics around. You know, this person was a really good salesperson, they sold a lot, and this person wasn't. Can we train algorithms to notice patterns between, you know, people who are top performers in a job and others? So that is the assessments product that I work on.

So would it be fair to say that you're kind of focusing on using machine learning to take the bias out of the process of hiring? And. And if. How does that work? How does that manifest itself? How do you train to get rid of that?

Yeah, so it is a common question that we get pretty immediately when people hear about what they do. What we do is a little bit of like, oh, this is creepy, and how do you know what the algorithm's doing? How do you know it's not bias? So algorithms are really good at stereotyping, and that can be an issue anywhere where AI is used. If there's any bias in the training data or just even underrepresentation in the training data of certain groups, the algorithm could mirror that bias.

So do you mean kind of like if there's only a representation of certain type of candidates, let's say, then your algorithm might behave differently when it's trained on that data. According to when it sees those candidates versus candidates that weren't in the pool, in the training pool, is that kind of a fair statement?

Sure. And I think an even bigger issue is if there's a small number of, like, say there's only one female software engineer and she wasn't very good, then the algorithm takes that and says, oh, every time I've seen someone act like this or talk like this, they were bad. So if there's no one, the algorithm doesn't learn as strong of patterns, although it could, and it's something you want to look out for. But a lot of times underrepresentation or just explicit bias in the data, which we do Sometimes. See, and depending on how subjective that performance metric is, that can be strong depending on the country as well. We've seen it vary and kind of like manager ratings and things that are subjective like that. So we definitely prefer objective metrics like sales numbers, call handle time, kind of productivity measures, things like that.

I'm curious, have you had more of a challenge on this front in certain industries? I'm not sure which industries Hirevue is working with. You mentioned sales a little bit. Maybe software engineering. Do you have to kind of approach this as far as your models go differently in different industries, or is this something that's kind of a problem across the board?

Yeah, I would say it's probably more on a company level or a cultural level that we notice differences. So a lot of what is important in trying to level the playing field is, you know, these interviews ask people very consistent questions. And that's something that's been done in hiring over the past several decades because, you know, hiring is very much about gut feelings. So we've improved it by trying to treat all candidates in a consistent way. But it's pretty much impossible for humans to actually do that. Humans have this implicit bias that we don't even know we have. So there's also a big culture recently of this concept of cultural fit, which is very popular. And companies say they want to hire someone who they like and that can communicate well with them and work well with their teams. But this often results in a similarity bias where, I don't know why, I just like that person. Well, you like them because they're a lot like you, or they are a lot like your team already. So you get this homogeneity and your team.

So to some degree, would it be fair to say that when. When a company is looking for cultural fit, are they almost acknowledging their bias and saying, we're gonna. We're going to accept that as part of the process, or am I misreading that?

I mean, I think some people have made that argument. There's, you know, there's articles written about the issues with cultural fit, which is just. Are you just opening the door for bias? I wouldn't go that far to necessarily say that that's exactly what's going on. I mean, I do understand the concept, but it is very tricky. And humans are probably going to be a part of the hiring process for a long time. So it's something that we need to try to deal with.

So I'm kind of thinking in my mind right now in terms of like, okay, we know that humans are biased in terms of these ways that we've mentioned. We know that we can kind of subtly introduce bias into our machine learning and AI models via representation in the data set and other ways. I'm just wondering, as kind of human AI developers, what chance do we have of kind of fighting this bias and how can we have hope to actually do something better?

Yeah, I think a big part of it is just becoming aware. So as data scientists, I think we've, we spend a lot of time just trying to optimize the accuracy of our algorithms and kind of not thinking about bias or fairness at all. As I've studied algorithmic fairness more and more, I found that it's a more nuanced, tricky topic than you might assume. So if you look up, there's a recidivism model. This is kind of a started a whole conversation. It's called Compass. And it was this recidivism model in Florida where they tried to predict the chances that someone would reoffend after they were released from prison. When you looked at the data, actually blacks had a higher false positive rate, so they were marked as being at risk when they actually didn't reoffend in the training data at a higher rate than whites. That algorithm was trained to optimize accuracy, but because of different base rates in the data, this was a side effect. So this whole thing spurred a really interesting conversation around fairness and how to define it. And the upshot is that basically there's many different notions of what makes an algorithm fair. And with most real world problems, it's impossible to satisfy all of them. So it makes things tricky for data scientists and we actually need to consider what notions of fairness matter the most for our particular problem. Another example, and I think marketing is a really interesting space because it relies a lot on demographics. So an example of a situation to think about is if you're trying to predict who would click on a data science job posting like an ad for a data science job, the algorithm could look at a bunch of browser data and say users who look at female type things online are less likely to click on that ad and end up making an algorithm that doesn't show it to any females. It's a really strict notion of fairness to say we need this to be shown to the same percentage of men and women. That's obviously pretty strict because there are more men that are interested in the ad and would click on the ad so the marketing company would lose money. But it's maybe realistic to aim for something else, like we Just want the same true positive rate. So out of the people that are interested, same percentage of men and percentage of women saw the ad, for example. So those are the kinds of things, and there's a lot more detail beneath that, but those are the kinds of different notions of fairness that I think you need to take into consideration when you're building an algorithm.

So we've kind of dived right into doing it from the algorithm and I guess I'd like to see if we can differentiate a little bit between what a traditional job assessment process looks like and how HireVue is approaching it algorithmically at this point. And what are the things that might be the same for companies going from one to the other and what are some of the things that might change for them and how do they prepare for that?

Sure. So yeah, a lot of people are familiar with this traditional job assessment, which is often like multiple choice tests. And they've been around for a long time. They are the result of trying to make the process more consistent. Some of the drawbacks are that they're closed ended. So you have maybe multiple choice, but none of those choices describe you. And they also can be kind of a bad candidate experience. So companies care a lot about that. Like they want people to come in and have a good experience even if they didn't get the job. They don't want to damage their brand by having this awful experience. So those assessments can be long and make that experience negative. And they also give results like personality traits. And the connection between personality traits and actual job performance is loose or it's maybe kind of made up by a person. So assuming, you know, we want a salesperson to have these exact personality traits is sometimes not validated in our process. We actually, you know, like I said, we train straight to performance. Like I mentioned before, we try to get objective performance metrics and that could depend on the job what exactly that means.

So like in the example of the salesman that you talked about, there's a stereotype that, you know, people have about what is a salesman, you know, what's that natural born salesperson, you know, look like personality wise? And that usually has a picture that, you know, is our the stereotype in our head. Are you essentially trying to take those stereotypes out of the process by validating which of the metrics are applicable for that job versus what we can see from the data is not?

Yeah, sure. And I think sometimes that does happen. That humans have an assumption about what is going to make the perfect person for this job versus what is actually in the data. And so I think a lot of times those notions are overturned by looking at actual performance data.

And one thing that I'm thinking about here is, you know, it might be like you already mentioned the example where you only have the one example of a female software engineer who went through and maybe performed one way or the or another. Is it hard for you to, as you're thinking about being objective in these ways? I imagine in some cases it might be hard for you to actually get the data that you need to be objective. Like maybe when you're first working with a company, you don't know the performance information of how the people that they've hired in the past have performed in this objective way. How do you go about establishing that data that you need as the foundation?

Yeah, a lot of times that's a process. So a lot of companies don't have really strong performance metrics. And so we have a team of IO psychologists or industrial organizational psychologists who go in from the very beginning and help our customers kind of get set up. If they're existing customers, they might already have their own interview and their own questions. But ideally we kind of start with them from the beginning. What is important to this job? We do a whole job analysis. Right. So what do you want to measure? What are you looking at? And our IO psychologists have a lot of experience with knowing which questions to ask to actually tease out that information. So it's kind of interesting, like there's questions like tell me about yourself, which are good warm up questions that don't actually differentiate people very well at all. Whereas questions that are about a situation like what would you do if this happened? You have this difficult customer and you know, some detailed scenario, how would you act in that situation? Those, those questions tend to be better at differentiating top and bottom performers. So the hope is we go in from the beginning and kind of design the interview. We design the process of, you know, how we're going to collect performance data. As you guys know, machine learning algorithms do rely on our training data being kind of representative of who's coming in the funnel. So we want to see distribution of people. Sometimes gathering enough data is a challenge though. So we have continuous monitoring of our algorithms. I can say a little bit more about that. After we release an algorithm, we're always watching for how it scores different groups of people and making sure that it's not treating different groups of people in a statistically significantly different way. That makes sense.

That was something, Chris, I know we talked about in our last news updates thing as Google recommending through their AI, forget what they called it, AI guidelines to always be continuously monitoring for those biases and everything.

Yep.

Yeah. So for us, I mean, I mentioned before, when I've done research on fairness and AI and bias in AI, there's a lot of problems that are really difficult to solve because the features that you're looking at, the inputs to your model that actually do matter for the thing you're trying to predict, have different base rates in the data. So an example would be like, if you want to predict who should be given a loan or not, you'd need to look at credit score and income. But credit score have different. And income have different distributions among different age, race, gender groups. So it's really hard to get away from that coming into your model. In a way, we're really lucky because we are only looking at this job interview. We don't do any kind of facial recognition. We don't find out who this person is and try to, like, scrape the Internet for more information about them. We're not throwing in a bunch of data that we don't understand. We know exactly what we're dealing with. And the way we take our video interview data and structure, it is intentionally made to kind of obscure some of the things that we don't want to know. Like, we don't want to know your age, race, gender, attractiveness. We want to know the content of what you said, how you said it. Like tone of voice, pauses, things like that, and your facial expressions. So those are kind of the three types of features we pull out and structure. So we're already kind of blinding the algorithm to demographic traits. But one thing to be aware of is that if there's bias in the training data, sometimes those traits can leak through somehow. So, for example, maybe you have an algorithm that was trained to be sexist, and it will notice some little difference in how men and women speak in the data set. So if that's the case, this continuous monitoring is really important to see how the algorithm is behaving in the wild. And if it does have any issues like it's scoring men and women differently, we can go back and say, what are the features that are even telling the algorithm who's a man and who's a woman and then remove some of those features. So we'll do a mitigation process. We are in the situation where we have a lot of features, so we can afford to throw some out. If they're contributing to bias, we simply remove them. In doing that, we might lose a Little bit of predictive power, but we mitigate that adverse impact. We're also lucky in the sense that our rules are very well defined by the EEOC or the Equal Opportunity Equal Employment Opportunity Commission. So there's federal laws about how assessments can need to behave, and so we follow those very closely. And basically, the rules say that if you have some kind of a cutoff, like people who score above this score continue on to the next steps. And if you score below, you're out of the running at that cutoff. No group can be scoring less than 80% or 4/5 of the top scoring group. So we have to follow those rules. That's U.S. law. And making sure that our algorithms are not treating people differently. And if we ever see anything, we can go through this mitigation process.

Okay, so coming back out of break, I have a question for you, Lindsay. What types of things cannot be covered, well, algorithmically, and starting with that, and then kind of, where do humans fit into the equation? You noted at the beginning that you thought humans would be in the equation for a while, to go for a long time, potentially. And I'd like to understand kind of where they fit in and how the human and the algorithm work together.

Yeah. So definitely we're not taking humans out of the loop anytime soon. I always kind of laugh when I try to talk to Siri, and she does a terrible job of understanding what I'm saying. And I think, oh, my God, we're worried about these robots taking over. There's still so many things that humans are a lot better at. I think the important things that AI will be taking over are the mundane, boring things that AI can do well, while humans still really need to be a part of making personal connections, making final decisions, and taking in other information that might not be available to the AI for hiring, that does, on the other side of the coin, mean that bias will still be a part of hiring. But we found that even removing bias from a chunk of that hiring funnel can help people get through to later stages that they might not have originally. We've had customers say they've increased their diversity by 16% or give us these great metrics around. If this initial stage of the funnel is open to more people, they tend to get further along in the funnel. So definitely for the slice that AI is taking over, we hope to remove that bias.

And one of the things you mentioned is kind of monitoring around fairness. And I was wondering. It seems like you have to kind of develop a certain culture as data scientists and as a data Science or AI team to really make that a core part of a goal on each one of your products to kind of monitor for fairness and all of that. I was wondering if you could kind of briefly talk about how you went about developing that of kind culture on your team and maybe make some recommendations for those out there that are kind of thinking about, oh well, this is something I'd really like to do on our team, but I maybe don't know where to get started or how to develop that culture.

Yeah, definitely. I think for us a lot of it came from our IO psychology team and being in the assessment space. So starting from there we kind of had, like I said, we have laws around what our assessment, how our assessment assessment scores people. Our particular assessment happens to include AI. We were coming into this space that the job assessment space that had been around for decades. So we got a lot of those ideas started there and then it's kind of blossomed more and more as we've studied. There's a lot of academic study going on around this and we collaborate pretty closely with some researchers here at the University of Utah who study algorithmic fairness. Like I said, it's, you know, what constitutes fair is not well defined. So it's usually something that needs to be discussed and refined for every individual problem. I would suggest a great place to start. IBM just released, it's called AI fairness 360. You can go on their website and just play. I played with it a little bit with just some Kaggle data and they show, you know, a lot of these metrics that I talked about, you know, the, these kind of definitions of fairness and you can kind of see how those things are related to each other and how you can possibly mitigate bias. Another recommendation I have just to kind of illustrate the concept is Google did some research. I think if you just, if you look for attacking discrimination with smarter machine learning, there's an article with an interactive portion where you can play with this fake data where it's credit score and you're trying to predict who would repay a loan. And this is something I mentioned earlier, but it's a great thing to play with and kind of see how there's trade offs. So there's, in real world situations there's really not one way to do things where you could satisfy all notions of fairness. So you're always dealing with these trade offs and I think that's something that's good to look at. And again this really varies from problem to problem depending on your inputs and how different your base rates are and how much you rely on inputs with different base rates to predict your outcome.

So keeping things practical, because this is practical AI, I'm finding all of this really, really fascinating. And I was wondering if you could just kind of walk through. So do you establish maybe based on looking at some of this Google work or IBM work, kind of figure out some metrics that at least make sense to track first and then how are you tracking them? So you're predictions with your model and then are you running those metrics on the predictions? Are you running them kind of on the training data that you're feeding in? What exactly are you monitoring and what's kind of the process you put the metrics in place and then you kind of send notifications to people to review them and who reviews them. I'm kind of interested in those sorts of details.

Yeah. So like I said, the notions of fairness that we look at are tightly tied to employment levels law. But we also do we look at other things as well. And we're always kind of interested in being ahead of it. I think it's kind of common that people assume data scientists don't care about this. And we really given it a lot of thought and we're always looking for different ways of looking at it and seeing how we can improve certain notions. But again, we kind of always come back to the regulations in the employment space as being kind of our most important base to cover. So I mentioned the four fifths rule or the 80% rule for us, which is something we closely monitor. And like you did ask before about training data versus kind of how the algorithm is behaving in the wild. So we're always watching that. Like, here's the customer's cutoff score. They are watching job interviews for everyone who has scored above this and maybe first or maybe they're not watching the lower scorers at all. So what are those ratios at that cutoff? You know, how are men scoring compared to women? How are the different races scoring if we ever have an issue there? Continuous monitoring is really important because we start off with a training set of maybe hundreds and hundreds of interviews and there wasn't a lot of diversity. Possibly there's groups that were small and it was kind of hard to see with all the noise how the algorithm's treating those groups. So watching how the algorithm actually behaves in the wild is very important as well. So we're always watching those numbers and being proactive about coming to our customers and saying, hey, we need to mitigate your algorithm. Obviously we also Mitigate at the beginning, but if we ever see that we need to mitigate after the algorithm's been out in the wild for a while, we will do that.

Have you seen, kind of have certain things surprise you as you've done this sort of monitoring, like biases or things pop up where you thought you did a really good job preparing the algorithm, but it turns out you didn't in some way or another?

Yeah. Most of that probably just comes if there's any bias that comes in later on. A lot of that is because your training group just wasn't very diverse. So that is something that we see when we have. Maybe there were very few people of color in this data set, or maybe there were very few women. So like I said, it was really hard to tell with just the training data that there was some feature that was allowing the algorithm to mimic bias in the data. But it becomes apparent later on, and we have seen that usually, usually not too badly. I mean, usually we're pretty on top of our monitoring. We don't see anything too drastically different than we expected.

Cool. Yeah. So do you see? I'm thinking, you know, maybe we can transition a little bit here to kind of the machine learning and AI community in general. Maybe outside of hiring, Are there things in, like trends in the community around how we're developing AI that that concern you, around the topic of fairness? And then are there maybe other things that are encouraging? Maybe these projects from IBM and Google, for example?

Yeah, I think the conversation, like IBM's 360 toolkit, is an awesome example of how this is kind of coming into the conversation and people are talking about it. For the last few years, I've sometimes been frustrated by the alarmism that goes on in the media, kind of calling out situations where data scientists did behave really irresponsibly or just absolutely didn't think about repercussions. And it's hard, as a data scientist who does care about this and works on it a lot, to not get a little defensive when you're stereotyped. But I think there are some legitimate concerns. And there are a lot of books and articles about algorithms gone wrong and kind of showcasing these kinds of examples. I think it's good that that conversation is out there. In some ways, it scares people and they kind of make assumptions that all algorithms are bad, which can be frustrating from the hiring point of view. You know, I talked about how broken hiring is, and I really feel like we've made huge improvements where with an algorithm, we can actually look inside the algorithm and say, okay, what features are causing this bias? You know, you really quantitatively see how the algorithm is treating different people where it's a lot harder to do that with human beings. Human beings don't even know why they made the decisions they made. You can't open up their brain and figure out, oh yeah, you're a little racist and that's why you're doing that. Let's just tweak your brain and account for that. So we have these tools that are amazing, but like any powerful tool, they could be good or bad. And so I think it's, think it's, we're reaching a point where people are having these really important conversations about using.

Them responsibly, talking about bias in these ways. We, we've had various conversations across different episodes with, with people doing all sorts of different types of work and, and it's, it really seems that you have a great process now on how you're approaching it from, with the monitoring and with the feature selection and trying to make sure your data fairly represents where you want to go. And in a broader sense beyond just the topic of hiring, we have so many people that listen that are faced with similar challenges. Do you have any more generalized recommendations that you would make to a data science team that is trying to get the bias out of their own situation, out of their own circumstances or something where rules of thumb that you utilize on that that is kind of broad based and simple for them to follow?

Yeah, I know. I've seen for example, checklists come out. I don't know if those are useful or anything around your data and process and all of that.

Yeah, I think like I said, it's hard to define what fair is and I think you have to kind of sit down and have a conversation with a lot of input about what you care about in this problem and being transparent about it. If you're not just trying to get a higher prediction accuracy. Be clear that we care about these notions of fairness and, and this is what we're doing, this is what we're measuring and this is what we're doing to mitigate. That's something that's just been really useful for us because we were doing this for a long time and not really talking that much about it. We were getting criticized when people assumed that we were being careless. So I think now this conversation started and people are being really transparent to be really open about it and say, hey, what we're trying to do is difficult. These are the notions of fairness that we care about and that we're trying to optimize and we're open to have conversations about that and we're open to, you know, changing that. I think everybody understands that, you know, machine learning can be very powerful, and if there isn't clear answers, we want to have a conversation about what we're trying to do with it.

One of the things that we've noted before is, you know, we're still in the very early days in data science, you know, especially if you, you compare it to software engineering, who has been maturing for decades now. And I'm kind of talking about the AI space specifically, but do you think that this period right now where we're all grappling with bias is a kind of growing pains that we're going through, or do you think this is going to be inherent from now on? Is it always something that we're going to contend with, or do you think we'll have better tools going forward to tackle it?

I think kind of both. I mean, I do think it's a growing pain. I think in five to 10 years, way more data scientists will be well versed in fairness and understand that it's a part of their job and it's something they need to think about. But at the end of the day, it's like any complex topic, there's always going to be different opinions. So because there's not one clear answer, I think there will always be debate about what an algorithm should be doing. And this is a great example with the compass model, the recidivism model that I mentioned. At the end of the day, there's no agreed upon way it should behave because different notions of fairness to satisfy them, you sacrifice another. And there will always be people that have their opinions about what the most important notions are. So I think it will be something that's controversial going forward.

I know that. I have definitely appreciated your perspective on this, Lindsay. It's been super enlightening to me. So thank you so much for being on the show. Are there any places where you'd like to point people to find you online or certain resources or blog posts that you'd like to highlight?

Sure. I'm mostly just on LinkedIn, Lindsay. With an E, Y Zulaga Z U, L, O A G A. That's where I'm probably the most active.

Awesome. Well, thank you so much for being on the show and I know I'm really looking forward to seeing more of the, the great content that you put out and, and the great work that, that you and your team are doing so. Thank you so much.

Thanks for having me.

Thanks a lot.

All right.

Thank you for tuning into this episode of Practically I. If you enjoyed this show, do us a favor. Go on itunes, give us a rating. Go in your podcast app and favorite it. If you are on Twitter or social network, share a link with a friend. Whatever you gotta do. Share the show with a friend if you enjoyed it and band with for changelog is provided by Fastly. Learn more at fastly.com and we catch our errors before our users do here at Changelog because of rollbar. Check them out@robot.com changelog and we're hosted on Linode Cloud Servers. Head to lino.com changelog check them out. Support this show this episode is hosted by Daniel White Knack and Chris Benson. Editing is done by Tim Smith. The music is by Break Master Cylinder and you can find more shows just like like this@change law.com when you go there, pop in your email address. Get our weekly email keeping you up to date with the news and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.

I'm Nick Niece, this is K. Ball.

And I'm Rachel White.

We're panelists on JS Party, a community celebration of JavaScript. Every Thursday at noon Central, a few of us get together and chat about JavaScript, Node and topics ranging from practical.

Accessibility to weird web APIs.

You could just eval the text that you're given and then. And I think that's basically what it's doing. What could go wrong?

Yeah, exactly.

This is not legal advice to eval text as it comes in. Join us live on Thursdays at noon Central.

Listen and slack with us in real time or wait for the recording to hit.

New episodes come out each Friday.

Find the show@changelog.com jsparty or wherever you listen to podcasts.

I'm Tim Smith and my show Away from Keyboard explores the human side of creative work.

You'll hear stories, sometimes deeply personal, about the triumph and struggles of doing what you love.

I got really depressed last year, and the reason it was so hard is because basically everything culminated at once.

All these things I'd been avoiding, all these things I'd swept under the rug.

They all came out at once.

New episodes premiere every other Wednesday.

Find the show@changelog.com afk or wherever you listen to podcasts.