Designing Artificial Intelligence to Navigate Societal Disagreement

I am a human-centered designer. Let's see, that should be presenting. Shout at me if it's not working. It's working. And that means that I'm interested in how we sort of design not just the technology qua technology, but how we design it to be a better fit for human needs and goals. And that raises some really interesting questions recently. recently as I take a bunch of the applications that we've been wrestling with in the space of social media design and try to use that to reconsider some of the core assumptions of how AI works. Just to set the stage here, AI has essentially been designed as if it's in some hermetically sealed chamber. Where we have inputs coming in, outputs coming out. This has been a tremendously successful model for a number of years of AI that we give it, say, an image pixels as input and it produces a label as output. Over the last few years, we've noticed that AIs are embedded in a social context, that we're using them for things and those things have values and have import. The most straightforward first recognition of this, which I think we've already made progress on as a field, has been to recognise that this deployed AI is in use by people, and it makes no sense to design an AI that isn't a fit for the people who are going to be using it. So we talk about what are we using this AI to achieve, who's using it, but there's much, much more to this social context. Whose labels are training the AI? Because depending on the task, that can have a huge impact on what the AI does. Who do we ask for the ground truth? Their opinions become the behaviors that the AI learns to emulate. And so speaking as someone who, as Danielle mentioned, has traveled far and wide in the data annotation side of the research world, I can say with confidence that this question is often ignored. So we need to start asking those kinds of questions. And of course, there's even more than that. The AI is embedded in a broader social world that we live in, one where the decisions that we make and who we help, who we respect, how we interact with each other. All these are impacted by the people around us and how we relate to each other. For the AIs to succeed, they have to learn to navigate that societal environment fluidly. But at the end of the day, we recognize this at some level, but the AIs that we build and deploy are created as if that social world just doesn't exist. It's totally absent, that it's AI in a hermetically sealed room. The order of the day for me today is to examine what it would take to unpack that assumption and build AIs that don't make that assumption. I hope to prove to you that doing so opens up lots of really interesting design levers and also creates AIs that are in many ways more performant and more honestly performant. I'll get to what I mean by honestly in a moment. Let's start with something that we just presented a few months ago, at CHI about how we design AIs that navigate the societal disagreement more effectively. This is work that was led by my PhD student, Mitchell Gordon, who's going to go on the job market next year, so consider this my recommendation for him, as well as colleagues, several other graduate students at Stanford, KR Patel at Apple Research, and colleagues at Stanford, Jeff Hancock, Tatsu Hashimoto, and myself. Let's talk about ground truth because at the end of the day, what the AI is emulating is ground truth. As I mentioned, I work in social computing quite a bit, which means that I'm interested in where AIs get deployed in social media contexts and many others. This is a very popular space where these AIs are deployed, where they're trying to do pre-moderation, meaning that they're taking comments that are getting posted, And before anyone sees them, they're trying to figure out if this is gonna be toxic or otherwise harmful to the community so that they can take it down. Most large online platforms have some sort of mixed AI-human platform that's doing this kind of work. So here's a real comment from a real dataset saying that it's terrible that people eat at Pizza Hut and it's shameful how this country has devolved into all sorts of ways. Now, if this were an in-person talk, I might ask you just for a show of hands if you think this kind of thing should get removed, if this is so toxic that the AI should remove it. If you're anything like our dataset, it actually turns out to be about 50-50. About half of folks think this should get removed and half shouldn't. Now, this kind of judgment gets baked into applications. This is Google's Perspective API. So you can see down at the bottom here, a little confidence score saying that you don't like my talk very much. This comment about my talk might be toxic and should be removed. And these classifiers are embedded in all sorts of these online environments. We use them for toxicity detection, misinformation detection, automatic moderation, all sorts of things. Now, We have a comment that I started with that where there's a lot of embedded societal disagreement. Now let's reflect for just a second on what we do with that when we train the AIs, because I'm going to argue it's basically swept under the rug. Basically what we tend to do is we ask a bunch of folks and then aggregate their votes, so to speak, to identify what we then call a ground truth label, which we use to train the AI. We ask 5 people, 3 of them vote gray, 2 of them vote red, we run a majority vote and this just gets labeled a flat gray. The fact that 40% of people disagree with that label is essentially just ignored. We take these aggregate labeled examples where we've taken these votes, we just take a whole bunch of them and we use it to train the neural network that's then going to operationalize this. Now, we can talk in a moment about how you can get more complicated about this. You can learn distributions instead of flat labels, but they're all going to run aground on the same problem that I'm going to talk about. For many tasks, even experts disagree on what the correct label would be. Machine learning's architecture came out of a world like this where it was a question that was built into our perceptual system. Is this a cat or a dog? It makes sense that there would be a binary label here that we would all agree on and that any disagreement would likely be an error. Now, in social computing and many, many other spaces, tasks like toxicity detection does not have that characteristic. It's more of a societally constructed result, which means it might differ across societies, but importantly, it's going to differ across groups, intersectional identities, and other kinds of of slices we might take. So when this kind of disagreement is irreconcilable, who we sample for the labels is gonna matter. If you ask a bunch of white cis men who look like me, you might get a different answer than if you ask a bunch of Black women who are constantly harassed online. So what happens is that even if we were to take— do the sort of social scientific correct thing of do randomized or stratified samples, you're often going to get these overridden by majority vote. The fact that they're minority or minoritized groups means that when you take that majority vote, this group that's saying it's toxic is just going to get flattened and squeezed out. This kind of disagreement is everywhere. In fact, just within the datasets that people use to train this, we went and looked at the data. It looked like a comment that was labeled by majority vote as toxic was disagreed with on average by 40% of annotators. So even something that we say is toxic, only about, I'm gonna argue here, about 2/3 of people think that that's true. You can go look at expert moderators online. They disagree with each other about 1/3 of the time. And I once heard a story in conversation with folks at a large social media company where they found that they actually had a bug in the software that they were using to feed misinformation links to fact-checkers, where the fact-checker got the same link by accident again, like a couple of days later, every once in a while, and the fact-checkers were disagreeing with themselves a third of the time. In many of these contested spaces, you can think of as roughly a third of the annotations, people were going to disagree with each other. This is all over the place in human-centered artificial intelligence. Doctors disagree with each other. We build these systems that are like medical AI systems. Well, what happens when the specialists disagree? In HCI, we build these AI design systems. Different designers are going to have different opinions about what a good design looks like. What are we supposed to do about this? So what we've been developing is a machine learning model or an architecture called jury learning, where the metaphor changes, where what we do is we say we're going to specify a jury. We're going to specify a jury of, say, 12 members, could be any number, but let's take 12 here, and articulate what proportion of that jury should represent each group, each intersectional identity, Things like this. So we might say something like with online harassment, again, this example I've been using, we might define a jury that more heavily represents groups who are targets of harassment online. What we then do is make the promise that the system is not going to output just a single label, but the interpretation is going to be something like this. For this jury, which has the following composition, might have certain gender identities or intersectional identities or non-binary identities, this percent agree that it's toxic. What we're doing here is essentially arguing for a shift where we're moving away from modeling some aggregated pseudo-human who's this majority vote, who's trying to model the majority vote and actually model individuals. We want the AI to, rather than predicting the majority vote label, it's going to be predicting each juror's response. And then outputting that, which then gets put into essentially the jury vote. So if you're using this, you're going to be defining who your jurors are going to look like. A large part of this is about saying we need to be specific and explicit about our values in these AI systems, and the way that we think is a reasonable way to do this is to get explicit about who should be on the jury. Whose voice should be represented. So you author your jury sheet here. This produces, in this case, a simple example of 4 juror members. And then we have a huge dataset. As usual, this can be done with most crowdsourced datasets that you might get. You have a bunch of different annotators, each of whom all we really require here is that you have demographic information or any other sort of information that you want to stratify over in your jury. So we identify everyone who matches those categories, and for a jury, we then will randomly sample from those who match each category to populate the jury. The AI is then going to predict each juror's response and then produce a result. Now, of course, that's how a traditional jury would work. You can also treat this as a regression task instead of a classification task. It would work roughly the same way. But of course, unlike a real-world jury, we can just resample juries over and over and over again. As we do this sampling with replacement, basically almost a bootstrap sample here, we can build out a distribution of what juries in this space on this problem would say. And you can sort of say, take a median of means estimator to say that here, the median jury outcome says that this is a slightly toxic piece of content. This is again in the regression context. We can even show the median jury what they looked like, who those people were. This allows us to make more sense of disagreement. We talk a lot about interactive machine learning and about interpretability. This allows us to say, hey, look, this is the distribution of all the annotators who match this juror sheet and what they think. Here's what liberals think, and we don't collapse it into a single value saying, a liberal says this, we say, here's the distribution and you can make sense of this yourself. And here are the individuals who we sampled to be in the jury. So I said this is a new architecture. I wanna just point out how this works. We need to unpack this box here that says the AI is predicting each juror's response. This actually is interesting and non-trivial in fun ways where what we're gonna do is take a traditional dataset that you might take, and we're gonna look at it a slightly different way. So we're gonna have all these training examples that have all been labeled by some sparse subset of the annotators. This will happen if you go gather data on Mechanical Turk or Prolific or any of these platforms. And we have a bunch of missing labels. So this starts to actually look a lot like a recommender system task where you have some observed characteristics that are going to matter and you have individuals and we're going to try to predict it on unseen examples. That's one place where many recommender systems don't do this. They're typically operating in a closed world like Netflix where all the movies are known. But we have something that we haven't seen before, so we're going to actually create a model architecture that is going to do a bit of column A, a bit of column B. It's going to do a content-based prediction. You can see here the inputs do include the comments, which then gets pushed into something like BERTweet that is trying to make inferences on unseen examples. But then you're also doing something with a cross-network recommender system, which is learning an embedding of annotators. If I know something about other annotators, I can learn something about this person and the context embedding, which is telling us about the meaning of what these annotator characteristics tell us. We put this all together, into the prediction. And this actually ends up having, in a technical evaluation, we find this achieves state-of-the-art performance on unseen examples, better than state-of-the-art models on the existing task, because what it— the reason we can do this and achieve state-of-the-art performance is because modeling individuals is better for these kinds of tasks where there's disagreement than modeling the overall outcome. So you might be wondering, can jurors be biased? Yeah, absolutely. People can be biased. But let's step back for a minute. If I'm given a choice between trying to debias a model post hoc, which is what a lot of the early work in fairness in machine learning tried to do, on one hand, and I'm given the other option that says I want to select and train appropriate voices to minimize that bias or at least recognize it and design around it, my vote is doing this latter thing. I think trying to take a model that already has a bunch of issues in it and then trying to tweeze them out is much harder than actually starting with a cleaner dataset. Cleaner by my definition here just means where we know, recognize, and work with the biases that we want. This project asks an important question, which is now, who gets to decide that jury? I've told you that one way that we're going to deal with this is by making explicit whose voice matters. But now there's contestation over this. Who gets to decide what the jury looks like? Does large company just decide and get to go with it? I want to start by just pointing out that all datasets already have a jury. Those juries are just hidden, they're implicit in the dataset. In fact, I'm not talking about it much right now, but we can backwards solve for what the implicit jury is in most datasets. And in the large toxicity dataset that we start with, it's actually 74% white. So I want to start by arguing that we think that doing— we can do much better than this purely by asking people to be explicit. And what I'm going to show you is some evidence right now, or in just a moment, that if we do this, you will get a better result. Here's some initial results that we have where we recruited a bunch of moderators from large online platforms, Reddit, Discord, and so on, and we asked them to author juries for this toxicity AI task. What we're interested in is whether the juries that they produce represent more diverse opinions than the state-of-the-art classifiers. Also, and again, just as a reminder, the baseline here is 74% white, but also whether jury learning changes the classification outcomes. Does the downstream AI classifier actually change? Here's what we see. The red lines in this graph indicate the proportions of individuals in the large popular dataset, or the large dataset. You can see here again the 74% white on the red line on the far right, and you can see other groups. This is a fairly United States-oriented set of ethnic and racial breakdowns here. The blue bars represent the proportions that were used by the moderators in our evaluation. So what you can see is somewhere like often like 2 to 3x more representation of minority groups in the jury learning evaluation and much basically by reducing the proportion of white annotators and of their voice in the jury. Again, what we found was that this does change outcomes. In fact, changing the jury composition changed about 14% of the classifications that the resulting AIs made compared to a current state-of-the-art classifier. I want to argue this is huge. We're often eking out 3%, 4%, or 5% performance improvements on these kinds of AI classifiers and those determine state-of-the-art. 14% of changes is huge. The kinds of things that were most likely to flip when we do a qualitative and inductive analysis here are issues around racism, death and suicide. I'm here. Okay, good. Stuff that was least likely to flip on the positive side were things that everyone agreed were not problematic, thank yous, happiness, hugs, and weddings. There's also some stuff that was very not likely to flip on the negative side, meaning that most people agreed that it should be removed, stuff around human trafficking, various epithets. And so on. So it was really around this middle ground where people were still debating— middle ground's not the right word— this area where there's not society total agreement over where the line ought to be drawn. There are some where everyone agrees, and many where we're still debating whether this should be— or maybe debating is the wrong word— where different groups within within societies hold maybe in some cases very strongly entrenched different opinions. And when I say most likely to flip, what I mean is that most of these issues, which I think personally should be removed, started to get removed and were not before when it was mostly a dataset representing white folks. So I just want to briefly touch on what happens after we deploy these things. Because what happens is that we have these leaderboards, tasks like this. This is again the same jigsaw perspective API task, and you can see here that like the, the ROC AUC or this performance score is very close to 1, which basically suggests almost perfect. And what I want to argue is that we're using the wrong evaluation metrics as a result of this disagreement, that 0.95 is inflated I want to actually just briefly argue that something closer to 0.73 is a more accurate metric for many of these deployed things. What I want to do is show you how we might come to the point of view that we can get more accurate metrics for these kinds of tasks where there's disagreement. The reason I think this is important is that I think product groups make go or no-go decisions. They make launch or don't launch decisions looking at some of these metrics. If your product is performing at 0.95, I think your product team is going to make a different decision than if it's doing a C grade here, like a 0.73. What we developed was something called the disagreement deconvolution, which is a method that we could use to transform machine learning performance metrics to adjust for disagreement amongst the labelers. The idea is that essentially our goal would be to give credit in proportion to the percentage of annotators who agree with a particular label. If instead of just producing one majority vote test set label, we might actually want to only give a classifier that predicts not toxic on this earlier example a score of 0.6 or 3 out of 5, instead of, you know, 1.0. Now, there's a lot of complications in making this actually work. It's generally acknowledged within the crowdsourcing literature that some annotations might be accidents and noise. They might misunderstand the task. There might be a temporary lapse in attention, accidental clicks. So you can't always just take the label annotations as a strict face-value distribution either. So while I won't get too much into the technical details, What you really want to do is remove these sources of noise and leave only the irresolvable disagreements in your estimation of accuracy. At a high level, and this is about as deep as I'm going to get, what we do is we take advantage of the fact that many crowdsourcing platforms actually repeat these attention checks that are used to detect if someone's actually just clicking randomly or paying attention to the task. There's a fixed number of them, like I have 5 or 10, Workers who do a lot of these tasks, they run out of them and they just start repeating them because the system doesn't have more to provide, so it just gives the same ones. It turns out that if you're repeating them, we can then observe how often people change their answer on the same question. This can help us estimate overall how often people flip labels just by accident essentially, and use that probability, that estimated probability to sharpen the distribution, to deconvolve that label set to create a disaggregated test set, that now says, here's what we think the overall distribution ought to be. And when we do this, many problems that appear solved, we see are in reality not. So, uh, the toxicity classifier that I just described, when we do this disagreement deconvolution, drops from a performance of 0.95 to 0.73. Um, and in fact, what we find is that no classifier will ever perform better than 0.73. Here's an oracle classifier that always guesses the, uh, the majority vote. Nothing could ever do better than 0.73 given the amounts of disagreement on the task. We see this drop, this precipitous drop across a whole bunch of different social computing AI tasks, misinformation detection, adult content detection, whereas traditional machine learning tasks where again they're more perceptually oriented seem much less of a drop. Word sense disambiguation, image classification, are more robust to this because people tend to agree about what the proper result is. So I'll close briefly just by pointing at the elephant in the room, identifying that what we need is not just technical solutions but institutional practices. This is a paper that recently appeared in PNAS to improve, and I don't think it would surprise anyone if I started to point out that, you know, AI presents societal and ethical risks. There's a lot of issues with deployed AIs harming minoritized groups around the world. What we need to do is not just create responses that are purely technical or purely social, we need this whole systemic change. How do we create that? Now, what we've been doing, if you're looking within an American context, you might be like, well, what about the IRB? This is the group that reviews human subjects research for harms. It turns out, at least in the US, AI research is exempt. The reason is that there's no human subjects in this research. They're just data annotators. They're basically like paid contractors, and so long-range effects don't matter. What we created and have been deploying at Stanford for the last few years is an institutional process Here in collaboration with the Institute for Human-Centered AI, or CHAI. And what we do is essentially a process that conditions grant funding on, on an ethics and society review. So you apply for funding, you go through a merit review by the funding program, the same kind of thing that would normally decide whether you get funding. And then what they do is they pass to our panel this ESR, ethics and society review, which is an interdisciplinary panel. You can see some of the folks represented here for feedback on the possible downstream applications of what happens after this leaves the lab. And they might provide feedback on things like representativeness, dual use, and so on. And a few things that have— I bring this up very briefly just to point out that this seems to actually work. I was expecting to get a lot of complaints about red tape, but a lot of folks wanted to— in fact, every person we surveyed wanted to keep this around. I added more work for PIs than they wanted, wanted to keep it around. About 2/3 of them felt like it had influenced the design of their research. We've, at this point, had— it's been around for more than 2 years, reviewed more than 70 grants, mostly providing scaffolding. So— wrong quote. You know, things like, well, you know what, we don't know what to do still, but this actually has sort of really changed the perspective on what we're doing. By doing it up front and not just at the point of paper review, we're getting folks to actually reflect on what these issues might be very early on. So I will, I will pause here because it's been about a half hour and we wanted to have more of a conversation than a presentation. And I just want to again thank the students and colleagues who supported this work, the folks who financially supported the work, and all of you. So I look forward to chatting. Thanks.