Stanford Professor Omer Reingold on what it really takes to build fair AI systems

Great.

So welcome back to everyone who is following our Micro One Forum. I see you trickling in and thank you for joining no matter where you are in the world. Some of you on the West Coast, of course, towards the end of the morning, some of you on the East Coast, beginning of the afternoon, but or anyone else between Europe, the Middle East, and some of you also in Asia. So thank you so much for joining. Today is another, I would say, series that we've been preparing for a while. Particularly happy to have both Omer Reingold, who is a professor of computer science at Stanford University, joining and talking a little bit more about algorithmic fairness, and Nima Yazdani, who is a member of the technical team at MicroOne. Nima's been on the series already before for a number of reasons, as a co-author of a paper. But, you know, Nima, your work is equally across so many different sides of the organization. So we'll try to create a conversation around, I will say, fairness, which is something that is becoming important, sometimes borderline with a buzzword, but it is also difficult to really sometimes get a clear definition of what we mean by fairness in models. And so with that said, Omer, I'm going to start with you as our guest today. Look, you have a stellar career, right? Beside your work at Stanford, you working in the industry, you have different roles in your career. And I was reading you're running a center on basically algorithmic fairness at Stanford, right? So it is really something that has become part of your life. Just share with us a little bit how you got into where you currently are, anything that you think is interesting for us to have. And very happy to welcome you to the Micro One Forum.

Thank you. Let me just say a quick correction. The center, this collaboration that we're running on algorithmic Fairness is not based in Stanford. I'm the director, but it's actually all over the world, and we are 14 PIs, and we have our yearly meetings, and it's, it's a very exciting collaboration that around it grew a really, really nice community. My beginning in algorithmic fairness was in 2010, and And then I was at Microsoft Research at the time, and I got to have various discussions with my dear colleague there, Cynthia Dwork. And she started to be interested in that, in algorithmic fairness. At the time, I mean, it was really quite visionary of her, because 5, 6, 7 years after, we already had various situations in which companies have, have so issues with fairness. But at the time, it was more a theoretical vision that this could happen, and we need to understand what, what would fairness mean. And it's not the first thing they've done in terms of society and computation. I also worked on the privacy, but, but yeah, 2010 is where we started, and with the renewed effort like 2017 2017, 2016, where it kind of became my focus for now.

Right. Fascinating. I'm going to try to go deeper into that, Omar. It's where we're learning a lot, right, from people of your caliber in our series. Anima, over to you. Like, you're a member of MicroOne technical team for a few years now. I happened to start to talk to you when I joined MicroOne in 2025. And so if you had to again, more of a warm-up conversation. Share with us where in your work fairness started to become important. I know that in, in previous conversation you were dealing with so many different inquiries of interest, right? Things that really are part of what you are interested about. But when is, when was fairness suddenly becoming more consolidated on your radar?

I mean, this is, I think, an extremely important topic that everybody keeps in the back of their mind through their formal education years. I mean, this This is something I was introduced, you know, as a data science undergraduate at UC San Diego and then during my PhD work at USC, then it became much more relevant. This is something that I think everyone takes extremely seriously these days, much more. Of course, it's always been very important, but you know, as soon as I began my work at MicroOne and that paper you've referenced as well, the research that I'm, you know, I was involved in with MicroOne and in collaboration with Stanford, evaluating our own, you know, AI tools like Zara that MicroOne uses, fairness has to be at the core. It is core. It is a non-negotiable. So it really came, we, you know, went from like deep, you know, thought experiments and discussions, you know, in my PhD work to like, 'Okay, here's your shot, you know, put it into play.' And defining fairness is pretty hard when you really start to confront the problem, you know, to its face. And I think it never gets fully defined and it's infinitely changing. But that's the real challenge that I faced, you know, when I, you know, first began at MicroOne and to this day operationally, like, you know, past the defining, past implementation, The evolution is always continuing.

It is. And I'm happy that somehow there is a common denominator in some of the series that I'm running on the forum is there is a lot of advances, but we continue to need that level of calibration that is required. Which brings me to you, Omer. When I was kind of preparing for our session, I tried to study a bit more about kind of the contribution that I can relate to in the context of what we want to discuss. And And I found that you have been pioneering this idea of multi-calibration. If I understand this correctly, it's on many predictors to statistical properties that you're trying to cluster together with subgroups, and they are helping, I guess, to improve the efficiency of the models. And I don't know whether it's the efficiency per se or is the efficiency towards the fairness. But for people that might have never heard about multi-calibration, how do you— First of all, what is it? And how did you basically develop it in the context of your interest in fairness?

So, perhaps I start a little bit before just to set the setup for multi-calibration. Traditional notions— so fairness is not just one thing. There's a variety of definition and for good reason because Every context, every society has its own perception of fairness. What makes sense when you're admitting students to university is not the same thing as when you're showing ads. It's not the same thing when you're making bail decisions. And so from my perspective, much of what I've tried to do in these 15 years is to provide a language that allows to express notions of fairness that are more refined than perhaps some of the traditional notions, and perhaps not something that immediately comes to mind when somebody thinks about fairness. So the traditional perspective on fairness would say, look at the protected group defined perhaps by race or gender or medical condition or ethnicity, religion, any one of dozens of attributes, and try to get some notion of equality in the way your AI models are treating that group and the general population. And there are various notions depending on what kind of equality you're looking at. Equality could mean that you're accepting people at the same rate. Equality could mean that you're making the same rate of mistakes across populations. And equality may mean also the calibration, which calibration essentially says— it's a simple notion that says that you want your predictions to mean what they say. So if I'm predicting how many people are going to repay a loan, and I'm telling you that 90% of the people are going to repay the loan, So, I'm— let me rephrase. I'm looking at all the people I tell you that 90% of them are going to repay the loan. Indeed, 90% are going to repay the loan. So, when I say 90%, I mean what I mean. When I'm saying 5%, I also mean that. And this allows you to make decisions or to treat these numbers that we're producing that are supposed to be probabilities which we don't exactly mean or know what exactly probabilities mean. It's a big discussion we can have later if you want, eh, to make sense out of them, to be able to make decisions out of them. So if I'm predicting 5% probability of rain, and I don't mind getting wet once in 20 days, and I won't take an umbrella when you're thinking 5%, but that only makes sense if this 5% thing is calibrated. These are the traditional notions. I want you to be calibrated on an entire population, on everybody that is part of a particular gender. But all of these notions are extremely weak, very easy to abuse them. We have tons of examples over the last 15 years that we came up with, and it all boils down to you may be treating everybody okay overall, but you're treating subgroups the wrong way. So you're inviting people to interview, but you're inviting the wrong people from a population and the ones that are going to fail. You're showing an ad to those that are not going to buy. And, and that kind of is where we're looking at multi-calibration. Multi-calibration says, and in general these multi-group notions —then it's not enough to look at the world and divide it to a few disjoint groups and say that overall you want some similar treatment of everybody. Because in any particular setting, there are many, many other groups that you may care about, and there are effects of intersectionality. If you're in this group and in another group, the intersection can be treated differently. So instead, what multi-calibration is trying to do is to look at a very rich set of affiliations that depend on demographics, but also depend on the various other aspects. For example, are you qualified for the loan that I'm going to give you? And I want to respect all of these affiliations simultaneously. So I want to get something that's much more refined than just this group fairness and treats people in a way that makes sense for a very, very large set of affiliations. And perhaps with one last sentence to your question, this— the original motivation is fairness, because I want to treat all of these subpopulations fairly. But over the years, we found that it is very, very effective in terms of kind of robustness, in terms of various kind of good properties that have nothing to do with fairness. And that's one reason this has been very popular and have been implemented in many real-life settings. Yeah, it's fascinating. Answer—

sorry, no, no, but I appreciate this so much that you give me the context and you give all of us the context and, um, understanding exactly how multi-calibration eventually, right, tries to to enhance the fairness, right, of a given process rather, rather than being just a mechanical adjustment. It's really more of a, of a way of expanding the representations, right, of this statistical model that we are basically dealing with, which I think is incredible to hear it explained the way you did, Omer. But when you were talking, I was trying to create a bridge to your work thing. And so if you think about it, And you take, for example, such an important product that we have had at MicroVon for so long, Zara, right? This AI interviewer. And so now there's a lot of preparation about rethinking fairness, almost like in what we call today the pre-deployment, basically in the study part. And then we are dealing with real-world constraints of any kind, right? Of latency, cost, scale. But, you know, Zara is like a life system. So how did you address or at least discuss this conversation of fairness and representation when you were dealing basically with an interviewer that was using information from people, right? I think it's a perfect implementation of the problem that Omar was talking about in the context of what we were trying to do with an AI interviewer. So what's your sense on that?

Yeah, awesome question. I could talk about this forever, but I just want to really set the stage on like the gravity of the importance of fairness of Zara, that that's Where we started, 100%. I mean, Omer brought up two great examples of algorithmic fairness that really affects people's lives. Like, you know, if there's people out there who are not very concerned about this, I think these two examples will definitely affect, you know, most everyone on earth and, or maybe not everyone on earth, but you know, you know somebody who knows somebody, you know, qualifying for a loan, that is algorithmic fairness. That's extremely impactful on people's lives. Maybe even deciding bail for people who are convicted of crimes. Both these things are algorithmic fairness and they very much affect, you know, a lot of, a lot of people's like key life moments. And I think getting a job or being assessed on your skills, that is also, you know, that's your career. That's extremely important. And at the core, this is what we're doing with Zara. So Zara is an AI interviewer. It can interview and you know, most any language, any topic, you know, you know, the humanities all the way up to like concrete PhD-level STEM, lawyers, doctors, really anybody. And so it needs to not only be fair for one group, it needs to be fair for every group, every language, every cultural nuance, every single education background. And so like to, to start on this journey of making sure Zara is fair, we had to set the stage, like just understand the gravity gravity of, we do not take it lightly that people's careers are like in the balance here. And, you know, one of our core missions at Micron is to make sure everyone is spending, you know, as there is increasing automation, as robots enter our lives, as, you know, LLMs are very much involved in our work and our play, you know, where humans spend their times is absolutely, So, so important. And fairness allows us to place humans where they're best suited. So evaluate them on their skills, showcase those people that are like the hidden gems, and make sure that everyone can have like, you know, maximize utility of their time. So I could talk about the whole implementation, but I think the most important thing is that everyone is on the same page of the impactfulness and the gravity of the fairness of Zara. That's fascinating.

You know, you said something before, Nima, when you were relating to the fact that so much of our life today is being exposed, right, from the physical infrastructure and how robotics is gaining traction to, of course, LLMs. And Omar, to kind of just like asking you a question that really triggers my curiosity. So today, algorithms, they are able to inform us in so many different ways, from recommendations to summarizing information of more specific domains to healthcare diagnostics. So some— we are really expanding the notion of domains in general, right? And so when you're looking at this plurality that we're currently dealing with, how do you basically consider this? Do you see this is simply that the same principle of fairness has to be applied across domains as an inter-domain conversation? Do you find that this explosion of data across domains is undermining the accountability of fairness. So how do you see this? And I'm making reference to the fact that we started with more generic models. Today, you see that the AI labs are equally moving to more specific models. But from a perspective of somebody who studies fairness, do you see this as something that will mitigate the issue or will actually eventually create a bigger issue or none of those?

Right, so I kind of hinted that fairness is not one thing. It's— it really depends on context. And so we will not have a single definition that would work everywhere. And to some extent, I think that, that one of the things I care about a lot is making these variety of definitions accessible and clear so that people that make decisions can interact with that. And decisions are made in a lot of different levels. The programmer that chooses, like, one variable over the other makes a huge amount of— has a huge amount of importance. And part of what I'm trying to do is to kind of make it more accessible. And our language of fairness is trying to be very general. But its implementation has to depend on context. On the other hand, there is kind of a disconnect. People that understand the context don't always understand the fairness and the capabilities, and so there is a bridge here to cross, and that's one of the big things I hope to do. But this notion of multi-calibration, to some extent, I feel it does have some universal aspect to it in the following sense. I think that one way to kind of summarize what we've understood in the last 7 years of working on this notion is that to some extent fairness starts at the level of kind of gathering information that is equally informative across many populations, many subpopulations. And so when you're trying to just to get information, something like multi-calibration can work in many contexts. Then the question is, what do you do with that? When you have a prediction that somebody would repay a loan, or you have a prediction that somebody would have a heart attack, or you have another prediction, each one of these contexts, the action itself has to be different. So to some extent, I would say there's something universal about multi-calibration, not completely universal, but where it tries to get as much information that is as refined and as accurate across many, many subpopulations. But eventually, you want the decision to reflect domain expertise and to reflect the values. After all, And there isn't a single thing that is fair. You need— you have values that depend on that.

Omar, if I can just give a very quick follow-up on this, just to close on a thought that I'm currently having. The way you describe it really shows why this should be an engaged— and to some degree is an intellectually engaging conversation, right? It requires some level of subtlety, right, to understand it. Do you find that— because you also work a lot in the real world, right? Do you find the practitioners lack of this understanding of fairness that is basically inspired by the idea of multi-calibration and plurality and the distance between what is domain-specific and universal. And they try to basically use a kind of a streamlined process. What are some of the most common pitfalls you find when you're talking to practitioners? Is it oversimplifications? Just to understand how we can also help them to preempt it.

I think that one thing that I find with talking with practitioners in different levels, people that are setting up policies and people that are really developing, is that they want a solution. Essentially, they want me to tell them, okay, you should do ABC. Yeah. And what I'm trying to tell them, no, I cannot tell you what to do because we need a conversation. Because I need to understand what's important to you. I need to understand your domain. And so this discussion is the challenging part. Let me plug a book that I'm working on. It's, it's not yet ready. It will be ready at the end of the, uh, of the year. So to some extent, this is a poorly timed book promotion for me, but still, just to kind of give the philosophy that I have, that these ideas really can be shared by everybody. They definitely can be shared with people that develop in companies, but they can also be shared with people that make decisions, with administrators. And I'm trying to write a book that will be a popular science book. Well, I actually wrote it, it's in long production now. And I think that, you know, like everything else, knowledge and awareness is important when you want to make decisions that are based on values. Because often we know what we care about, but we don't know how does it translate to something concrete. So this bridge is something that's very important.

Omar, look, if that's something I can tell you, I would love to have you back once your book is out and we can do an event about it. I know that in our initial conversation with Sidney, the book was already mentioned, and I know that, you know, the production cycle is something that is outside of our control. But why don't we kind of give a rain check on having you again when the book is out, right? I would love to do that. Thanks so much. I have a question. You know, Omar gave us so much content to eventually try to apply this to the nature of the work that you're doing in MicroOne. And one of the things that was coming to my mind is, you know, and equally to reflect on how I started, right, MicroOne, when I was working on human data So human data is at the heart of basically what we do at MicroOne. There's a lot of data collection and we don't call them annotators anymore. I think it's a fair distance from what it used to be. But let's say there is a way to basically label and try to process the data. And this decision that we do are basically operating in the upstream. Eventually when they go to the downstream, we might see some biases emerging from this, right? So when you're looking at this from a perspective of the work you're doing as a member over the technical team, how do you try to intervene to the point that you're realizing that eventually if you don't do anything, the fairness will be compromised, right? So what are some of the conversations you're having? I would love— these are again all of these questions that I'm learning from you guys because I have no answer, right? This— that's the downside of coming from economics and not coming from computer science. I'm trying to learn as I'm asking this question, but I'd love to have your thoughts on this, Nima.

Yeah, I think the number one tool that I have, you know, finally decided on and over my time working at MicroOne, and I think the industry as a whole has also converged to and has been the key tool in the toolbox is diversity of thought. 100%, the best tool to minimize bias is consensus. And this is something that we implement in all of our human data pipelines. Is a lot of times you're trying to take some things that are like, you know, math and physics are extremely objective. They're at the core of sciences. And I always say, you know, like physics builds on top of math, chemistry builds on top of physics, biology builds on top of chemistry, and it continues to go until you get to like philosophy, psychology, theology. It's really, and as you go up this chain, it becomes less concrete, more subjective. But of course there is, psychologists have licenses, you know, there is still objective truth. And there's a standard that we have come to for many things. There's still the gray areas, there's still the edge cases. And the way that we tackle that is through consensus. So in all of our human data pipelines, it is so important that it's not, you know, one group of individuals from some background that is determining you know, this entire, you know, the construction of this entire corpus of, say, medical data that we're using to train some medical model. Our experts are sourced globally. And, you know, people have this impression that these LLMs are, you know, siloed to the United States and their focus is a completely Western lens. This is something that I think a lot of people have acknowledged and a lot of people have tried to remedy. And even, you know, some of our pipelines are quite literally remedying this, like, currently, you know, we, you know, for example, you know, close to half of the world lives in South Asia. Basically half the world lives in South Asia. We need to make sure that as LLM adoption becomes increasingly global, that this consensus tool, this diversity of thought is so, like, even more important. So, We don't want it to focus only with a Western lens, though of course LLMs are most probably adopted in the West, but it is a global phenomenon that needs to be treated as a global phenomenon. And, you know, at MicroOne, this is something we've 100% acknowledged. We need to use many different tools, you know, like, you know, pre-screenings using artificial intelligence to give as much of an objective lens as possible and final, you know, sign-offs of all of our work. Of all of these human data individual points are always by a human. We cannot leave it into the hands of synthetic data generation or like automatic, even like really that we cannot take humans out of the loop here. And though humans are a great source of bias, they're an infamous source of bias, they're also the solution here. It always comes down to that diversity of thought.

You know, Nima, you said something that Omar, you said differently, but it's the same, right? When you said when I'm talking to a practitioner, I just said let's have a conversation, right? It's only that human interaction that helps you eventually calibrate, right? In this case, which brings me to, I think, a trade-off, Omar, that I personally perceive a lot when we were talking about technologies like AI, right? There has been for a period of time almost this obsession on the efficiency of the model, the accuracy, right? How accurate is the model? And if you are trying to introduce fairness, that might decrease the accuracy. It feels like you're creating a trade-off or a cost. Where is the conversation right now? Do you find that there's still conversation that are trying to praise the accuracy over fairness because it's economically more viable? Or do you find that there is more acceptance that actually fairness is a better way of accuracy in the long term? So where do you find that the pendulum has shifted?

So, and there are trade-offs, but I wouldn't put it as fairness versus accuracy, but perhaps one kind of fairness versus a different kind of fairness. Because fairness doesn't necessarily have to be disjoint from accuracy. For example, when you— let's look at medical predictions. and I don't want the predictions to be equal. I don't want to say that every population has the same probability for every disease when some diseases are really correlated with particular individuals. So notions of fairness that are based on equality don't fit in that world. And so sometimes accuracy is fairness. And even more, we were finding more and more— so this multi-calibration perspective is really grounded in accuracy, and we found more and more that, that it has to— that that's a good place to start with. Then if you want to apply things like affirmative action, which I personally support in many contexts, you, you need to do it on the basis of something that's very accurate. And so perhaps in privacy there's been more of a tension between accuracy and privacy. And even there, some of the work we've done in the past shows that privacy actually gives you robustness that is very, very useful even for accuracy. We're finding it even more in fairness. We're finding that we start with the, with, with the motivation of fairness, but then it gives you robust— for example, I'll give you one example. We get this notion of universal adaptability. So you're running an experiment in Stanford Medical School, and then you want to apply it all over the world. How do you know that your findings are still relevant? Notions like multi-calibration actually guarantee it because when you care about fairness, it means that you care about all those subpopulations. And therefore, even if you go to a different hospital where the, the balance between these subpopulations is different, your, your work will still be relevant. On the other hand, there are places in which you need to, to decide that, um, that you value equality in one way or another more than, than fairness. And so it's kind of very, very context-based, and public opinions about it is also very, very much based on the context. And there have been lots of experiments about what do people care about, and it will change if you, if you ask for something in the medical arena or you ask something about welfare.

Mm-hmm. Omar, look, I have to tell you, again, as a colleague in this side, right? Thank you so much. I mean, you have a fantastic way of helping people understand the extent of the conversation we're having rather than coming to a quick conclusion. And I think this is precious to hear it this way. And so look, I love the way you have been able to move us away from this impasse on the trade-off to say, well, maybe every week consider that accuracy is actually better, right? For fairness is better for accuracy. Sorry. And to bring it over to the work you do, right? I'm thinking it's true when you're mainly training on something like Zara, right, which is a signature product that has been renowned around the world for what we do. But we have seen more and more shift towards agents lately. And you could argue that agents are sometimes measured by speed. And it's not always easy to determine the level of human oversight. We had a conversation together with Ali and Aaron from Vox, right, about agents. And so my question is, shifting the trade-off question to you and the work you're doing, how do you guarantee the level of fairness when now the speed of agentic models sometimes is faster than our ability to oversight things? So where do you find that balance happening these days?

Yeah, so this is an ongoing problem that's being tackled industry-wide. These agents are able to go off and, and, and produce very valuable work over very long horizons. And our human data pipelines are adapting to this as well. You know, tasks, the data that's being produced, each data point, say each example of these agents like going out and doing their work, that's trying to replicate a human going out and doing, you know, long horizon work. It results in extremely complex human data that takes even sometimes weeks to produce one data point. And, you know, I think one of the tools here that can really help, and time will tell truly, is these verifiable rewards. So we don't want to take human judgment out of things. The human judgment is extremely important. And like I said previously, you know, subject— there's a lot of things that are subjective and we need to, you know, build consensus around them. And human judgment is how we do that. But verifiable rewards. So having checkpoints at each key moments as this agent goes off and uses tools and navigates maybe even people's personal computers. I know a lot of people are giving full access to their, like basically their lives to these agents, but I'm not advocating for or against that, but we need to treat it very cautiously. These checkpoints, verifiable, rewards to train these models to not accidentally, you know, maybe release your banking information somewhere or publish something online that's not actually your words, or, you know, you can get pretty out there. That's what we need to do. This is ongoing. I mean, I love the question because it is so at the forefront. This is not only MicroOne that's thinking about this. This is something that we're working on with others as well and in conversation with others as well. Maybe indirectly more so because the whole industry is building consensus around this. Zara has actually passed over a million interviews. And so, and these interviews can get increasingly complex, increasingly long-winded sometimes, and even on-the-fly dynamic exercises that are produced. So this is getting into real long agentic flows. Yeah. So I, you know, I love that Micro One is tackling this problem, is at the forefront of it. And, and time will tell what is the ultimate solution, how we're going to get even better beyond, you know, what we already have right now.

Yeah. You know, in your conversation during that, during our, our series today, you kind of COVID a bit more what kind of guardrails, you know, we, we're trying to put in place. And I think there was a question from Nandy about this. So I'm happy that was directly, directly answered. And there was actually Actually, early question from Juan about ethics and philosophy, and you kind of mentioned that they are not decoupled, they're just an evolution, right, of the consensus that you want to build on. So I think there was also something that was somewhat indirectly or directly answered, which, which I find it great. One of the questions that I think, um, a couple of questions over that goes to you, and I'm now looking at the Q&A considering that we have just a few minutes before bringing over to closure. There's one from Reynaldo, and Cecil, I think, equally goes in the same level. So Reynaldo basically say, I'd like to understand better how basically multi-calibration— I think you really made an impact, Omar, when you were talking about that. And I know it's an important part of your research when I was doing my due diligence on that. And basically say, is it possible to disclose the approach or general methods on how multi-calibration could get a quantifiable prediction across domain? I think the question for Reinaldo is, is this something that you see this as an empirical tool that give you a specific sense of strength like we, for example, could do by running regressions? Is this something that gives you signals? Is this something that gives you actually an indication of further studies? So how exact or precise is multi-calibration in the kind of challenges you're trying to deal with fairness over? And that was a question that I think covers both Reinaldo than Cecil.

Let me just say about the question about philosophers and ethicists. Yes. It's a very wide community that is working on, on on this, and we do collaborate. I'm just writing a paper to a philosophy of science journal with somebody who's also a philosopher. But to some extent, I view our work as a work in philosophy, just in a different language of philosophy that, that has definitions and theorems. And, but this is a longer question But what is multi-calibration? It's two things. First, it's a property. You can look at every one of your models and ask, is it multi-calibrated or not? And then it's an algorithm where you can take your models that are trained with all the smarts that you have in the world and make them multi-calibrated. Kind of in a game of auditing versus correcting. You found— you find a population that is mistreated, and then you correct it. And through these stages, very quickly you get into a position where none of perhaps an exponential number of possible affiliations is mistreated. So it's not something that replaces anything that people are doing. It's kind of a property that you want perhaps to impose and a property that in various ways— and by now there are open source libraries in different languages that do that, and it has been implemented and reported to be implemented in high-level medical cases, but but there's a group in Meta that has been reporting a lot of success, internal success, with multi-calibration. And I have my suspicion about other companies in which it works. So it's a very doable thing, but essentially it's a property that we feel is very important, perhaps as a first stage towards decisions of value. But we want to base our decisions on information and knowledge. And then value actually makes a lot of sense then, rather than basing it on values but with no underlying information that sometimes leads to the wrong, wrong outcomes.

Yeah. All right. Well, I do invite, you know, the attendees that have followed us and those that will watch us on YouTube later on to keep on studying your work and reading it. That's how I found out about multi-calibration myself. And I find that I like to read even more, right? Because I think it's a fascinating topic that maybe could be elevated even more in the public discourse and what I'd love to eventually see. So that was great. Nima, look, as probably the final question to you as well, you know, one of the questions that sometimes I have is like we set up maybe standards, we test models, we feel they're fair. And that's great, right? But it could happen that the model drift. How do you realize about this? How do you suddenly realize that what you had defined as a parameter or reference, the standards are no longer working? And the reason why I'm asking this is I think in the business we all understand in this sense of continuous, right, that we need to basically put in practice. But how do you practically do it when you're running this from a technical perspective?

Yeah, there, there's two ways that you have to, that we've come to basically. One, and I've briefly touched on this, is we cannot take humans out of the loop. Humans have the most up-to-date judgment, of course. Um, the, the, the, the second thing is recalibrating. You, you do need to recalibrate always, you know, for, you know, one example I can give is that, you know, internally within Zara, one tool that we have is, is to kind of assess the, um, Authenticity, or maybe like how truthful someone's being about their skill sets, you know, and of course a live interview, there's— we want to make sure that we are evaluating people accurately and those who deserve the roles, those who deserve or who should be placed somewhere maybe differently or placed, you know, more precisely can be done. And this is By evaluating, you know, the honesty of people's interviews. And so this honesty evaluation is constantly being recalibrated. People have, like, you know, the way people even interact with an AI interviewer is constantly changing as humanity becomes more comfortable, for example, with interacting with agents. It changes how they how they speak and how they navigate these interviews. So that's, you know, one example, but of course we're never gonna let any algorithm, even say a proctoring algorithm like this, make the final call on any human being's evaluation. So that's why I wanted to start with the fact that we do not take humans out of the loop. And that is, you know, a key way that we can, combat this drift is all final decisions. Like I said, it's someone's career. This is a real person with a real career and, you know, a real family to feed. All final decisions are made by humans. And so that's the real, um, that's the real kicker to fight this drift.

That's great. Um, Nima and Omar, we are kind of at time. I, um, cannot thank you enough for having taken accomplished questions like this and share it with, you know, we had always over 100 people that were watching live and more that will watch afterwards. This is a topic that people are interested about. And, you know, Nima, I appreciate a lot working with you. It's been, it's been, you know, a place for me knowing people of your caliber. And I learn every single time that I talk to you. Omar, what a pleasure to meet you. I look forward to actually have you back with you. What's the name of the book? Do you have a title?

Yeah, they changed the title, but I like it too. It's Equal Under the Algorithm: A People's Guide to Fair Computation.

That's the current title. And it's— look, I look forward to that. I think I will be so intrigued to figure out where our understanding of fairness moves from now until the time that the book is coming out, because one of the things that is for sure is that we are a generation that cannot take the next 6 months for granted, right? Not in the same way that we conceptually know it. So, Omar, Nima, thank you much for being here. I really appreciate it, the partnership in making this a dynamic one. I'll prepare a bit of an article that will try to synthesize my thinking about this work. But I am thankful for the time you've taken in the middle of your schedule. To do this with us. From Mark and from the series that we're running at MicroOne, all the best.

Cheers, cheers. Bye-bye, guys. Thank you so much.