#237 Guardrails for the Future of AI with Viktor Mayer-Schönberger, Professor of Internet Governance and Regulation at the University of Oxford

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com.

So whenever there is a very straightforward, efficient decision to be made, then that can be automatized. But whenever it gets more complicated, whenever the context is changing, whenever goals are more muddy, sometimes contested, or battling with each other, then I think it is perhaps better to keep the human involved, not just in the loop as sort of at the end pressing the button, but to be really involved in the decision making process. I think there is also a good division of labor, if you want, between AI and the humans. AI is very efficient for alerting us and focusing us on a solution that is already available. Humans are perhaps better positioned to come up with novel solutions.

Welcome to Data Framed. This is Richie. In theory, it sounds brilliant if everyone has the freedom to do anything. Unfortunately, given enough time and people, someone will do something stupid. That's where guardrails come in. Whether it's a process or standard operating procedure at work, or a countrywide law, or simply a rule set by parents for their children, guardrails provide guidance to encourage people to do the right thing. In this episode, we're going to learn what makes a good guardrail and when you might want them at work or in life. Our guest is Victor Meyer Schoenberger, a professor of Internet governance and regulation at Oxford University. Victor's research currently focuses on how processes, data and AI are used to make and govern decisions. He's the author of eight books, including the award winning Delete and the international bestseller Big Data. His latest book is Guardrails, co written with Urs Gasser, which forms the basis of today's discussion. One of the big arguments in the book is that data governance isn't enough. What you really want is decision governance. I'm always keen to find out how to make less stupid decisions. So let's find out what Victor has to say. Hi Victor, thank you for joining me on the show.

Hi Richie, great to be here.

Wonderful. So, since we're talking about guardrails today, to begin with, can you Explain what is a guardrail?

A guardrail is a guidepost. It helps us in our decision making. I mean, we make lots of decisions every day, literally thousands of them. But some of them are really consequential. And if we make the wrong decision, that could have huge consequences. That could mean difference between life and death. And so making good decisions is really helpful. And having somebody that can provide some guidance on how to make good decisions is even better.

Wonderful. So I like the idea of tools to help you make decisions. So can you give me some examples of how guardrails might be used in a business context?

Sure. But let me give you an example of what a good guardrail is first, outside of the business context, just so that you understand and the listeners understand, what's the difference between a good guardrail that's flexible enough and adaptable enough and something that's really not good. So a really good guardrail is, for example, the rule that we should drive on the right side of the road if you're in North America, or on the left side of the road if you're in the uk. That's a guardrail. It helps us avoid accidents. It's hugely efficient and beneficial. But we can break that guardrail if we want to. For example, when we want to overtake a car and there's no other car coming our way, we can actually change to the other side and overtake that car. The real good guardrail helps us, guides us in our decision making. But we are still in the driver's seat. We still make the decisions. It doesn't nanny us, it doesn't tell us what we do.

It doesn't.

It empowers us to make good decisions, but it doesn't go beyond that. That's a good guardrail. So in the business context, for example, if we now bring it back to the business context, a good guardrail is one that helps the manager, that helps an organization to reach its goals, to reach its aims. And that may come in all kinds of forms and shapes. Standard operating procedures in companies, our guardrails, in fact, standard operating procedures in airplanes, are guardrails too. And pilots should keep to the standard operating procedures, but there are exceptional circumstances where they can break them. And the same is true in the business context, of course, as well.

Okay, so it sounds like there's quite a range then, from just sort of simple rules of thumb in order to give you good advice through to processes and procedures up to, I guess, the laws count as guardrails as well.

Absolutely. And some laws are quite strict and you can only do a certain thing. And other laws are quite flexible as well. And keep in mind, there's a lot that we cannot speed on the highway beyond in the US 55 or 65 or sometimes 75 miles per hour, but we can still speed. We just have to live with the consequences of a speeding ticket, for example. In other words, that is a pretty good guardrail because it contains and it has that flexibility built in. And similarly, in commercial context, a good guardrail helps decision makers, but doesn't disempower them, doesn't take decision power away from them.

Okay, so it's like this is a really good idea, but you have the freedom to break it if you deem fit. You mentioned that sometimes making decisions can be a life or death situation. Do you have any examples where a lack of guardrails has caused some sort of problem?

Yes, and if you forgive me, I'll offer an example from the aviation history. Again, we open our book, Guardrails with this very stark story. It's about 20 years ago, 20 some years ago at that time, there was a new device or relatively new device that was built into commercial airliners to avoid head on collisions. It was a collision warning system. And these two boxes in two airplanes that got too close to each other, they would negotiate and one would order the crew to ascend and the other would order the other crew to descend, thereby avoiding the head on crash. And when these boxes came around, initially there was no good guardrail, no good rule that mandated that pilots in all or almost all circumstances should follow the directive of the machine to avoid the collision. And so two airplanes over the Swiss German border got very close to each other. Both had those boxes in there, but only one airline had a standard operating procedure and the pilots complied with it and thereby descended. The other side did not believe the machine did not have standard operating procedures, not guardrails in place, descended as well. A crash and 100 people died.

That's about as bad as it gets in terms of outcomes there. So that's a pretty tragic event. So yeah, I can certainly see having a procedure and making sure that other people are forwarding that procedure as well can be incredibly important. All right, I think we need a happier story to balance this. So do you have any examples of when guardrails have been helpful and there's been a positive outcome?

Absolutely. And you see already, as I'm drawing in examples from the aviation industry, from automobiles and driving all the way to the commercial context, guardrails really are everywhere. One of the happy outcomes In a way, with respect to guardrails is from Africa, where there are oftentimes contested resources just like water. And in West Africa, a number of small nations depended on the Volta river water supply, but some nations took more water than the others. And so they came together and in a very inclusive process, came up with guardrails, with a framework on how to divvy up that limited resource, and then ran by it. It was not a fixed framework. It was somewhat flexible. It also needed to be adapted to changing uses and changing context and also climate change, of course. But overall, it was a very positive, as it enabled not just a limited number of people to get access to good water, but a very large group of people that transcended a particular jurisdiction or a particular nation.

Okay, yeah. So certainly different nations agreeing on how to share resources feels like quite a tremendous thing. It doesn't happen very often. They're more likely to argue or go to war with each other over resources.

So what's really interesting is I think that you would never think that this was easy or possible, but it actually happened. So we are capable as a species to come up with quite flexible, but also quite pragmatically operative, efficient guardrails.

Okay, so I think you've convinced me that guardrails can be very useful. Now, I'm curious as to what makes a good guardrail. So you mentioned before that they need to be empowering of individuals. Are there any of the principles like that that make a guard Good stuff?

Right. If I know exactly what I want to achieve, then I can design the guardrail. I can design the regulatory mechanism to achieve that goal. The problem is, what if the goal is somewhat wobbly? Or what if the context may change? Think about artificial intelligence regulation these days, and how do we go about this? It seems to me that the European Union, with its AI act recently passed, seem to understand perfectly what the problem was and then create the perfect solution for the problem. Except what if they didn't understand the problem very well? Then the mechanism to achieve the goal may ultimately turn out to be quite inefficient. So in other words, if we don't really know what we want to achieve or we don't completely understand the problem, we need some guardrails that are perhaps a little more flexible, but most importantly, help us to learn, to learn from the decisional mistakes that we make, and also help us to then adapt the guardrail itself so that it is better suited for the context in which we're in. And that's really hard. That's unbelievably difficult to do, especially for classical state regulators or multinational regulators who want to lay everything down in great detail so that there is no ambiguity whatsoever. Urs and I in our book argue that when we don't really understand the problem completely yet, one of the design principles is to build some flexibility into the guardrail and some ability for the guardrail to be adapted, but also to create guardrails that help people, people learn from their decisions that they make so that they don't make the stupid mistakes. Again, that sounds sensible.

It also sounds a bit like the agile software approach coming to creation of guardrails as well. Like you need to do something and then a few weeks later you review it and change your mind about what you're doing next.

Yeah, and what is interesting, Richard, that you bring this up when we talk to programmers or people in the software industry, they say, yeah, exactly. That's what I do a lot with agile programming and all that, because oftentimes I don't really understand my user perfectly yet and I need to adapt. I need to also adapt to new user wants and new user preferences. The problem is really that the regulators haven't understood this. The regulators have, over the last 150 years, gone from relatively flexible rules to ever more detailed and tight rules which squeeze all the flexibility out of the software system.

Actually, that's interesting that you mentioned flexibility and having very detailed rules. So my understanding of like very lame's understanding of law is like in the US you have these very deep, detailed rules, and in Europe you tend to have like broad rules that sort of covers everything, but to a lesser extent is one better than the other when it comes to guardrails? Do you want deepness or broadness?

The truth is much more sobering. The truth is, yes, in the United States we have case law, and that is quite detailed, but we also have statutory law rules that have been enacted, lots and lots of them. And by the way, in Europe we have pretty much the same thing. So when you look at the number of laws that have been passed either by legislatures in the United States or by legislatures in Europe, whether it's the UK or the European Union, you pick. It has grown almost by leaps and bounds over the last two decades or so. Legislators have gotten incredibly productive in enacting laws which of course limits the ability to be flexible and to stick to flexible rules. And it may reduce ambiguity and interpretive flexibility, but it also creates the potentiality for statutes that, that are outdated and no longer are in sync with the reality on the ground.

Okay, that's very interesting. And I had no idea that there were just dramatically more laws than there were a few decades ago. All right, so the example you gave early on was about speed limits in cars and how there's a sort of law there that says you can't go fast in this, but individual drivers have the ability to speed if they want to. So it seems like a good guardrail is one that people are actually going to follow, at least most of the time. So how do you design your guardrails to make sure that they are actually followed?

Mostly by keeping your customers in mind, much like a good user interface designer. Keep your customers in mind and their goals, what do they want to achieve? If a guardrail helps people ultimately to achieve their goals, and if you can communicate that, then they will stick to it. If you tell people that the speed limit of 65 rather than 85, they still reach their destination within just a couple of minutes later than they would otherwise. But with 20% less fatal crashes and accidents, then that might be the sensible thing for a lot of people to do. And you may not ever reach 100% compliance, but you don't need to. If 90% or 95% of the people follow the rules, that's pretty darn good. And that reduces, for example, in car traffic, reduces accident rates dramatically. So what you are trying to do is basically look at what people want to achieve, try to empower them, and try to communicate how you empowered them and hope that you did it well.

Okay. So really it's about, like, making sure that people are aware of, like, what the guardrail is. So I guess in that case, it's the signpost that tell you what the speed limit is and also educating people as to why the guardrail exists. So if they know why it's there, they're going to want to.

You know, I'm a rock climber, and a lot of times when I climb, there's fixed ropes, and you clip into the fixed rope and then you climb up and you're always secure because you're clipped into that rope. But there are sometimes there are people who just come from behind and want to overtake. They can unclip and they can overtake you and then continue on, but it's their own decision and they have to live with the consequences. And I think that's the important point. We don't want to nanny people. We don't want to disempower them. We don't want to take the responsibility. But also their freedom and liberty Away from them through the guardrail, through well designed guardrails. We want to empower them to make better decisions without actually caging them.

I was getting mildly terrified about people unclipping themselves from a rock face and trying to overtake. That just sounded incredibly dangerous. But, yeah, I can see how you'd be want it to be their own decision to do that rather than be standard procedure. All right, I'd like to talk a bit about decision making. So you mentioned, like, one of the goals around guardrails is to help people make better decisions. Could you just talk me through how this works?

For good decision making, we basically need two elements. One, we need to have the right information available in order to make our decisions. And then two, we actually have to somehow weigh and calculate and balance all the information that we have and factor in our preferences in order to come to a decision option. And that's the hard part. If you think getting all the information in place is the hard part, just think about the fact that we have learned over the last 40 years or so that we have numerous cognitive biases that shape our decision making and oftentimes hinder us in making the right rational decision. Whether it's confirmation bias or availability bias, there's all kinds of biases that we are, as humans susceptible to, and there is no easy way to untrain those biases and to get rid of them. So what cognitive psychologists have said is that rather than oftentimes trying to find the better between two mediocre decision options, it is better to broaden your decision space so that you have more options available, which ultimately may lead to better outcomes. We are not particularly good at choosing with our biases between two options, but we are, as humans, pretty good in coming up with lots and lots of decision options. If we try, we can kind of dream up new decision options relatively well. And so good decision making should therefore provide us with good information, but then also help us to broaden our decision space and then to navigate that decision space appropriately.

Okay, yeah, that certainly seems to be a good idea to help compensate for all these sort of human biases we have. Like, the tricky decision is, do you want the salad or the French fries? It's like you're going to be biased towards one. Okay, so it does seem like there's often a lot of uncertainty in decision making. And you mentioned that sometimes you don't know quite what your goal is or what your users want. Can guardrails be used to help with this level of uncertainty?

Yes, they can. Now we need to Understand that there is no certainty there. Uncertainty means that even if you have guardrails that work 80% of the time, it means 20% of the time they're off. But they were sort of a good starting point, if you want. Atul Gawande, an author and medical doctor in the United States, had a wonderful book out a couple of years ago called the List. And it was a list of standard procedures in the emergency rooms for in US Hospitals. And what he showed was that if doctors stick to the list of the 7 or 10 things that need to be done in this order when a new patient comes in, it significantly improves the chances for a positive outcome for most patients, not all patients, because you have those odd cases that unfortunately the list doesn't cover and it may make the situation actually worse. That's why you need the empowered physician who can then kind of discard the list and say, in this case, I need to do something else. But overall, help us. These standard operating procedures, this list help us to cover the most obvious cases. And that's why they're useful.

Okay, Absolutely. I'm a big fan of checklists. I'm quite often forgetting things, particularly admin tasks, where I can't hold them in my brain. So having those checklists just help me go through and make sure that I'm doing absolutely everything I should be doing. All right. So you mentioned one of the big principles of this is about making sure that you have enough flexibility around the guardrails so that people can be empowered. So what's your strategy for deciding? Is this guardrail too strict or too lenient? How do you get that balance right?

These rules of thumb that I was alluding to, whether it's the rule of thumb that a guardrail should enable learning, or whether it's the rule of thumb that the guardrail should facilitate the empowerment of individual decision makers, Those are more like design principles. They are not fixed rules themselves, but they are kind of principles that should guide our designing of the guardrails. And so the designing process is not a scientific one. It is artistic in a way, and it requires a lot of trial and error, which by the way, is very difficult for regulators, as I mentioned before, because regulators usually want to regulate and then they want to forget about the regulation for the next ten years at least. And that's not what flexible good guardrails are. They need to be constantly checked and fine tuned and adapted to changing environments. And so what we require is not just a good guardrail, a well designed guardrail, but what we require is an institutional structure around it that can kind of revisit it constantly or frequently and adjust it. And to set that up is even harder than to come up with a flexible guardrail. A flexible guardrail doesn't help you if you have no institutional structure around it that can then step in and increase flexibility or sort of rejig the guardrail one way or the other. So you always need to think about the sort of institutional environment, the context in which guardrail exists, and how to make sure that you stay on top of keeping it flexible and adaptable.

Okay, so it's not just the guardrail itself, it's the all sort of the institutional framework to say, is this actually going to work or not? All right, this is getting slightly abstract, so maybe we need a concrete example. Can you just talk me through some examples of guardrails in business that are used for helping you make decisions?

Think about environmental laws. For example, you can try and limit the amount of carbon dioxide that is emitted by factories. And you just have a law that says no factory can emit more than X number of tons of carbon dioxide a year. That's a very inflexible, very fixed rule. But it's very clear, doesn't have any room for interpretation or very little room for interpretation. Think about a very alternative concept where you say, look, every company, every factory that's emitting carbon dioxide gets a emission certificate. And if you don't emit as much carbon dioxide as you have the certificate for, you can trade that certificate on an emissions market and emissions certificate market. And every year we cut 10% off the total amount of certificates that are available. So what we do is we sort of set the goal, but we do not define predefined the pathways that individual companies set to reach that goal. And that's a pretty flexible guardrail in a way, because it leaves open a lot of different ways to achieve the envisioned goal. And it creates incentives for innovation. It creates incentives for companies to even go beyond what the current limit is because that gives them an opportunity to trade their emission certificates and make extra money of it, all these kind of things. This is a pretty flexible framework and a pretty flexible setup. It uses the market as an incentive structure. It uses innovation and human ingenuity that's built into it. And it is capable of sort of lowering the limit of carbon dioxide emissions over time, while the hard rule just sets a limit and sticks to that limit.

That's interesting because it seems like the focus there is less about having rules in place to enforce stuff in the Same way the laws are. And this is more about the focus is on this is the goal and these, the incentives to make you adhere to this goal.

Yeah, and it helps individual decision makers in the companies to make the right choices because it creates incentives for them to make certain choices rather than others.

Now, you mentioned that you don't always get these guardrails right the first time. So you need some process of, well, having feedback to make sure you can improve the guardrail later. Can you talk me through, first of all, how do you test whether or not your guardrail is actually working?

Absolutely. And that's one of the hardest parts, Right. You have a guardrail and a guardrail is working if it helps people achieve their goals more effectively than before, whatever their goals are. We're not going to judge the goals here. The guardrail is a mechanism to achieve an exogenous goal. Now, measuring that is actually not easy. And when you start measuring it just in terms of economic impact, a sort of cost benefit analysis of some sort, then you are starting to measure something. But you're not capturing, of course, the full comprehensive picture. You're just capturing what can be economically captured in data. So it's a good first step, but it's not complete and comprehensive. And what we need to do is to therefore develop better measuring tools to measure the effectiveness of guardrails. But we are at the beginning of that. But we have made quite some headway going beyond simple cost benefit analysis, capturing longer term externalities and these type of things. But there is a long way to go. However, having said all of this, the truth of the matter is I've advised politicians and policymakers over the last 25 years. An ugly truth is that they don't even do, most of the time a simple cost benefit analysis. Even that would be better. Then what they're doing is oftentimes shoot from the hip. So even small steps towards getting some assessment of effectiveness would go a long way to improve guardrail design.

Okay. Yeah, it does seem like if you're proposing a law, you should do some kind of thinking about what are the costs and what are the benefits.

I advised a German government and that was one of our recommendations. It was the one recommendation that immediately got binned.

That's a little bit worrying. Okay, so it seems like a lot of the key then to being able to test whether a guardrail is good or not is the goal that the guardrail is for well defined. So I know there's talking like smart goals where they're like testable and all that. Kind of stuff. So if you've got a well defined goal, then hopefully you should be able to test what's happening. So I think we've mostly been talking about guardrails so far in the context of helping humans. But now AI is sort of reaching a point where it can automate some human decisions. Can you talk me through? First of all, when should artificial intelligence replace human decision making and when should it complement it?

That's the $100 million question at the end, isn't it?

At least.

Or even more than that. I think the answer is relatively straightforward. It may be surprising, but it is relatively straightforward. If the decision making is very routine and doesn't require change, context or anything like that, then there is no reason why the machine shouldn't make the decision. If you enter an elevator and you press the button for the third floor, you let the machine make the decisions of closing the door and getting you to the third floor. That is a very efficient process, and you don't need to question that, never mind artificial intelligence here. So whenever there is a very straightforward, efficient decision to be made, then that can be automatized. But whenever it gets more complicated, whenever the context is changing, whenever goals are more muddy, sometimes contested, or battling with each other, then I think it is perhaps better to keep the human involved, not just in the loop as sort of at the end pressing the button, but to be really involved in the decision making process. Not because the human makes the better choice between two mediocre options, as I imagine, as I said, but perhaps because the human might be able to come up with a third or a fourth or a fifth option we haven't yet considered that is actually better than the options that are already on the table. And so in that sense, I think there is also a good division of labor, if you want, between AI and the humans. AI is very efficient for alerting us and focusing us on a solution that is already available. Humans are perhaps better positioned to come up with novel solutions.

Okay, I like that. So if you need some kind of creativity or there's some novelty in the situation, then probably humans are going to perform better than AI.

Oftentimes, it's important to then keep humans in the loop, even for relatively routine decisions. I'll give you another example from the aviation industry. Some years ago, Asian commercial airliners were always using autopilot to autoland their airplanes when they came to the US because it was very smooth and everything. But then one day, the autoland function at San Francisco Airport was down, and a very large Asian airliner had to land manually. Beautiful weather, everything. But they crashed the aircraft and people died. And that led airline companies all around the world to mandate that their pilots have to continue to land by hand so that they continue to train, experience, learn what they're doing and how to land the aircraft so that they know what they need to do when they have to do it.

That's, again, quite a horrific tragedy there. And I can certainly see how having that continuous training is going to keep people keep their brain sharp and make sure that the quality of the landing when they have to do it is getting better. Like the similar thing in the data world, there's like, generative AI can write your code for you, and it's fine until you don't have access to it, and then you need to write your own code. Okay, so if you are designing guardrails for AI, then what would those look like? Are they going to be different to guardrails for humans, or is it just the same sort of thing?

No, if you design guardrails in the context of AI or how AI should be used, it's important to keep in mind that same design principles that we mentioned before, for example, to empower individual decision making. So rather than delegate away a complex decision from the human because it's too complex for the human, that's not a good idea. We should keep it with the human, and we should provide some guidance on how options could be generated, how the options space could be broadened. We should also understand that guardrails should be designed to enable human learning rather than AI learning. It turns out that cultural learning is actually really powerful and has propelled our species from a relatively middling mammal species about 100,000 years ago or even 25,000 years ago, to something that can actually land on the moon. And there is no other mammal that even aspires to do that, at least to my knowledge. And so, in that sense, learning and cultural learning is hugely important. And we can design AI systems to enhance and enable and facilitate human learning rather than take that chore away from the human and embedded in the AI.

Okay, that's kind of interesting. And I like the idea of cultural knowledge. And you can sort of build on what other people have done before you. Yeah, there aren't many other apes going to the moon, I suppose. Okay. So I think this leads to the idea of, like, transparency of how decisions are made. So certainly once you've got AI in there, sometimes you end up this black box. So when do you need to worry about how decisions are made rather than just the answer is a big question.

It's a big question because behind it lurks an even larger question about causality and the role of causality in our understanding the world. And of course we are living beings that make sense of the world through cause and effect. And that means we always want to peek in the black box, even when it is not possible. Now what is really interesting is the truth is other people's minds or our own sometimes is a black box. We may arrive at a decision not completely knowing why we came to that decision in the first place. And then we take that decision and then we post rationalize. And the goal of the post rationalization, of course is to tell us a there is cause and effect and B we knew it all and calculated it accordingly. And that gives us that warm and fuzzy feeling that we are in charge. As post rationalization, I am not sure that actually we are so clear eyed. In fact, I think we are quite often deciding based on black boxes. So maybe therefore, dare I say, but that's tentative. Dare I say it's less important to peek into the black box of AI than to understand and make sure what role the AI box fulfills in the decision making process. So what is the tool for rather than how does the tool work?

That's interesting. Now I can certainly see how you got that sort of confirmation bias way something goes right. I always think, yeah, definitely I did that on purpose. But if it goes wrong, then we just sort of quietly forget about it or blame others. Oh, blame others, yes, that's another good trick. So you think you just need to define what the role of AI is within the decision making process. Can you make that concrete? Have you got like a real example of how you might go about doing that?

I think I mentioned that before already and that has to do, for example, with positioning the AI as something that does not take the decision power away from a human being, away from a decision maker, but rather facilitate the ability of the decision maker to, for example, broaden the option space. So in concrete terms, if I'm facing a decision, I shouldn't ask ChatGPT which of the two choices should I go for, but ask ChatGPT tell me all possible options. That gives me a sort of a lay of the land, a sense of the option space available, and then use that as a jumping board to come up with even further additional decision options that perhaps others haven't considered. That is taking the AI tool and utilizing it, giving it the role of facilitating and enabling my coming up with additional decision options rather than Delegating my decision making to the AI box.

I like the idea of an option space. It reminds me of when you have small children, you say, do you want to eat the carrot or the broccoli? And they never think about the third option that they could ask for ice cream. So it ends up encouraging them in the right direction to eat vegetables.

And by the way, if I may add, one of the interesting things is that children hone over the first couple of years, from about year one to year four or five, hone those what if skills, their skills of sort of thinking in larger option spaces. What if I don't choose broccoli or carrot and go for something else? Eventually, by the age of three or so, they will be there.

Okay, they learned the scam very quickly. All right, so just going back to the relationship between AI and human centered decision making. When you have these systems that are sort of partially automated and you have some model or AI saying okay, the answer is no, it's very easy for a human to say, oh well, computer says no and they don't do any thinking. So how does using AI change the accountability in decision making?

Yeah, exactly. That's a great question in a way. I alluded to it when I gave that example of the pilots that did not know how to land the aircraft anymore because they were so reliant on their technology. It wasn't AI, but it was a similar technology that kind of took their decision power away from them, that they forgot how to do it in the first place, that they had unlearned that particular skill in a way. And decision making, decision is a powerful, but also a hard skill to learn. And it requires constant practice. We are lazy, very lazy beings. And so we don't like to make decisions. It's easiest if you can delegate decisions and don't have to make them particularly hard ones. And that's why we have to practice system. And so what we need to design AI systems for, and in general decision assistance systems for, is to force us to continue to practice our decision making and to help us get better at it, to learn, to experiment, to help us experiment to an extent, but not to nanny us or to take the decision making power away. And that's a very tall order because of course there is a huge temptation to say we're going to use a machine because the machine is going to reduce fatality rates, accidents rates, whatever, bad decision rates significantly. But the problem is that if we use the machine to make decisions and we use generative AI that's trained on training Data of actual decisions that generates new decision data that then the AI can be trained with. Again, if we do that over and over and over again, the option space is narrowing because the AI will always make the same decisions. So the training data will always confirm an ever narrowing band of decision options. That only works if the reality doesn't change. If none of our goals, none of our preferences, none of the context ever changes, if it does, we require diversity in our decision space, which the machine cannot provide us easily. That's why humans are better have a leg up there. We kill each other once in a while making bad decisions, but overall experimentation has furthered the species.

This is like very close to the idea of model drift in terms of machine learning, where it's like the reality of what you need to make decision about is change from what the model's predicting and the cause of the model decreases over time. Okay, interesting. All right, so in business, suppose you're a manager. You're now convinced that guardrail is a really good idea. What do you do in order to start increasing the number of guardrails or implementing them in your own team or company?

Start an inclusive process and bring the various people and stakeholders in. Because like you're designing a user interface, you are trying to bring the stakeholders in and trying to get them to accept some guardrails. And so what they need to have is a say in how they are designed and what they're aiming to do. To the extent that you do that, you have a higher chance of compliance, you have a higher chance of acceptance, and you may also end up with higher quality guardrails at the end of the day. So that's very important. The process of generating the guardrails is an important one. It takes time, it takes effort, it's costly. But at the end of the day, you come up with something that's quite flexible in the book. We conclude the book and the final chapter with the Rule of St. Benedict monks and the Order of St. Benedict. These rules are a thousand years old and they were designed to be so flexible to start with that they're still in place today because they have had flexibility built in and they had a lot of buy in as well over the years and generations of monks that were involved in it and fine tuning it. That's an example for a good buy in if you want the design of the Creative Commons licenses or some of the sort of Wikipedia setup or also good buy in designs, Volta River Resource example that I gave earlier as another one. But keep in mind, in all of These cases, this wasn't a cheap overnight process. It took real effort to get all the people involved at the table to get them to discuss and clarify the goals, the preferences, to make sure that the mechanisms, the means built into the guardrails are robust enough and are effective enough to further the goal in mind. All of that doesn't come easy or doesn't come cheap.

I like this idea of just speaking to all the stakeholders, like figuring out who's going to care about this and getting their opinion on what they actually want. I think that's going to help define the goal more clearly. And probably you'll find out if there are many problems up front before you put this card rate in place. But since that sounds like it might be a fairly sort of time consuming process and quite difficult to set initially, are there any examples of really simple guardrails that make for a good first guardrail project?

Yeah, I wish. I think there is actually a very pragmatic way to start and that is to come up with a guardrail, but give it a sunset clause and say, okay, we'll use that guardrail for the next X amount of time and then we'll take stock, we experiment. I think that in itself is incredibly helpful because it also tells everyone around the table that this is now in place, but it's not edged in stone. It's something that we'll try out and we'll then huddle and reconsider if it was the wrong choice and it is perfectly okay if it was the wrong choice and we get rid of it. That's just how we learn. So also have a bit of an arrow culture in your organization is useful. Spotify is famous for its error wall. A fabulous way where if a sub team at Spotify makes a mistake, especially if it was a costly one, they have to write it on a little piece of paper and put it on the mistake wall so that others can read it. The idea is that you should always make an error, a mistake, it just happens. But you should not make it twice. And that's a way to learn from other people's mistakes. And that's not a way or a strategy to blame others, but a strategy to learn from others. There we are again, Cultural learning, standing on the shoulders of giants. And if you have sunset clauses for guardrails, then what you're signaling is this is not permanent, this is just temporary. And we're just starting off with this. But we understand that we may need something very different.

I like that idea of the error wall. I think I Probably need to. I need to get a bigger office just to fill all my mistakes on it. But, yeah, I can see how you want to make sure that it's very clear that this isn't a wall of shame. It's a learning tool. Because, yeah, there's a possibility that could get very misinterpreted. Okay, so we're talking about, like, trying to get in the easy first project. On the flip side, are there any guardrail projects that are likely to have a high impact on your business? Like, where's the big money in guardrails?

In most organizations, the big money in guardrails is when you look at the role of people and what the role of people is. If the role of people is to make standardized decisions or relatively standard decisions, then that is not a particularly good utilization of human ingenuity, of human fantasy, of human dreaming. And then you need to rejig the role for this particular poster position, it seems to me, because otherwise you are utilizing humans for something that they are not particularly effective at. And similarly, if you are using AI and AI tools for things that require creativity or unbelievable dreaming of things that don't exist, then you are misusing, if not abusing, AI. AI is a phenomenal tool to tell us what's already out there, what's known, what kind of pattern are implicit that we may not have recognized as important yet. And so, in a way, sharpening and shaping the roles in organization that people have can unleash enormous amounts of productivity that I think has not been tapped into. That's latent productivity.

Okay, I like that. If you've got routine decisions, then that's something you want to automate away. If you've got something that requires creativity, you want to get all your best humans onto that and get them being creative and having interesting thoughts. And in terms of who gets involved in creating guardrails, you mentioned that you probably want to get all their stakeholders together to talk about stuff. What does that mean in practice? Like, who's going to be in charge of setting up a guardrail? And what can you give me, like a. Have you got a concrete example of, like, who might be the stakeholders for a particular guardrail?

Obviously, that depends on the project. But let me take the example of Creative Commons, for example. With Creative Commons, you had content providers as stakeholders, but then you also had those that were involved in the design of large platforms, online platforms like Wikipedia, that utilized a lot of content stakeholders. Then you had some of the large search engines, like Google, for example, being part of it, because you want to be able to find Creative Commons content easily and straightforwardly. That's findability. Then you want to have some of the regulators and lawyers at the table so that you're not constructing something that is against the law or has other regulatory problems. And at the end you create a user community that is quite heterogeneous. And when you talk about Creative Commons, a lot of people say, oh yeah, that's that, that licensing scheme. Or they say, oh yeah, Creative Commons, that's the, the kind of huge repository of content that can be easily used and reused. But the truth is that that's just the hardware, the wetware around it are the conferences, the online discussion groups, the forums, the hundreds of thousands of hours of volunteers that people put in in order to evolve the Creative Commons licenses over 20 years, for example. And what was quite interesting in the Creative Commons process is that they were quite inclusive, welcoming NGO groups, but also welcoming industry stakeholders as long as the goal was clear and remained clear to create an easy and straightforward copyright licensing scheme that was machine readable, standardized, and enabled the cost free reuse of intellectual property.

I like that. It's not just people getting together as being the sort of method of feedback. We also got like discussion forums and reports and all that kind of stuff. And there's a lot of sort of communication infrastructure going around that then as well.

Exactly. And we have a lot of the wonderful digital tools that enable us to do that. You know, you have wikis and you have forums and then you have zoom. And then there's lots of ways by which we can engage with each other. And then of course, we can also engage with each other face to face in actual conferences and workshops and symposia.

Okay, wonderful. All right, so I'm hoping people have got some good ideas on how to get going with creating guardrails then. Do you have any final advice for anyone who's interested in this?

Just get started, get going. The one mistake that you can make is not start, because then you will never have a good guardrail. Only when you start. Yeah, you'll fail. You'll fail half a dozen times or a dozen times, doesn't matter, get up, dust off, rejig your guardrail and continue doing it. We all have started like that. Every good guardrail framework started like that. But I think we don't have any other chance than to have good guardrails to improve our decision making so that we can better face the challenging and perhaps even existential questions that we face as a species.

All right, super. I like that. Just get started. That's good motivation. Excellent. Thank you for your time, Victor.

Thank you very much. Richie. Wonderful to talk to you.