#349 From AI Governance to AI Enablement | Stijn Christiaens, CEO at Collibra

In the eyes of the world, everyone on Earth, AI became the GPT. AI governance naturally evolves alongside it. Clearly, people are not focusing on the controls or on the brake, if you will. They're just pushing the gas pedal.

Data and AI governance teams often get a bad rep in trying to prevent disasters. They often become the team that says no.

Governance is a control mechanism, right, over a management layer. Most people associate the word governance or control with a negative. To control a system, you need first to have visibility on the system, right? You need to have transparency. You need to know what's out there.

I want to know how to change unhealthy cross-team conflict between builders and governors into healthy enablement in order to ship things safely. Our guest is Stan Christians. He's the chief data citizen and co-founder at AI governance platform Collibra. Stan spent 18 years at Collibra, having been COO and CTO. Currently, he runs data strategy, data infrastructure, and product evangelism.

Don't complicate it. Make it practical. Make it about not talking, but about rolling up the sleeves. If you want an LLM to perform better, you gotta give it better data, better quality data, better quality context makes the model do better things.

Let's find out how to do AI governance the right way. Hi, Stan, welcome to the show.

What's up, Richie? Good to be back. How are you?

Doing great, thank you. So to begin with, I need some disaster stories. Tell me like what's the biggest problem you've seen from poor AI governance?

Oh, disaster stories. I gotta flip through my list of them. So I think there's like a whole wide range of them. I think we've all seen examples like, you know, chatbots recommending a competitor's product. Or, you know, people getting a policy and then following the policy from that chatbot turning out that is a completely made-up policy, but, you know, the airline or the insurance company still has to sort it out. And then it gets, you know, really dodgy in my view because what I learned recently about Claude, right, the Claude bot is that it's powerful because it has all the Linux commands. Now, Linux is a very nice operating system with the Bash operator, so you can do really a lot of things, especially if that machine is connected to something else like the internet. But the horror story is there that people actually do this. People actually will have taken AI from the internet and run it on a machine that can do whatever as root. And then, you know, that leads into all sorts of horror stories. I mean, most recent social media phenomenon, Heat, was around that. You know, the social network for, for robots, right? Where they started talking about conquering the world and turns out 80% of them were people in the first place. Really? Yeah, yeah, yeah. So I think the horror stories are all over and we'll see more, quite frankly, because clearly people are not focusing on the controls or on the brake, if you will. They're just pushing the gas pedal. They're excited to play with the technology. The technology is evolving, so we'll see more horror stories come out. But we'll learn as people, as society, as organizations, we'll learn how to use this new technology widget better.

Absolutely. And I think there's quite a wide variety there. So I think a lot of the chatbot cases have been sort of fairly well publicized over the last few years, like the idea of chatbots saying things to customers that they shouldn't have said, and then the company being liable for that. But this Claude bot thing, that's new. So these sort of computer use tools where they can take over your computer. I mean, yeah, there's a lot of security risks there. Maybe we'll dive into that more in more depth later. But for now, I also need some motivation. So can you talk me through any success stories you've seen? So do you know about any companies where they've put a lot of effort into AI governance and it's paid off?

Yes, of course. We've got a number of them on Collibra's website where we literally advertise the stories. We have worked with, for example, McDonald's, for example, or Siemens or other companies. So the organizations that are really serious about working with this AI stuff, they know that they have to— this is not, you know, I put AI in my company and I push a button and tomorrow everything's solved. They know this is a new technology. It's a platform shift. So they're going to have to roll this out across, you know, functions, departments, lines of business. They're going to, you know, recreate new business. And they know that, you know, just like with the mobile phone or the cloud before, they know that you need to put some controls around this, right? You need to have some maturity with respect. I mean, yes, experiment with the technology to learn what it can do, but also experiment to learn how you need to do it well so that that liability that you were talking about earlier doesn't start to become a problem.

With governance, that's one of the things where it seems like a lot of it's about avoiding liability, avoiding, reducing risks, and avoiding problems. Are there any kind of positive business metrics that come out of it? Like, can you talk about, I don't know, like improvements in revenue, improvements in productivity, all this kind of stuff? Are there any like more positive metrics rather than just avoiding negative things?

Oh, absolutely. I mean, I want to be clear about that, Richie. Governance, whether it's, you know, data governance or AI governance or You know, even corporate governance. Governance is, is a control mechanism, right? Over a management layer. And it has the purpose of making that management layer for data, for AI, for corporations run better and run well. Uh, that's it. So imagine any place where that control function is gone. Well, yeah, of course it's gonna bring in negatives potentially, right? Because not definitely, but. For sure something's gonna happen if you take away control function. But ultimately it's about making it run well, right? Making it run as well as it can. Uh, so yeah, I think there's even MIT research that shows that, uh, there's a measurement between how well you do data, how well you do AI, just is a percentage point, like, I don't know, 10% or whatever it was better in your business. So that's, that's a net positive. Which makes sense. You know, you're, instead of doing, you know, an experiment with an AI just like that, doing it well and learning how to master that thing for your business is just gonna give you a more positive output. It's just that most people associate the word governance or control with a negative, you know, they associate it with, for example, you know, compliance or, or regulations and laws and fines and, and liabilities to your point. Um, but I think that's a perception issue. Now, perception is reality, so you have to recognize that people see it sometimes like this, but show them, no, this is about, you know, the negative, the risks, uh, and avoiding them, right? As well as it is about, you know, making our business work better. And executives are starting to pick that up, right? There's, there's, there's taking the learnings themselves. Yeah.

Think about if you avoid doing stupid stuff, that's actually a pretty good competitive advantage then. So yeah, I like the idea. Do less dumb stuff and that's going to help your business. It feels like AI governance is a relatively new field, but data governance, of course, is much older. Can you talk me through what the similarities between data governance and AI governance are and where they differ?

Yeah, so I mean, we can speak about this topic alone for a very long time, mate. And what I want to say, Richie, is that data governance has reached some level of maturity curve. Uh, but I wouldn't say, you know, maybe if you want to put it in human terms, it's maybe left home for university or something like this, right? So it's, it still needs to get to the peak of its career, right? So that, that maturity curve will continue a little bit and there's gonna be evolution there in the data governance requirements still as well, because you'll have new data storage and warehousing and movement technology. So that will come with new requirements and you'll have new laws and regulation and policies that will come with new requirements. You require. So it will evolve a little bit, but you, you can see, clearly see it's further advanced on the maturity curve. Now when you look at AI governance, then I would say you're much more in a moving target area. And the moving target is on the one hand tied to the underlying technology. For example, if you would look at AI governance topics when GPT just came out, and by the way, there's AI governance topics pre-GPT as well, right? But I like to make GPT like an AC/BC moment because AI got redefined. Before, AI was like machine learning and you do some data science. You take data, train a model on it, and it makes predictions. And after GPT came out, in the world of the, the, in the eyes of the world, the, you know, everyone on earth, AI became the GPT. Right. So in that first year, what AI governance was, was around use cases. You know, organizations were trying, oh, you know, somebody's going to upload our IP in the LLM. So we got to put, you know, protections around the use cases. What are we allowed to use it for and what not? And which one are we allowed to use and which one not? So the use cases was sort of the first step of AI governance and that it doesn't end there, right? That's going to continue. But then the second year, we would, I think we're going back now to last year, agents became a thing, right? Because they took the LLM engine, if you will, and then realized, well, to make an engine run as a plane or as a car, we need to put something around it, like a steering wheel maybe. And so that's the harness of the agent. So then AI governance naturally evolves alongside it. And then AI governance became. Also, so use cases, but also about, for example, having an agent registry. So you know which agents, quote unquote, live in your environment. And then inevitably, you know, an organization will not have one agent, it will have multiple. So then you can sort of predict that the multi-agent system will come up from the olden research days, and people are already talking about swarms, right? So then you you'll have AI governance requirements that have to do with that whole system, uh, where, you know, the traceability of which agents are important, but also how they interact with one another and also how the whole system as a whole, uh, acts. So you have AI governance requirements. That's a moving target there. And across all of them, uh, and then this is where you start to go into the overlap of, of, of, of data governance across all of them. It's gonna need to be clarified which agent or which LLM or which agent system uses which data. And what you start to see there, um, is that, um, that Richie, that the topic of context is sort of bubbling up, uh, in the world as well. And context is essentially everything that you give to the agent so that it's informed to make its decision, right? The semantics, the context of the, the business policies, regulations, but also data, datasets. And so there you'll see that there's some overlap between data and AI governance because let's take a simple topic like quality will become important because everyone knows if you want an LLM or a, you know, a non-pre-trained model to perform better, then you got to give it better data, better quality data, better quality context makes the model do better things. That's proven mathematically, I believe. So that's clearly an overlap topic. Between these areas. And then, you know, when you mention governance, it'll take 10 seconds and somebody brings up lineage. So if you look at similarities and differences there, in a data governance world, a lineage requirement was very much about how does my data hop from its source into its ETL and then its target in the warehouse, for example. And it gets a lot more complicated, but it was more about how the data moved and, and which points it moved through and which controls you could put on it. Whereas if you look in that word lineage in an AI governance world, people even start talking about provenance. But ultimately you start talking about, well, how does one agent connect to another one and which data flows in between them? And make no mistake, right, Richie Cotton, the people will set up these agent systems with separation of concerns, right? Not all agents. Will be allowed to do all things and will be allowed to touch all data, right? Because you want to box up an agent in a controllable unit so that you can actually manage that overall system. So that lineage, if you will, of which agents and which data and how they all interconnect, that will be like an AI governance requirement. And it'll look a little bit like your data governance lineage, but it will also come with new things. So I would say, a lot of similarities, including on the organizational side, which is to say, um, what you saw 10 years ago in data governance organizationally, you saw people wondering who needs to do this and why and how do we need to do this and how do we measure that we're making progress on this topic, which have all sort of been sorted out, right? There's, there's a lot more clarity in the maturity curve on data governance. But if you look at AI governance discussions today, you'll see those same questions bubble up. So for AI governance, is that the lawyers, the privacy lawyers who have to look at this, or is it the head of technology or the CIO who's responsible? Or is it our compliance team, or is it the chief AI officer, the chief data? So you see that confusion happen, including why we need to do it, when we need to do it. How we need to do it and how will we measure success? And of course, security people pop up as well. So a lot of similarities and a lot of differences as well, including differences on, well, the maturity curve means the organization is figuring out how to do this new thing.

I have to say that was an incredibly thorough answer. I'm like, I shouldn't be surprised, but there's a lot of things to think about with governance. So Before we get overwhelmed, maybe let's figure out like, where do you get started then? So one of the last things you mentioned was that there's gonna be a lot of different leadership teams involved. You want, you can have like legal, you can have IT security, like the data team needs to be involved. There's a lot that needs to go on there. So talk me through what sort of organizational structure do you need to put in place in order to govern your AI effectively?

Well, I think most of the organization will likely already be in place. Let's unpack this with the simplest example, right, Richie? So let's do it the way that probably a lot of people will do it. Some CEO or board members like, we need to do something with AI. And then they find somebody in the company who does something with AI, not like, oh, first we need to think about X, Y. No, no, I need somebody to do this. So you start developing a prototype chatbot agent. Let's forget about that for a second, what it is exactly, but they're just doing something with AI, right? And now they didn't take into account any of the other parts of the organization. They just did it because that was their assignment. And now a problem occurs, right? Oh, you know, there was a prompt injection on the chatbot and now it did something bad. It's, you know, we gave it root access on the, with Claude bot or whatever. And it deleted all folder files or it dropped the table in the database because it will do those things, right? And then, you know, that's when out of the woodworks, based on the incident, you'll see security people come in and they'll be like, well, we need to protect ourselves against new sorts of attacks, so are we going to do that? Then obviously you'll see the privacy or the compliance lawyers come into the woodworks because they'll say, well, you know, you were building this chatbot or agent, which model did you use? Because now we need to like start assessing this model, whether, you know, we think it's okay for our company to use that model and not another one. And obviously you'll see, you know, some technology leaders swoop in and say, well, we're standardized on, you know, hyperscaler such and such or frontier lab such and such. So you should have known not to use that one, right? So instead, you know, come to the dark side and use the company-approved platform for which we're already paying credits and tokens and whatnot. And then inevitably, as that chatbot— so now you have some of those stakeholders that surfaced, and then inevitably at some point in time, the chatbot doesn't know— one of the agents doesn't know anything anymore. It's lacking context, so it needs to be fed with additional data. Somebody gives it a RAG architecture or some other setup, you know, says, oh, but we have that data in the source system, or in our fantastic data or delta or data warehouse lake or, you know, whatever consolidation place of data the organization has. And then you'll see the data boss come out to say, hey, well, if we're going to feed that into you, we need to make sure that this is done appropriately and put a quality control in place or whatever else. So now that person surfaces too. And then of course, at some point in time, then it starts touching multiple parts of the organization, you know, sales, marketing, or different lines of business. And people will still start to realize, well, if that agent operates like this, it actually generates an organizational cross-functional problem. Uh, how will we solve this? Well, we'll assign a steward, AI steward, agent steward, whatever name they give it. They can call it AI Ninja for all I care. And, uh, they'll form some sort of, you know, committee that maybe on a strategic level, on a quarterly basis reviews, well, what kind of issues did we experience cross-functionally with this? New AI widget and how will we decide to prioritize solving that together? Because you need that senior leadership. So you see, many of those roles and functions, I believe, are already in place and I would recommend you tap into them. You make them friends from the get-go rather than first have a problem and then have them be angry at you and then make friends with them. It's fine too, right? Both ways work. Um, but, um, on top of that, you'll inevitably have, you know, you're seeing that in the markets just like we saw before with other functions. You'll have some people opining that there needs to be now a chief AI boss, and maybe that rolls everything up, right? Maybe it's chief AI, chief data, chief data analytics, chief AI and data analytics officer, whatever it's called. But that function is starting to materialize or some, in some cases even consolidate, I would say. So many pieces are already in place. And a new leadership function will probably, you know, grow or evolve from, from an existing one. And maybe that's a temporary one, right? Because the way I see it, Richie, is that, uh, when you're dealing with a platform shift, organizations and business will start to operate differently, right? So job responsibilities will shift, business processes will shift. 3 years down the line, this is now business as usual, just like the cloud is or the internet or the mobile phone or the desktop is all business as usual now. And then maybe that role turns out to be a temporary transformative role rather than a permanent one. Or maybe it just rolls up to the CIO again, or the CTO, for example. So many in place and some new ones, probably transformative, but maybe permanent, will start to materialize.

Okay. I say when you, when you were naming your, bot for controlling things. I was like, you called it Stuart? What, what would you call it, Stuart? Or is the, the AI steward, uh, agent? Yeah, so I like that. But, um, it's interesting, that last point about, um, maybe it's only gonna be like a— this Chief AI Officer is only gonna be a temporary role, like having someone coordinate, uh, these activities. That's interesting because I think a lot of companies are just starting to think, oh well, do I need to hire a Chief AI Officer? But actually saying maybe it might be better to do this in a federated manner and have AI responsibility within different departments rather than like try and centralize it across your whole business?

I think it already sits everywhere, Richie, because if I look at the AI phenomenon, it captivates everyone's attention no matter your job or country or department or company or industry you work at. It's keeping all of us busy, even, you know, even people who are not working even yet. So, but Uh, in that sense, I think it's already everywhere in all functions. And then putting like a, a leadership role on top can help to corral efforts or, uh, can help drive focus on it so that, you know, all of these individual efforts maybe go in the same direction. But if you say, okay, I'm an organization and I don't have that and I'll hire one from outside to help change our business, well, you're not necessarily setting that person up for success either. Because maybe that person doesn't know anything about your organization. So to be effective, they will need to work with the organization that's in place because they'll need to, you know, extend it or transform it. And if they're an external candidate, okay, maybe they have the AI chops, but then there's still a little bit of the business chops in the company or the organization as well, of course.

I guess, uh, a problem with a lot of these sort of technical roles. I mean, it's been a problem with data science for a long time. It's like you need business skills, you need the technical skills as well, and the ability to communicate in both ways. And having all those different skills in one person is often a bit of a challenge.

Yeah, no, it's a team sport in my view, just like what you saw in data. People often refer to it as a team sport, and data and AI, we know, goes hand in hand. So AI in that sense is also a team sport. I mean, I gave the examples earlier with various roles involved. Even if you're— I mean, whether you're a large organization or a small organization, the same thing applies. And maybe in a smaller organization, more of the roles are bundled into one, but then you're gonna start hitting a point where this is a white raven type scenario, right? And then, you know, it's such a unique profile with that company of skills and 10 years of GPT experience, right? If that's even possible— it is, by the way— but that'll be a handful of people worldwide, and they might you might not be able to afford their multimillion-dollar salary either, right? So I think it'll be indeed a mix of people collaborating to make this a successful initiative. Absolutely.

Actually, do you want to expand on that thought? Like, if you're interested in a career in AI governance, what sort of skill set do you need? Like, what are the technical skills? What are the soft skills?

Well, anything governance is always going to come with a lot of people skills, right? So a lot of people, program management, project management, communication, even change management skills are definitely going to come in handy. Uh, technical skills. I think that's a developing topic. If I break down technical, I, I don't want to say, you know, just technical coding technical, but coding in its technical in its broad sense, because you'll need legal AI technical skills. For example, you need to be able to comb through, let's say, the EU AI Act a little bit and, and try to map how those external requirements can be executed upon pragmatically inside your organization, right? So that requires some translation, but it also requires for you to understand, you know, how AI functions a little bit. I mean, you don't need to know the exact workings of a neural network, but you need to know how that system operates and which, which risks it produces, right? So I think some legal, you need to be able to at least read it, right? And, and interpret it a little bit. I think you need also data skills for sure, right? Because. I mean, again, not the deepest, but inevitably the chatbot, the agent, it will need to eat up data in some way. That doesn't mean you need to be the, like the ETL developer necessarily, right? There's all sorts of, um, low-code, no-code self-service connectors to say, okay, my agent connects to my CRM or whatever it is, right? But you need to be able to piece those together. And maybe if you're the AI governance person, maybe then you don't do doing that yourself, but maybe you need to drive a more technical person to help you do that. So you need to understand at least the requirements of what's needed there. And I think then there's also business skills you need to learn. But if you are going into AI governance right now, it's a great time because you're looking at the start of a discipline. And if history repeats, which it doesn't, but it echoes, then you're looking at at least a 10-year window in which you can develop that career. Because it's a new discipline, maybe, you know, anyone who applies themselves a little bit in the short term quickly knows more than most others. So yeah, it's a great time to get started. And this podcast and the DataCamp learnings are a perfect place to start, you know, increasing those skills.

Absolutely. Oh, thanks for the pitch. That's wonderful. But yeah, I like that there's quite a wide variety of ways in. So you can go through the more technical angle, you can go through the more sort of program management or project management angle. Or even the legal angle there.

Now, on the legal angle, Richie, I would be cautious also because if it's too legal and too compliance, one of the pitfalls you have there is that it becomes a checkbox skill. Like you say, okay, but I've checked the model, we've assessed it, and it's fine, so my work is done. But if that activity is not connected to the actual agent system that runs and purrs and hums and does things, then yeah, you have the checkbox, but the problem still exists. So I would say if you're in that angle, make sure that you understand the, the actual operations of the system well enough and that you're plugged into that. Then that's— it's not just a checkbox exercise because that won't be enough.

Ah, yes, this is a common problem where you have a governance team is kind of the team that says no, and they can be a blocker on other teams, and you can end up with the sort of bad culture where different teams are fighting each other. Do you have a sense of how you can avoid that situation? Like, how do you stop governance just being a blocker on other teams?

Yeah, that perception is real, right? So people look at governance, they often say, oh, there's the police or the ones who say no, right? But on the other side of the fence, when you bring up the word governance, you also hear things, well, that's important, but somebody else has to do it. It's important, but for later. So this, this sort of a mixed bag, but to avoid that kind of situation where increasingly you're, you're becoming opposed to one another, I think you need to lean in and understand the other party. So let's say you're in the governance unit, or maybe that's, you're the governance unit, and now you found somebody in the organization who's like, Oh, I'm doing an AI project. My boss told me and he gave me a lot of budget. So I'm, you know, I'm running and I'm racing and I don't want the governance guy to come break me. Right. So you can already assume that they'll take a bit of a, you know, stance like don't get in my way because it's their little project. And essentially anyone who comes from the outside is a risk that they're going to make it muddy and more complicated and slow it down. Right. So. I don't think that's necessarily a threat from just a governance person, but from anyone who comes into that, hey, but it's my project, leave me alone. So I think you need to just, you know, openly go and be curious about what they're doing and why they're doing that. Because honestly, Richie, if you find out that this project that they're running is just some sandbox, maybe you don't need to be involved, right? Then you could just say, hey, I love this project, great work. Can I just look over your shoulder and learn from it from time to time? Because it'll inform me of how we can do governance right and make many of the more of these initiatives. I mean, you're doing the experiment, so let's learn from that and I'll do the learning. You don't need to do anything extra. Don't worry about it. So you're sort of looking for a need, you know, what, what need exists on the party that you need to collaborate with, because maybe that you discover that the project is an experiment or it's wider. And then you detect, well, they, they hit a point where they do need interaction with, you know, the lawyer or interaction with the security person or interaction with the data person. And then you're like, well, no problem. I've done this many times. Let me find you the right person right away. Let me give you here, look at this button, click this button, and then you get like access to the data. Somebody will approve it. So you're actually looking where there's, where they are having problems and seeing how you can help them. And that changes the whole game, right? You're not the person who's going to send them to a three month governance course in Siberia. You're the person that's going to help them with their needs and respect the constraints or the boundaries of what their project or program actually is. And you're helping them achieve those goals. But the perception is real, right, Richie? People look at, if we compare it with racing, people often look at, You know, I'm the one who's driving the wheel and hitting the gas, right? And governance is the one who's doing the brakes. But that's a naive perception, right? Because really governance is around the track that is there, the road, which signs are on the road, which curves are coming up on the road and how those curves are indicated, what the rules of the road are. You know, you can drive 50 or 100 or whatever, and even on a racetrack, You know, your car, yes, it has a brake because that brake allows you to, you know, slow down a little bit at times so you don't hit the wall or you can take the curve in a better way and win the race as opposed to, you know, crashing out in the first kilometer of riding. So governance is more about the rules for the whole system that make the whole system work at its most performant way for the long term. Not just for the next month. I hope that helps, you know, give a bit of a different perspective of governance.

Absolutely. And I suppose I should have guessed that the solution to having different teams not fight with each other is just more communication between those teams. So have the governance team talk to the people who are doing things with AI. And once you find out what they're doing, you can have a dialogue and that's going to resolve some of the miscommunication, some of the tensions there.

Now this is on a spectrum, right? So I'll try to make it a simple picture. You got lovers and you got the haters and that's the spectrum. So all of, there might be 10 or 100 initiatives, so you can't do them all. Governance units, the person or the team is typically a very scarce resource. And if you want a scarce resource to be successful, you got to focus it at the right thing. So essentially. Once you identify all the ones that are out there and you know how they look to, uh, lovers or haters, then you start out with the lovers, right? So you make the ones that are most inclined to collaborate, you make them champions first. You, you know, practice your approach, your method with them because you have most chances of success. So you work with them on their use case. And then, you know, a little bit later they're champions. And then you use that momentum of those champions to go to the next ones, and then you sort out their use case, use the first ones as a testimonial or reference, like, this is how we did it and that's why we did it and that's what it, you know, turned out to make better. And then you, you know, convince the next ones, and then you keep going. At some point in time, sequentially, you're gonna hit the haters too, but you're gonna hit it with an army of lovers who are like winning the hearts and minds even of the most hardcore resistors.

Uh, okay, I like the idea of kind of winning your colleagues over a bit at a time. Uh, one thing I'm curious though, uh, when I speak to security people, part of their job function is a mindset of suspicion and paranoia because you have to have at some point a certain cynicism about what your colleagues are doing, making sure they're not breaking rules. Is that mindset also important in governance?

Well, it is, but I think it's still slightly different. I mean, if I look, try to look inside the mind of a security person, they have to be very cautious and risk-avoiding all the time because even on a personal level, their job is always on the line because the security profession is a cat and mice game where the cat is the security people and the mice is the external attackers that are trying to get in. And those mice are always looking for the easiest way in, right? So the cats, the security people, they're always trying to make sure that all the holes are covered and there's no easy way in. But ultimately, there's always a way in, right? There's always like, depending on how many resources, and state actors have significant resources, so in a way they could almost get in everywhere. I mean, you see it, for example, through breaches. You'll see at some point in time, all companies in the world get breaches, right? So when a breach inevitably happens or an incident inevitably happens, the security person is, well, you didn't do your job well. So they know it's a, they're on a timing game in that sense. And from that sense, they're, I think, also naturally cautious, right? Like, We gotta avoid that situation. Governance is a very different situation, I think. I don't think with governance you can only think about, oh, what could go wrong? No, with governance you also got to think very clearly about what needs to go right, uh, otherwise you're in the wrong side of the fence, I think.

Absolutely. I mean, that's, uh, a big thing is that if you govern too strictly, then you're going to miss out on opportunities, uh, that you that you would have said no to. So yeah, it's very much a balanced sort of field in that sense.

Exactly.

Okay, so let's talk about some practical things you can implement in order to improve the state of governance at your organization. So before you mentioned the idea of having an agent registry, and it just seemed really sensible to just track what you're doing with AI. Is that the place to start, or are there other things you want to do first to get your sort of governance infrastructure in place?

Yeah, an agent registry, you could almost compare it to, uh, in previous times, you know, we have registries for applications, systems, devices as well. So naturally that pattern extends into this new phenomenon because you need to control a system. You need first to have visibility on a system, right? You need to have transparency. You need to know what's out there. So, you know, you quote unquote gather all which models, which agents, which use cases, because that gives you visibility. And then you can sort of start to classify them. For example, you can come up with criteria of priority because you can't handle them all at the same time, right? So you, high risk, low risk, whatever criteria you choose in your organization, you pick them and you say, okay, well, let's focus on the high risk ones first. And see what we need to do about those, and then you could move on to the next one. So I think the first thing you can do, that's an easy one, is just getting that transparency, you know, what exists out there. And by the way, for a number of the regulatory drivers, that's literally one of their requirements. Absolutely.

So just understanding what you're doing with AI seems a very good first point. You mentioned the idea of categorizing the AI use cases by risk. So this is an important component of the EU AI Act. Do you want to talk me through what are some high-risk use cases just so people have a sense of it?

Yes. So the way it goes is that in an organization, somebody comes up with a use case like, I want to do this with AI. For example, I want to use it for sentiment analysis. I want to use it to make our coding better, whatever it may be. When a use case comes in, right? It needs to be assessed according to a number of things. For example, I want to use AI to do sentiment analysis or to automate our customer support more or do coding. That's a use case. And then the organization wants to look at how can we assess that use case from a business point of view? Does it make sense for us to do this? Right? Are we gonna get value out of that? Is this gonna, you know, match with our brand or ethics or policies and whatnot? Right? So, There's a number of business assessments to take into account as well as there are other kinds of assessments, like for example, related to data privacy. But then when you look specifically to your question, Richie, about risk, high risk versus low risk, let's say you have a, a chatbot on the website and it helps you navigate the website. Uh, well, that's a low risk, you know, because what could happen is that they end up on the wrong page and have to search again. So there's not a lot of high, low, high risk in there versus when you have an automated system, right? So a chatbot is just human input and human reading output. Whereas an agent has autonomy, it can do things, do things like, you know, running root commands on your computer, or in the case of physical AI, actually perform actions that affect the real world around us, like a self-driving car or people think about robots, they always think about the human on robots, but drones flying around are, are robots as well, right? So when, uh, you touch systems like that, that go into the physical world, that go into personal data, that go into medical situations, uh, maybe that go into manufacturing, you know, affect the manufacturing process of food or drugs or whatnot. That's when you start to go in a much more high-risk situation, uh, where it's about life or death or other significant risks to human beings or society. High-risk ones, you definitely need to control them in your organization, but the real high-risk ones will need to be controlled at the society level as well. I mean, nobody is going to want a self-driving car or a self-flying drone on the streets or in the air without knowing about the safety parameters of that device. Somebody has to go first, obviously, but that you're going to do that in a, in a very controlled environment, right? So a high-risk system, you're going to put it in a more controlled environment before you sort of relax the parameters and, and open it to a broader population, for example.

Okay. Yeah, that certainly seems very sensible. If you've got something where it's going to have life or death consequences, you want to treat that a little bit more seriously than, I guess, like your document processing agent, whatever, that's just kind of gonna save you a little bit of admin time. All right, so in general, how do you assess the state of governance across your organization? So I suppose you want to like audit your own capabilities in order to figure out how do I do things better? Is there a good way of measuring like how well you're doing?

Yes, so I think we'll see just like we again, the echoes of the history repeating, echoing. Just like you saw with data, you start to see maturity frameworks pop up. So what I expect is that over the next 3 years, you'll have probably 10 or 20 maturity frameworks to pick from, right? So it doesn't matter which one you pick, but pick one. I believe like there's one from a security angle with OWASP that is clearly more geared towards security, but you know, you may as well start from that one. And like I said, other ones will come, you know, the schools, the universities will come up with them, the big labs will come up with the maturity frameworks, and even the consulting firms will come up with their own, right? So pick one and then just measure yourself consistently against that framework. That would be my advice.

Okay. Yeah, it seems like I guess there are quite a few sort of general AI sort of maturity frameworks around. Maybe there are going to be some more governance-focused ones coming soon.

And the maturity framework, Richie, is not a, is not a, it's a means, it's not an end, right? So you just, you, the reason why you can pick one, you can throw a stone and pick one. And the most important thing is that you just consistently measure yourself against that framework, let's say over a period of 1 to 3 years, because then you have some sort of anchor to measure like my efforts that I'm putting in. Am I actually inching forward on that maturity framework? Or am I sort of stuck in the same place? So you just consistently apply one.

Absolutely.

Yeah.

A lot of these frameworks, they're about just giving you some inspiration for what do I need to do next? And it was like, where are my weak spots? Where can I go about fixing them?

What you see in the space, because remember the underlying technology is rapid evolution. We talked about that earlier, right? The LLM, the engine, the agent, the framework, the car or the plane, and then now the agent system, the swarms. And tomorrow or next year, we'll see more. So, um, because it's a young and evolving discipline, uh, you are going to have to, you know, accept evolution in the maturity measurements as well.

And you mentioned rapid evolution. So, uh, we've gone from like a year ago talking about agents as being coming soon to— you just mentioned like agent swarms being the next big thing. It is moving fast, and governance is traditionally a very cautious, let's do things carefully kind of discipline, rightfully so, but is there a way to do AI governance fast? Like, how do you keep evolving your capabilities at the speed that the technology is moving and with the speed of adoption?

Yes, so let me give you there the example, Richie, that I used earlier today. So at one point in time, I was in the Yellow Mountain in China, And they have stairs carved out of that, uh, mountain. So, but it goes up high, 1 or 2 kilometers. So it's quite a walk. And what you see is that you have people who are like going for it. Wow, we're going to go up the stairs fast. And, uh, you know, you meet them 10 minutes later, 500 meters later, whatever it may be, you meet them heaving strongly and, you know, needing a break and taking some rest. Whereas the people that, you know, take this, you called it carefully, but I would say it's thoughtfully, just step, step, step, step, step. They pass all of those fast ones by and they actually typically reach the top faster than the faster ones. So when you say, how can you do governance fast? It's about being pragmatic and it's about being consistent., and it's about, you know, if you see that an approach is not necessarily working or, you know, creating some fuss or not solving a problem, then iterate fast enough, right? But mostly have a consistent discipline, uh, and that determines your steady pace that will put you on top of the mountain much faster than a quick experiment that, um, you know, one of the young interns in the company hacked together over the weekend. Yes, that was done fast., but then you push a button and you put it live for, you know, 10,000 users and everything breaks. Uh, that's not fast, right?

That's just silly. Okay. Yeah. I feel like, uh, you've, I love the analogy of, uh, going up the steps. It reminds me of, uh, the story of the tortoise and the hare. It's like the classic, uh, fable there.

Yeah.

So, uh, I like, uh, just sort of consistently doing things better. Um, I suppose the risk then is that if, uh, you've got Engineers being more productive because they're generating stuff with AI and a lot of teams are trying to adopt things quickly because there's a bit of a rush on. You can end up with a governance gap in your organization because people are doing stuff and the governance team can't keep up. So do you have any sense of how you avoid having that gap there?

I don't think you can because again, the scarcity of the resource, governance resources versus everyone else. It's miles apart. If you have 5 governance people, you might have 5,000 engineers, right? So they, you know, you can't keep up with everyone all the time at the same time. There's always going to be somebody who's pulling out a little bit ahead on certain things, but you try to, you know, manage the overall value and the overall risk of the whole system, not of an individual unit. Because you need to make sure that your top priorities from a governance point of view are catered for. And that, for example, if you, if you talk about engineers coding, that could be as simple as agreeing which coding agent you're gonna test out. And I say test out because you want to pilot a little bit, and you would, you know, want to put, you know, boundaries of your pilots, like who are we going to test it with, what are we going to test it on, and how are we going to make sure. And then you pick, and that's— then that is the governance activity. And then like a quarter later you evaluate how did it go and let's roll it out to the rest, as opposed to saying, well, it's a free-for-all, everybody pick your favorite coding agent and let it rip and we'll see which one works best. Because I'm pretty confident that your GitHub or whatever repository is going to be full of steaming pile of something, um, very quickly.

Yeah, uh, I love the idea of having some sort of like broad, uh, standards, uh, so the idea of like, okay, we're gonna at least decide on which tools you're using and what you can do with them before, because that seems something that's fairly scalable. But also what you said is it feels kind of terrifying, the idea that you have got now maybe tens of thousands of people using AI and a very small governance team. So Do organizations just need to radically rechange their expectations of what they need to spend on governance? Like, do you need to dramatically increase the size of governance teams or have organization-wide governance training? How do you prevent these disasters then if there's this new imbalance in amount of AI usage versus existing governance capabilities?

Well, you know, Richie Cotton, if you talk about investing more in governance, I can always recommend it, but I got seen as having a huge bias, right? But I can tell you, uh, one thing, uh, if you invest zero in governance, you're gonna have— you're not gonna have a great time. So there needs to be some investment, uh, but like I said, from what I've seen, I've always seen it as a scarce resource, and I don't think that needs to change. It needs to be resourced sufficiently to be able to be effective, of course, but I don't think you make it effective by saying, oh, we have 100 engineers, so we need 100 governance people. That's not the way it works, because then, you know, people are just tripping over each other. Again, governance is a control function, and a control function is about setting the rules of the game so that the game can be played in the right way, in the most optimal, performant way. And you don't need a lot of control resources. It's always going to be a scarce resource. You just have to make sure that the scarce resource is focused on the right thing and in the right direction. Because if they're focused on the wrong thing, or if they're going in the wrong direction, then yes, for sure, it will be seen as, you know, an obstacle that is not contributing anything. It needs to have the right focus. And I think that's the challenge, um, right now is that because there's a newness to the discipline and there's a lot of evolution still, knowing what the right direction is, is a bit more challenging. So you could make an argument now that an overinvestment in some governance resources to make sure that you're covering some of the bases is a wise choice because clearly the other side, which again is more, much bigger anyways, is experimenting all over with the AI. So because of that increased activity, you know, we're in a race. People are FOMOing, right? Oh, I got to do something with AI. Otherwise my colleague or my competition will beat me. So they're, they're sort of racing and sometimes they don't even know what they're racing or who they're racing, but they need to go fast. Right. So that means there's a lot of things being tried out. And if you have more things being tried out because you, you, you give that freedom and maybe yes, you do need some more control resources. I wouldn't advocate for changing that scarcity ratio all too much.

Okay, yeah, I can imagine a company that's almost entirely staffed by governance people. I mean, it's a very niche thing, but in general, it's not going to be useful. I like the idea of setting the rules of the game so that everyone else can play fairly. All right.

Do you want to play soccer or do you want to play basketball? What game are we playing, right?

Yeah, yeah, at least decide one or the other because if you got half the team playing soccer and half the team playing basketball, it's going to be chaos. Do you have any final advice on how to do AI governance better?

First of all, do it. And, uh, doing it in my view is very simple. Uh, don't complicate it, right? So make it practical, make it about not talking, but about rolling up the sleeves. Uh, because a lot of governance challenge comes from Here's a 100-page slide deck about all of our rules and our policies and the roles and the responsibilities and the thises and the thats and the don'ts. And then, you know, you start presenting that slide deck and within 30 seconds, you see the eyes glaze over. That's not what you want, right? So if you want to do it better, keep it simple. It's not rocket science. Roll up your sleeves and help get the job done and learn. From that experience how you can do it faster or quicker the next time or with the next champions you gotta make.

Absolutely. Yeah, extensive slide decks that send people to sleep are no benefit for anyone, whether you're the speaker or the audience. So before, actually, before we finish, I always want people to learn from. So tell me, whose work are you most interested in at the moment?

As in books or anything?

Books, people, companies, anything that you've learned about that you want to recommend.

Okay. Well, my main recommendation remains consistent. I would go for MIT's scissors data monetization work. And the book is called Data is Everyone's Business. And, you know, includes portions on AI as well, because AI is a data product in the end. Because it's really simple. There's no jargon. And it's really about connecting it to value. So it's accessible for non-technical audiences, all audiences, and I would consider it to be a strategic framework that's easy to digest. You can read it over the weekend and you can get started on Monday. So that would be my recommendation, and that's Data Is Everyone's Business by Barb Wixom from MIT.

Wonderful. Yeah, I mean, I love the idea that data is for everyone, and of course monetization is such an important thing, like trying to figure out where the value comes from is Sometimes challenging, but incredibly important. All right. Thank you so much, Stan.

Great to speak with you. Richie Cotton, thanks for having me over. Have a nice day.