#338 The New Paradigm for Enterprise AI Governance with Blake Brannon, CPSO at OneTrust

Generative AI is transforming industries at an unprecedented pace. But as AI changes how you or your team work, one thing is clear. Your skills also need to evolve. At datacamp, we offer everything you or your team need to adapt and thrive with AI. Whether it's business users looking to get the Most out of ChatGPT and Copilot, or developers and data scientists looking to fine tune models, you can learn the entire AI skills spectrum on Datacamp. Power your AI transformation today. Start learning@datacamp.com.

So the speed mismatch of wanting to bring new AI online, agents kind of reasoning and almost in real time deciding something new to do that wasn't predetermined in the past. How are you going to govern that stuff? You have to reimagine governance. You have to be AI ready with your governing processes and you've got to shift some of the fundamentals of how you've been approaching doing that for years. AI agents are going to fundamentally change the way security operations and security operations centers work. It kind of puts us in this mindset. We're seeing it in a lot of different groups in a company, but how AI agents are going to be doing the lion's share of the work that you have to do as part of compliance or governance teams. You can almost imagine there's an AI agent that is governing the AI use that is happening. And when an AI use in the business is reasoning and making a decision, they're not going to ping a human to say, is this okay to do? They're going to ping an AI agent.

Welcome to Data Framed. This is Richie. In an enterprise setting, AI adoption and AI governance often go hand in hand. If you adopt AI without governance, your business won't last long. I'm convinced that a lack of AI governance is often the biggest blocker to corporate AI success. So today we're going to learn how to do it better. The devil is naturally in the details, so I want to know all the practical steps, but running an AI governance initiative. Our guest is Blake Brannan, the Chief Products and strategy officer at AI governance platform OneTrust. He spent two decades as a technology executive with a focus on trustworthy AI. I'm keen to hear his thoughts. Hi, Blake, welcome to the show.

Hey, Richie, thanks for having me.

Yeah, great to have you here. Now, just to motivate things, I like to start with the disaster story. So can you tell me, like, what's the worst AI governance disaster you've seen in recent memory?

Well, there's, there's two that come to mind that I'll give you the first one was a organization that spent a lot of money on acquiring customers and getting customer data. You know, typically called first party data. Right. Because that's the gold mine of how you're going to innovate as organizations. But they made a subtle mistake in doing that. They got the data, they then fed it into their entire AI pipeline. So they trained on that data to do facial recognition and things, and then fast forward, they ended up the ftc. So this happened here in the US The FTC ordered them to not only delete all that customer data, but also delete the AI algorithm, an engine that they spent all that time and money training on it. So it was really impactful from like, like that's material. In fact, this is an interesting word the audience might, might find interesting. The regulatory ruling of that is what it's called. It's called a algorithmic disengorgement. Which, like sounds really scary, maybe by design, but that, that's what it is when you have to delete that the algorithm that you use to train your data on. And so all that was rooted in something that I spent a lot of my time and the history of the organization I'm with doing, which is privacy. Right. That is personal data. You used it without the proper consent and notice to those individuals. And that was kind of the consequence of it is you had to delete not just the data that you gathered and spell that time and money doing, but also the AI algorithm that you spell that time and money training as well. So make sure you are getting consent and, you know, not misusing data that you're gathering and collecting.

Algorithmic disengagement. That's a new phrase for me. But yeah, one to remember, I think. But yeah, that seems like pretty much the worst possible outcome there because I guess at this point they've annoyed their customers. Yeah, they've annoyed the government.

It sounds like some beheading thing or something weird. Right? But, but yeah, it's. I mean, kind of by design and I think it was actually rooted. The term was rooted post Cambridge Analytica. Right. And kind of the order that ended up having. Which was like you, you used all this information, you trained a brain, the brain did all this stuff. You got to destroy the brain. Right. Kind of a thing. So, okay, yeah.

And at that point, just everything, all the work you've done is wasted and you probably engendered a lot of bad. Well, with a lot of people. Okay, so that's one to avoid. You said you had two stories. What was what was the other one?

So that was one. The other one is more like the. The actioning and safeguarding. And this got a lot of publicity even a couple months ago. But it was a company, a startup that was using AI to do a bunch of coding. So they had these AI agents that were doing the development and they were vibe coding. And the AI agent, you know, from the derived reasoning and instructions, effectively deleted the production database. That was not good, let's just say it that way. So emphasizing the importance of why you need things like guardrails and controls, especially as we move to this era of AI agents in the world where there's certain things you obviously want to automate. There's a lot of manual work we all do every single day, but we need the right boundaries of like, okay, wait, when you cross that boundary, like you, you need that double check, you need that human in the loop intervention to make sure something like that doesn't just happen and cause something that could be pretty catastrophic. I think this individual company was a startup and, you know, really just emerging and building, but if you could imagine a large enterprise or something like that happening to how impactful something like that could be.

Oh, yeah, that. I mean, that could be completely disastrous. So this is a database in production then?

Database in production, yeah. It just decided, oh, yeah, the right thing to do is just delete the whole database and then I'll rebuild one.

Yeah, I mean, it's very easy to kind of laugh at that kind of treasure. You think, well, there should be some kind of guardrails in place. But like, yeah, I mean, once the AI goes a bit rogue, it's like, yeah, it shouldn't happen, but it could.

Okay, laugh at it until it happens to you.

And then, yeah, yeah, it's like, it's not as funny when it happens to you.

Exactly.

All right, so I feel like AI adoption has just skyrocketed everywhere over the last few years. So at least in theory, AI governance also should have skyrocketed. Do you think the governance side of things is keeping up with all these kind of new use cases for AI?

No way close. Right. And maybe to set the aperture of the scale of the problem, I think the world's kind of walking into is IDC predicts that there'll be 1.3 billion agents that are running around in the world in the kind of commercial enterprise sectors by 2028. So not too far away. Like, that's, that's pretty close. But put that number. Because it's hard to understand these numbers and Absolutes. There's only about 400 million knowledge workers that exist today or around the world. So you're talking about something like a 325% increase in the workforce that we're going to have in just a few years. And that, I mean, there's all kinds of stuff that's going to kind of break down, right? That's going to create a lot of challenges and struggles for how are we going to govern all this? It's one of the things that is like, so top of mind to every CIO in every organization today. And if you put that in some parallels that we've probably all recently experienced, like when Covid happened, the. We probably all remember a lot of things at our work where like the VPN was saturated and we couldn't all get connected. We were trying to join like webinar meetings and things. And there was capacity issues. That was from this massive shift of remote working and online. But that only was about a 50% increase in Internet traffic. Right? So only 50%. The infrastructural boundaries still were impacted by that 50% spike. We're talking about a 300% spike in, you know, work that is happening. So just imagine all the things that kind of break down. I mean, it's not exactly the same, but if you imagine bringing on 300% more employees into a company, like, think of all the infrastructure, the training, the processes, the. The systems to detect all these things. They're going to struggle, right? They're going to. They're going to kind of have some level of parts of it that fall apart and have to be rebuilt, reimagined, right? Not just extended to this new scale. And that's kind of the state of AI today, right, where everyone's looking at this future that we're going to be in and looking at all these things that make up traditionally how you govern IT and technology systems. And what are we going to have to change as organizations to be able to fit into that new world and new reality that we're living in?

I mean, that's a really huge shift then, like 1.3 billion agents. Like, it sounds like a big number anyway. So, yeah, acknowledging big changes, where do you think the gaps are going to be then? So once, once we go to like a. A point where you've got more agents and people, how does that change your approach to governance?

Well, the biggest thing to break it down to something super fundamental is the speed mismatch for how companies have traditionally done governance. It kind of fundamentally falls apart and What I mean by that is a lot of governing processes for all technology today, whether it's compliance, governance, security, governance, ethics, governance, all these different things, they're kind of rooted in a lot of human committees and processes where you evaluate, looking at things you're doing and you go through some kind of a framework and methodology. And, and those are like semi automated. I wouldn't say they're completely manual, but they're semi automated. But they're designed for companies to go through like what we've historically done, which is, oh, we do three or four new projects a year. Right. New product launches or new things that are kind of need to be assessed, evaluated. Right. AI use is, is all about data use under the hood. Right. It's democratizing that. But now you're moving into an era where there's, there's more agents than people and within those agents there's, there's orders of magnitude of data uses that are happening and they're almost like instantaneously happening. So your governing process of taking something where you're going to say you built an engine in a factory that could review four or five projects a year to kind of go through these governing processes and frameworks to have safety and trust with ensuring that, okay, what is actually happening we feel confident in it just falls apart. So the speed mismatch of wanting to bring new AI online, agents kind of reasoning and almost in real time deciding something new to do that that wasn't predetermined in the past. How are you going to govern that stuff? You have to reimagine governance like you, you have to reimagine. This is something we talk a lot about. But you have to be AI ready with your governing processes. And you've got to shift some of the fundamentals of how you've been approaching doing that for years.

That's interesting. So with speed, I imagine governance is something you don't necessarily want to do quickly. It feels like it ought to be something that's done slowly and carefully. So how do you reconcile that difference then between like, you have to go fast, but you also have to go carefully?

Well, I think, you know, it does put a lot of governing teams in this crux of how do we not slow down the innovation that the business is trying to do? And it, it's hard because you don't want to be someone that just says no. You don't want to be holding the business back, keeping them from being competitive at an advantage. But you also then are accountable for being compliant, making sure you don't have mistakes that put you on the front page of the, the newspaper. Right. Or have a algorithmic disengorgement order, whatever. It is hard. But what, what is starting to emerge is like new technology and new approach to how we solve this. Because it's, it's not any different than any new problem that the world has to solve where we just, we put our smart heads together and we say, okay, how can we build new technology to figure this out and solve this problem? And that's kind of what's happening. And traditional governance required or relied a lot on manual kind of human entry. Right. People would meet and review. Maybe you've gone through this yourself. You would join a call with team members to understand. Well, explain this project to me. What are we doing with this? Where did this come from? Let me understand the context. Can you tell me about how this is designed? Where's that data go? Where does it flow? You know, all this information and that is starting to shift to things that are like continuous observability technology. So looking at the systems and the, in this case, AI agents as they are a being built, as they're connected to new data sources, as they're reasoning and taking action, how do you observe that continuously and start to change the way you govern to kind of fit into that more automated approach to it? So not trying to take everything upfront and deterministically go through some flexibility, flow or process, but to say we're going to deploy like default guardrails, we're going to observe what's happening, we're going to pull the anomalies out, we're going to block things obviously that deviate from the guardrails, we're going to pull things out and then we're going to manage the exceptions. Right. So that shift is, is kind of what's happening right now. And I'll give an analogy to the type of shift that kind of ties me back to some of my background in history, which is what the world saw happen when we went from corporate networks where traditionally security was built around the primitive. And the, the main assumption that you could solve security by protecting the perimeter of a corporate network. Right. So we built up our firewalls and you know, everybody kind of all the corporate data was inside the corporate network. And then there was this technology disruption similar to AI. But back then it was called mobile and cloud. And in mobile and cloud, your data wasn't inside the corporate network anymore. It was actually kind of out all over the cloud in a, in a bunch of apps that you didn't directly control and you definitely didn't control the network around it. So you had to shift your approach to security. You no longer could approach solving security by protecting the firewall and the perimeter. You had to solve protecting the identity and the person. Right. And this is where you saw this explosion of a new approach to solving security for the mobile cloud era. And we're kind of seeing a similar level of a fundamental shift to how are we going to solve governance in this new AI and agent era. Right. It's that order of magnitude of a shift that we're talking about doing here.

Okay. So I love the idea of you trying to figure out like which bits are automatable and putting in I guess guardrails and rules for like a high level. So you're not having to worry about what does this specific agent do is like does it comply with the general rules? And that's it. That feels a bit more scalable compared to doing this on a one off basis. If you're doing it for many, many agents, that seems a bit of a mess.

Absolutely. I think pattern decomposition of what are the agents actually doing and if you broke those into the primitives of the patterns of things and say let's apply the decisions we've made already into that same pattern, you, you kind of auto have reviewed, right? You've auto approved, you've auto made that exception. Instead of trying to evaluate every AI agent as if it's like a snowflake, like it's a whole new thing. So let's evaluate and assess it. You just, you can't do it, right? You can't evaluate hundreds of thousands of agents as unique use cases. You've got to break them down into the fundamental primitives and kind of apply governing principles to those to scale it.

Okay, that makes a lot of sense. So I'd love to figure out a bit about implementation and basically I guess to be able to just work out like who needs to care about this stuff. So who is generally in charge of AI governance within an organization?

It's pretty fragmented right now. You know, I think you see a pretty broad set of people in one lens. You have teams that are the AI technology teams. Right. And they're, they have general guidance of governing principles at the corporate level, but they're the ones that are responsible because they're actually implementing it into the product. I see this a lot in technology, like heavy technology centric companies that build a lot of pure tech and the AI is really part of the product in a very heavy way. So they kind of put the governing on those teams because it's closest to, you know, the people doing it. Then you see these other set of use cases of AI that maybe are a mix of back office, some things that are involved, customer data, personal data, things like that. And that's where you start to see traditional compliance or governing teams. And a lot of organizations form AI governing councils or committees that are made up across people from privacy, security, ethics, the data or AI office. And sometimes they sit, you know, they kind of sit under whoever is the appointed person that can be, you know, influential. Some organizations have a lot of personal data, so the person in privacy typically has a huge influence over that because AI is using a lot of personal data. Therefore privacy is a heavy play in the governing process. In other organizations there's not as much personal data use. So oftentimes those organizations think of it a little more like a security oriented lens. So you have someone in the security group that's sort of the head person sharing that. And then some companies are super data focused and they have a very mature chief data office or chief AI office type role. And you typically see that. But in all of those, I, I do commonly see a committee or a council that is formed to allow all those different stakeholders that come from different parts of it to be represented. But then you have one of those that's kind of taking the operational lead at driving the actual program.

Okay. Yeah. So it seems like there's more than sort of one way to, to do this, like depending on the existing setup you have within your organization. So I guess the details of the committee are going to get tricky. Right? Because if you do have a talking shop and everyone wants different things that can get complicated. Do you want to talk me through like how a good governance committee works?

You know, I think the world's still trying to figure out good governance committee and exactly what, what flows with that? I think there needs to be a strong alignment to like probably the business, the business goal of why they care about governing the AI. Right. And what are the use cases and scenarios that ultimately need to be prioritized? And does the team that is been appointed like that AI governing team? Do they have the authority and the charter within the organization to operationally like make the changes and enforce the changes with it? So for something to be successful, at the end of the day, the team has to be effective with the organization leaning into how they are using AI, having the governing controls around it and being embedded into the business processes to effectively ensure that kind of on the Onset as early as possible. Just like software development, you had this shift left, you know, philosophy. It's kind of the same thing. As early in that process as AI is being crafted and created, that governing program is being put in and ultimately that is what's key to kind of help it be effective. So it's aligned to the priorities of the business and it's embedded into the process where the people using and implementing AI, it's effective inside of those processes.

Actually. Do you want to tell people more about like shifting left? I mean, I guess the vague idea is like you find problems before they become big problems, right? Like you don't want to wait until stuff's gone live.

If you can catch, you know, a potential AI issue before, you know, as the concept or the prototype is there, it's a lot less costly than going back to my original example of like, you've actually rolled it out and now you've got to delete all the data, you've lost trust with your customers, you've got to delete your AI algorithm. So the earlier you can do it in the process, the better for kind of everyone. But ultimately the closer you can get to like, oh, there's a new data project that's being spun up or an AI project rather in this case. And I instantly, the governing council and team sort of is aware of that. Right? That's a good sign of a healthy governance organization where they're not blocking it. They're not blocking people from ideating and playing around with concepts, but they're aware of it and they've put some preliminary guardrails in place right on that to ensure that the people using AI are aware of the, a potential harms that could come because most of the time people just don't realize that could, that could happen, right? Or this could be an issue, or we've got to worry about that. And then as it progresses and it goes from a concept, an idea to actually a real prototype and we really want to take it to production. They're, they're on the journey with them. So you're not waiting to get like, hey, now we're ready to go live. Let's talk to the governing team to make sure all this is okay. And we're trying to un engineer and, you know, delay the project. Everyone, you know, feels like it's a failure because we have to stop it. No one wants to, like, it's just kind of a mess. But if you're aware of it along the way and you're anticipating, you're Inserting yourself in the middle of that, it just goes, you know, it goes live with confidence. Right. At the end of the day, you want to be able to say we are trusting the AI that is live and we trust that we've gone through all the right things as a governing team and process to be able to ensure that we're mitigating as many of the risk as possible. And we're confident with everything that we've done to ensure that we're not misusing customer data, going to break trust or end up with something that's harmful to the business.

Okay. Yeah. So, I mean, this seems like a really important thing. I mean, you mentioned, like, it's very common for a government's team to be the people blocking stuff and that's a terrible outcome. Like, you don't want to be like the department that says no. So. Yeah. Do you have any more advice on like, how you can get good, I guess, cross team interactions between like a governance team and between like the people who sort of rely on governance to sort of approve or deny projects?

Well, one, I think a couple, couple tips there. One, I think the language you use matters. And we have one of our customers that they call their governing program a data enablement plan. Right. It's all, it's all centered around like, how do we activate data, how do we activate AI use? We're here to help you do that responsibly and we're here to be a partner with you doing that. Right. Not, we're here to govern and to ensure compliance. Right. So I think the language matters just to set the right tone from the onset. The second thing that I think is important, especially for people in governing roles, is you kind of need to be a practitioner to be able to have credibility with the business stakeholders. And what I mean by that is, like, you need to practice and use AI even kind of on the side, but at a level that makes you come across as not being like living under a rock. Like, you've got to be someone that understands and can kind of speak the language, walk the swagger, you know, with, with coming across as like, hey, this person kind of, you know, you could almost come across as like, I, I vibe code. And, and maybe you don't just do that, but you're familiar enough with the art of what's possible, you're familiar enough with how it works, maybe some of the lingo and language as well as the value and benefit to the business. Because if you can align with the, the business stakeholders on principle are really Deeply understanding and being respected of like, this is the future of how we've got to do this right in this part of the business and you really come across credible. They're going to be much more willing to understand your point of view on governance and ensuring safety and trust. Right. And they're going to lean into you on that front. The next thing I would say is there are a lot of frameworks for doing governance. Like one of the most popular ones is the NIST AI Risk Management Framework. So leveraging something that is, you know, a little more of an industry standard and others in the industry are using to kind of guide your governing framework helps give you credibility of what you're asking them to do is something that, you know, other organizations are doing as well. It is a best practice in the industry and oftentimes people are very familiar with NIST from the cybersecurity practices. Most people use the cybersecurity framework from NIST as part of their internal program. So putting the AI framework in that same kind of umbrella also comes with a lot of credibility. And I think people's openness to say, okay, this makes sense, we're very familiar with that. We get why we would have to do this is another key thing. And then lastly, you'd just be a really good leader and a good cross functional, you know, speaker and communicator. Because at the end of the day you're, you're trying to get teams that are innovating with technology to ensure that they are putting some of the governing, you know, policies and principles and things in place and not just running, you know, wild. Like the worst thing you can do is, is swing the pendulum so far where people are like, it's too hard to try to follow your process or to do something. I'm going to ignore it, I'm going to go rogue and I'm going to do my own stuff. And that's like a lose, lose for everyone process.

Sounds too difficult. I'll just do my own thing. Yeah, that's a definite disaster. Okay. I really like your idea of just rebranding governance as enablement. It just makes it feel a bit nicer. It's like a very simple change. Hopefully helps the different teams get on better together. You mentioned the NIST AI Risk framework. Now I was interested about like where do you focus your efforts and I guess a risk framework is going to help you with this. Like, can you talk me through like what the risk framework involves, like what counts as a high risk AI use case?

Well, there's different classifications of the types of systems and the uses. But the way to think of that is actually look at what are the. Actually the first lens. What I'm worried about the AI system in the thing. Think about where you're putting AI. Think about how big of a spotlight does that have? If you were to, as a company, forget about AI, if you were to go do something that's like, this is a change in the branding and the terminology that is the front door to the company. Think about all the eyes, all the scrutiny, all the things. Like anything you do that has a huge magnifying glass on it is going to be like, it's going to have a huge magnifying glass on it's going to be scrutinized with whatever you're doing, and you're going to amplify people. And then most of the time, AI projects, people further amplify because they're so excited that they're using AI for something. So you put even more eyes on it. So wherever you're putting AI, I think there's a very simple, like how, like being aware of how much visibility this has to what you're doing as a company, just in general is like one Lance. Then you want to think about, okay with whatever we're doing. Forget about frameworks and all these things. It's like, what are the potential harms that could come from us getting it wrong? And there's different harms that you could think about. But, you know, things like, are we going to lose just trust of the customers? Would the customers be proud of being aware of how we were able to do this? Like, did we use their data and information? And if we were to be very completely transparent, would they be okay with this? Right? Or did we miss a step in doing it? Are people going to feel like, you know, we're treated unfairly and these types of things? So the NIST framework does have, you know, methodologies for how you assess your own program. And like, kind of the pragmatic, like, here's how you assess the maturity of your program, here's the steps and things you need to do. Create policies, do these types of things. But then you can go through these kind of risk evaluation frameworks to sort of evaluate what are the potential harms that could come from going live with this AI system? And have we done things to sort of mitigate, like assess that risk and what have we done to mitigate it? And it kind of prescriptively walks you through how to do that. And there's software by the Way that takes that kind of framework as it is and helps guide you through it, automate steps in that process, things like that.

Okay, I like the idea of having a bit of software assistance if you do want to like audit like all your different use cases, I guess, or you got a new project, you don't want to be just like making stuff up from scratch and spending a lot of time working at risk rather than building stuff, I guess. So you, you want the process to.

Be fairly straightforward, a hundred percent. I mean there's a lot of steps in that process going back to the slowing down. A lot of the reasons innovation teams and companies are frustrated with governing teams is because the steps that they have to go through to get those governing teams up to speed typically require them to like do a lot of data entry and sort of restate a lot of information that should be self discoverable. So for example, what categories of data are going into this AI system? What AI model are we using? How is it in the, you know, runtime, where's it hosted these things you've been. Well, if you could just read the engineering code, the documentation that we have already, all these different artifacts that you would be able to answer those questions. But most governing teams don't have systems and software that are that technically connected. Right. And translate at that level. And that sucks up a lot of time and slows things down. So if you can shift to using a solution that enables you to kind of skip those steps, you can automate doing that and you can focus like the governing team, brain power and capacity on making the risk trade off decisions. Right. For the business. Not spending a lot of calories and time trying to gather information, understand context from people in the business.

Yeah, certainly if you like having to copy and paste stuff from like a project description document into a governance document is like, is not a great or exciting use of a lot of people's time. Whereas I guess, yeah, if you have the governance system hooked up to like the GitHub, repo or whatever, it's like, okay, you can see exactly what it's doing. That seems like a good automated solution. Suppose you want to get better at governance, like maybe this is like coming from your CEO who says, okay, we're using a lot of AI now we've got to do better AI governance. Where do you begin? Like what's step one?

Well, I would start with aligning with the CEO on what, what's driving him to want to say that, you know, and is it tied to, you know, specific ROI of certain projects And AI initiatives that are most important to get right, is it tied to compliance concerns, you know, fines, regulatory conformity, these types of things. So really understanding and aligning with what, what's the why behind that, I think is key. Then I think there's a huge element of even though you want to govern everything, you've got to prioritize and you've got to prioritize with what are the projects that are going to matter the most. Because at the end of the day you only have so much time and even with as much automation as possible, you've still got to focus on what are you, what are you going to do and how are you going to prioritize. So I think getting the alignment on the what's the why and then what are the, of all the AI initiatives that everyone's running around doing, which ones matter the most so we can really lean into them. And then I think you, you start to tackle going back to that point of which of these are the highest risk. And it's an element of have the most visibility because it touches the most customers, or it's the broadest visible on, you know, our site or our properties or things like that. And then what could go wrong that could be the most impactful, Right. And in a lot of times a simple tip or trick is look at the data instead of trying to figure out, oh, could I hallucinate? And there's something that's kind of hard to determine. Look at the data that's going into the AI. The data is much more deterministic, right? So I know that this data is from certain people, right? Is it personal data, is it governed under health care regulation, is it governed under banking, you know, regulation? Right. So just looking at the data, where it come from, what is in that data, what categories of data is it, is it biometric data? Okay. That is much more heavily governed from a compliance standpoint than things like email addresses. Right? So just looking at that is going to help you answer that. If, if this thing did go haywire, what could be the worst thing could happen? Well, if you look at those data sources, you can kind of extrapolate into that pretty, pretty accurately. Like these are more important to focus on governing and really making sure we're doing that well versus these others. Because the harm of getting it wrong is probably much more impactful.

Absolutely. I'll let that you're focusing on like what are the high risk projects, what are the high visibility projects? Because these are the things that matter and like worry about governing everything else. At a later date, I guess. Okay, so that seems like a good place to start. You've mentioned data governance and like some of the things like, like healthcare cases, like there's a lot of governance around healthcare data, like this is very personal. Is there a difference between your approach to AI governance versus data governance?

Absolutely. But they are also coupled together when you think about the controls. Like AI fundamentally is just a new engine that is using data, right? So it's existing data that we have and we're having AI use it now. There's completely different things that are happening that add to the amount of governing you have to do. But doing good data governance is almost not a prequel, but a parallel to doing AI governance. And when you look at a AI system you have to answer like kind of two questions is what data is it using? And from that you able to understand. Well, there's certain policies and governance that go into governing that data and then what is it doing in that data? This is actually the part that is kind of more fascinating to solve because historically. Well, let me, let me Describe 1.1 1, using data, data in itself is not like evil, it's what are you using it for, which is what causes problems. And so to ground that, right, Having someone's phone number, having that data is not in itself good or bad. Having someone's phone number to do multi factor validation for them to log in is normally a good thing. Having someone's phone number to SMS market message to them without the proper consent or opt in is normally a bad thing, right? So it's the purpose, it's the what are you using the data for? Historically, prior to generative AI and AI agents, what the data was used to do, the purpose of use was something that was deterministically coded into a software system, right? Like someone was basically whether it was low code or actually coding, like you were telling the software, use this data and do this thing. And that required human judgment, human decision and typically human governing processes and things like that and human review. It moved at human pace. Now when you fast forward to the AI era, you have AI agents that are reasoning and they're coming, it's non deterministic, right? They're coming up with given a problem statement, an objective and looking at the tools, they're connected to the data they're connected to, they reason and they determine what to do. So hypothetically, to give an example, you can imagine something like a customer support agent and a customer support agent that has access to the engineering backlog of all the enhancements and tickets and bugs and things. And if a customer opened a support case about some issue, it would be able to look and say this is a known defect, we're working on getting it fixed because it's got access to that. And that would normally be a great thing. Because you automated an end to end support case. That same AI agent sort of says, well actually what you asked about is similar to this other feature over here that's also in that same engineering ticketing system. And I want to reach back out to you to let you know that not only is that bug fixed, but here's these three new features that you might also be interested in. Like where did we go from we were doing customer support to now we're kind of marketing something maybe new to you. Right. And maybe it's like an additional purchase. So that was not something someone coded into the system. It was, it was like reasoning from, from the AI system to do it. So to, to get back to your question that gives you some analogies and understanding of there's difference in the controls and the guardrails. You have to have to actually govern these systems. And there's data controls that hit the data sources and then there's AI controls that sort of do things like grounding the AI into the task at hand and making sure it doesn't deviate, you know, making sure it doesn't respond to certain types of questions. Right. Because you just don't, don't want those types of things answered. Like this AI system that's supposed to do support should never answer political questions. Right. Type of thing. So there's different guardrails and different governing problems you have to worry about in AI systems because of that non deterministic nature of how the software technology works.

Absolutely. I mean this seems like a particular problem with LLMs that a lot of these, these jabos, I mean you think about like ChatGPT or Claude or Gemini or all these, they can answer almost any question in any tone of voice on any subject. But for solving business problems, like if you, if you're a customer support chapel, you really don't want it talking about politics or religion or you, I mean you don't want it writing in a poetic style. Yeah.

And even other, I mean a lot. Especially when you look at AI and enterprise, it's coupled to enterprise processes and enterprise data and even the same thing, you don't want someone that is in marketing to get access or information that should only go to someone in finance or even a certain level of someone in finance. Right. So there's a whole new set of problems that you have to think about and solve as you take AI out of the chat. GPT, Gemini, you know, kind of hallucination type concepts and actually put it into the enterprise. It's, it's, it explodes the problem set. It's like much harder to solve.

Okay, so I guess once you start thinking about governing agents, then is, is the trick to start with simple agents where there's sort of less surface area that you need to worry about or can you just dive straight into? Like we've got sophisticated agents, we just, we need rules in place to govern them.

Well, I think there's definitely layers that I think about, of, of how you start on your governing journey. So there are basic things like even the example you said that even a chatbot that someone deploys on their site in the enterprise, that you should probably put in the filtering guardrails to say don't respond to these types of questions, only pull information and answers from these trusted sources of data, not anything on the Internet. Right. Type of thing. So there's like very low hanging fruit basics that you should implement that are kind of layer one of defense. Then you get down to the next level of things like going back to the use case if you're connected to the company, you know, sets of data lakes and databases, but there's some data that shouldn't be included for certain agents or certain AI use cases. Right. So going back to my customer example, I may have an AI system that has access to the data lake that has customer information and in that customer information because we ship products to the customers their physical address. But for what the AI is doing, there's probably very few scenarios where I need the full physical address for that customer. So to prevent a potential leakage of that information or misuse of that information, I probably should mask that column of data out and maybe, maybe if I do need to know the area, I just mask it to the postal code like you, the AI system can only know the postal code, not the full home address of an individual to mitigate risk. So these things kind of come in layers and when you, when you do your governing framework, you just start to approach that methodology. You look at like, okay, here's all the potential risk. We're going to put default guardrails in place. That kind of are these broad strokes. But then we're going to refine and refine and refine and we're going to put additional controls at different levels. Right. There's identity level protections, there's AI agent level guardrails, there's data level controls. All these things become important mitigating measures. But. And they're all additive, right? Okay, yeah.

I mean, I, I can certainly see how the scope for a lot of big problems there. Like if someone starts asking you at Chatbot, like, oh yeah, can you give me the address of like your other customers in my area, that sort of thing. Or that, that could become a real problem if it starts giving out personal details.

100%. Yeah. Which is which? The importance of AI governance. Right. You want to make sure that teens that are just trying to move fast to turn on those things, think about those scenarios. And at least it's not like we're going to as a society, nail every scenario and get it right and not prevent anything. But you want to just try to mitigate things that, that you can do up front and be able to constantly be measuring and observing. So when you see something start to happen, you're able to quickly react to it.

Okay, absolutely. And you mentioned compliance before. So are there any particular, like laws people need to be aware of or particular compliance or particular rules they need to comply with?

There's lots of emerging, you know, AI regulations and laws. The big famous one in Europe is the EU AI Act. Even in California you have an automated decision making act. Colorado has an AI law that's been proposed regardless of even those coming out. One of the other things most people forget is there are laws that are enforceable today and effective today that govern AI and they're called data protection laws and regulations. And because most going back to point I made earlier, most AI systems use data, those data protection regulations and laws don't care if it's AI or non AI. It's using data for something and you still have to adhere to that. So even the, the gdpr, the, you know, CCPA here in the US and various other state laws and regulations, like they're all applicable even to AI systems. But there are new specific regulations because of the difference in the technology, the concern of the technology. Right. That are coming out that are going to further add to those regulations that organizations have to navigate. This is something we focus a lot on because it's a challenge for organizations to just keep up with what do I have to comply to? Like tell me which, which regulations and policies, because they have to then translate that internally to the teams to say, you know, we can or can't do that. Well, it's derived from, well, what are all the things that govern me as a company and organization that I have to adhere to so I can translate that into a process and a program that I can roll out to my company.

And I get the feeling for product teams it's not the most exciting thing to work out. Does my feature comply with a particular law or not? Other tools to help automate this sort.

Of thing increasingly more and more every day. And you know, the, the concept that I think people want to get to is can I, can I just call an API to sort of say solve compliance? Right. To tell me like, here's my code, tell me if it's compliant, here's my website, tell me if it's compliant. Here's this AI project, tell me if it's compliant and tell me the, when you find issues, no doubt you're going to help me mitigate and remediate. Right. Those issues. And that typically comes with some of these controls I was talking about of like, you know, let's mask the data and take it out of it. Boom. Now, now it's fine to do. And that's, that's really interesting because it gets back to that point of the data enablement, you know, plan and platform of like, how are these governing teams sort of effectively enabling and activating AI and data use for the business and speeding it up? Like this is actually really important. It's like you want to be able to speed up your ability to put more AI and more data online. That's actually a good sign of a healthy program, is you're doing more use of data and you're able to go live with projects faster. That's when you know you've got something humming and, and doing that is automated as possible is kind of what every organization is progressively moving more and more to.

Okay, yeah, I would say, I love the idea of like call an API. Just say, does this comply with the laws or not? Easy peasy. So we've had a recurring theme then throughout the show about like automating a lot of compliance, automating a lot of governance. Do you have a sense of like, which bits of governance at the moment are easy to automate and which bits still require human in the loop or, or have to be done manually.

The part that is much more automated today is the like information gathering piece. It's the going into the code, it's going into the AI project, it's going into the database and figuring out what's all the data, categorizing the data, classifying the data, what is it doing, you know, what AI system, what LLM automating that piece of it is something that A, is possible to do and B, organizations are, are very open to allowing that to happen. The parts that I think everyone's maturing in is the ability to truly have AI mimic human judgment at risk decisions. Right. So basically assessing that business trade off of is this okay to do or not okay to do, it's like, well, it's a, it's a risk decision. It's. And there's a lot of context and human judgment that goes into that. And both a, from a comfortability standpoint, people aren't ready for that to be automated yet. Right. People want to be in the loop, human in the loop control, bring me all the information so I don't have to manually gather it, but let me make that business decision and call. And then I think the, the technology is progressing, it's moving fast, but it equally is not there to be able to truly mimic all those different parameters that people put in their head, that different context, to kind of mimic the same decision consistently and accurately and confidently. But it's going to get there. And I think what you'll see over time is as decisions get made the same like primitives of the decision that historically get made again and again and again, people will be more comfortable with AI sort of automating that. It's a low risk activity, it's a low risk decision and it's high volume that we have to do and we're going to be comfortable with automating those, those level of decisions. And then of course the high risk ones are probably always going to have human in the loop review.

Absolutely. Yeah. If something is going to have an effect across your whole business, you probably don't want to just be like, oh well, what does ChatGPT say? You probably want the human to make the decision at the end. Okay. But I do like the idea that once you've made the same decisions, lots of times that's going to give, I guess it's data then that can feed into some sort of automation system. And then at least for the low.

Risk cases, you'll be able to reinforcement learning. I mean you're, you're taking all these risk decisions and you're, you're reinforcing learning and then you're looking at a new scenario and you're like wait, this is 90% the same as these other scenarios. Right. Or you break it into the again, fundamental primitive parts, decompose the problem and then you're able to say I don't need you to evaluate this whole decision again, 60% of it. If you agree with your historical assessment, you've already decided this and here's the net new that you should look at and consider. And again, why do we care about that? It speeds up the time to get through that risk review and decision process so that you can go live with the project. That's brilliant.

And I think one thing we've maybe been missing is that people are going to need to understand something about AI governance throughout your organization. So can you talk me through what sort of skills do you think you need? And maybe it's different for different teams. So maybe who needs to know what across your organization about AI governance?

It's an interesting question. You know, there's different altitudes. I guess you could answer that with one being if you fast forward like I'm a firm believer if you fast forward what data use in the world and what software computation is not going to be using AI, it's kind of like saying what software percentage uses electricity or something. It's like it's going to be, it's like 99.99% not saying everything's going to be non deterministic but, but it's hard to imagine like you know, any product or process that doesn't have AI in some portion of it. So when you think of it that way it's like everyone has to, this is just kind of the normal. It's like everything in a company is a digital process. We're going to say, well every process in a company is an AI powered digital process. So we all in an organization are going to need to understand what is, you know, the new primitives of AI governance that we have to all learn and have to become part of our just default muscle memory and things like that in an organization. So I think, I think it'll evolve to some kind of a state like that where everyone's going to need to have some baseline understanding of how the technology works, how to use it, how to kind of have the spiny sense of like sniff test. Something feels off here, right? That judgment. I don't know how to, I don't know how to articulate that, but kind of like the, the way we would receive a, a spam or a phishing call today and we all kind of have like signals start to go off because we've, we've been trained and conditioned of like this doesn't feel quite right of what, what normally is and you kind of know that this is something that could happen and there's the equivalence of that, right with, with all AI And I don't mean just like deep fakes, but like any AI where you can kind of, the human starts to build these judgment calls up where we can sort of see and feel that something is going off. Right. Something's not right with this process all of a sudden. So I think that will, you know, develop over time, but it'll become, you know, just like today where you have people that just have a knack for something in the technology stack is off or something's here, you're going to, you're going to have a little bit of that skill set build up where people are going to be able to pinpoint and isolate where they think. You know, hey, there is an issue here and there's, there's something to go look at. Okay.

Yeah. So I love the idea that everyone's going to need some level of understanding of AI governance because everyone's going to be using AI. But also like it's kind of cool having a spidey, A spidey sense for this has got, this is something that's gone wrong with AI. So. Yeah, yeah, a little superpower there. Before we wrap up, I'd like to talk a bit about like success. Like how do you know whether AI governance is working? How do you know when, when your initiatives have gone well?

I go back to the, the time and the number of AI uses that you are doing as an organization is a good, healthy signal that you are confident in the technology and deploying it at scale. And that means your governing process seems to be working. And if you are able to say the time it takes for us to review a new initiative to getting it live from a governing perspective is reduced and going down, that's a good indicator of a successful program. Right. And it, it shows that partnership from, you've invested in the automation, you've invested in the ability to repeat those risk decisions in a reliable way. And you've invested in a way in a partnership with the business where they're reviewing and leaning into the AI governing processes and teams and you're, you're reacting and responding at the speed they need you to. Right where they don't want to kind of go rogue and, and do something separate or just ignore your process altogether. So the speed of your program and the volume of governing use cases I think are, are the two top things that define success.

Okay, I like that because it's kind of, you're, you're seeing it's productivity gains there really about like the, the speed of governance and then it's happening before Something bad has happened. Rather than like, we've had no algorithm. Algorithmic disengorgements this year. That's our measure of success.

We'll eventually say it, right? Yes, yes, that's a good indicator of success. I mean, I think those are kind of lagging indicators. More so. I think the ones I said are a bit leading. But yeah, you're right at the end of the day, ensuring you didn't have something go wrong in production, whether it was a fine or regulatory penalty. Honestly, those are, those are like not as impactful as the, the distrust you have with your customers in your market. Because your AI is just, you know, a, it's massively wrong and does something bad, it takes a bad action, like deleting the database in production. B, you use customer data that you shouldn't and you end up with bad mud on your face. Right. And you have to kind of rebuild trust from that, which is really, really bad. Maybe you get a fine or a penalty or something, but those seem kind of pretty minute compared to like, wow, you, you had to delete all your customer data and, and you lost trust with all your customers. That's harder to recover from.

Absolutely. I'm trustworthy customers. I mean, that's like a, almost invaluable resource there. Yeah. Certainly something you don't want to squander on some silly project. All right, super. So finally, I always want more people to follow. So whose work are you interested in right now?

I'll give, I'll give two that we kind of highlight publicly. One is Telus, who is a telecommunication company in Canada and their team that is run out of the privacy group there is who actually coined this data enablement plan term for it's not a governing risk assessment process, it's a data enablement, you know, plan. And what's interesting, what I love about the work they're doing is they don't view their work as a. A governing and a let's mitigate risk and kind of prevent bad things from happening. It's how do we activate the business and enable this data use? And the second thing they do is they don't think of it just from one risk domain. Most organizations have like, privacy is kind of1group.AI governance in some cases is a separate group. Compliance is a third. Security is a fourth. Like, they're different frameworks, they're different processes, they're different teams. Often they're different tools they're using. They're really unifying those together. They think of it as like, how do we converge all these things into one thing that respects all those different disciplines and domains and frameworks, but allows us to deduplicate the overlap so the business gets a better experience. Right. And that's part of what speeds up the process. So unifying that together is a key thing. I see that trend happening across a lot of customers, but they, they really highlight and come out on top, I think, with really pioneering and getting a lot of innovation awards for doing that. So I really like the work they're doing there. The second one, I would say is one of our key partners, Microsoft and Microsoft and the group under the security group at Microsoft really had this, I think, very leading vision for how AI agents are going to fundamentally change the way security operations and security operations centers work. And it kind of puts us in this mindset. We're seeing it, you know, in a lot of different groups in a company. But how AI agents are going to be doing the lion's share of the work that you have to do as part of like compliance or governance team teams. You can almost imagine there's an AI agent that is governing the AI use that is happening. And when, when an AI use in the business is reasoning and making a decision, they're not going to ping a human to say, is this okay to do? They're going to ping an AI agent and they're going to reason with an AI agent that is skilled on compliance and governing principles and data ethics and these things to reason with. Wait, you can't do that because we as a company have to comply to these things and, and we have these data ethics standards that we adhere to and things like that. So that's just super fascinating. I believe that is the future where we're going to be going as well as governing and compliance teams where AI agents are really, you know, augmenting, obviously human in the loop is still very, very key, but they're, they're massively augmenting and giving scale to all these teams that are trying to govern and comply to this continuing to explode use of data that is happening, the volume of data, the number of AI use cases. And I think they just have a great vision. And that team, we've been working a lot with them on partnering with, you know, building, building some things and integrating. But I think that is the future of many things in enterprises and we're excited to be on that journey as well.

Absolutely. I mean, so very cool stories there. And I love the idea you said from Telus, where it's, it's a mix of, you got Some technology there, you've got the process there, you got the people together and all those things are coming together. And yeah, certainly the Microsoft story is like, well, okay, you got agents checking all the AI governance, I guess probably like more lays like AI agents checking the AI agents. It's probably just agents all the way down, right?

Yeah, exactly. It's a good analogy. It's like, but if you think about the sheer volume like of think about security, operations center, the amount of events coming in, it's impossible to pre AI causing events. Right. Coming in, it's impossible for enough humans to be hired in a company to go look at every single event and investigate. It was impossible. You can't do it. You have to prioritize today. And when you have outside actors and the bad actors are going to start using AI agents to try to attack, you're talking about orders of magnitude of those events going to explode. The only way to defend against that AI is with your own AI and you have to have your own sort of AI agents that are doing this. And the same is true for these compliance and governing teams of like, it's impossible to go look at every single scenario of here's a new AI agent that is using or reasoning something or new data that was added. So AI agents help scale these teams and are going to get us to the ability, going back to the very first point I made of 1.3 billion AI systems running in just a few years in AI agents. It's going to give us the scale to confidently govern all those agents.

Okay, yeah, definitely a lot to think about and some big challenges here just to make sure that all your governance does scale to I guess to match your AI adoption ambitions. But yeah, exciting time for sure. All right, super. Thank you so much for your time, Blake.

You bet. Thanks Richie, for having me.