How To Build Responsible AI With Agentforce | Salesforce CIO Corner

One of the favorite things my team hears me say is if you put AI on top of crap, it's just faster running crap. You've got to have that knowledge of exactly where the data is, where it's flowing.

Welcome to CIO Corner. Today I'm really happy to be joined by Anne funai, CIO at IBM, one of the largest technology companies in the world that helps businesses develop responsible AI practices. Ann, thank you so much for joining me today.

I'm so happy to be here. I love every conversation we have. Juan.

So today let's talk a little bit about what CIOs should know about AI governance. What challenges have you faced to ensure that responsible use of IBM's AI systems takes place in your company and in other companies as well?

What's been fascinating for me is really the AI is bringing to the forefront the importance of understanding your business, understanding your business processes and understanding the systems and the data flows behind all of those processes. You've got to have that knowledge of exactly where the data is, where it's flowing. Because with the AI, AI is already prone to hallucinations as it's under training. But what we're finding is just in this AI world, it's also not just single systems. Like you don't live in a single system anymore. Yeah, there's CRMs and ERPs, H R I S and this wonderful new AI world we're living in, there's data coming from all sorts of distant systems together that drive insights.

I think that that's very, very important for all CIOs to keep in mind. Right. As you build your AI strategy, your AI agent strategy to ensure that you have the appropriate platform that to have visibility to what's going on in your AI world.

I know Salesforce is heavily focused on agentic AI right now with the launch of Agent Force. How are you guys ensuring that AI agents built by non developers with low code, no code, you know, have the appropriate guardrails?

It's a great question. You know, I'm coming fresh from our internal Agent Force kickoff where we had 600 of our top leaders in a room. We then invited the rest of the organization to participate in that kickoff. We all ended up getting hands on training on how to build an agent. And what really was fascinating to me is how this platform technology that we have been building for quite some time and now extends into Agent Force allows people with huge technical expertise to quickly understand how an agent, that it's intelligent, that it's capable of taking action for the user, can be built, but also people who perhaps have not been at the forefront of building or working on technology, how they can sit in front of the machine and with relative ease understand how to build a prompt, how to actually utilize the salesforce technologies to articulate actions and ultimately build an agent that can take action for a user. And my goodness. One of the things that is really clear in my mind now, Ann, is that we are now in a completely new place when it comes to using AI, because these agents make AI accessible and make AI actionable, which to me is incredibly important. And making that easy for people of all different functions, all different aspects of the business to understand and use is going to be critical in the success of implementing these agents.

I don't think in any of our conversations we talked about this one before, but that's an actually really interesting parallel to me because similar here at IBM we've done multiple of what we call the Watson X rallies and it is really to bring the whole company along. It's not just technical people go figure out how to use this. But like if you're in hr, if you're in supply chain, if you're, you know, if you're finance, hands on. And it started with some really simple training, like just know the terminology, what is an LLM, what's a rag model, like what do these things actually mean? Because we, I think in the technical world we tend to throw these things out. And one of my past lives between a few years ago I worked for Under Armour and like one of the things that really helped me bridge that gap was talking in analogies and I think this, I think IBM, we've tried to do very similar stuff, bring folks on the journey and I think one of the most fascinating photos from some of our rallies is seeing our CFO seeing our Chro, like doing the training and they're building models together and it's really, you know, just give everyone a chance to be hands on and feel like they're part of that new world. So, you know, I love that. And one of the things this is actually leading to over here, some conversations on what does a future workforce look like for us as well. You know, you think about kind of the normal path is, you know, you're going up a career ladder like this. But it's like, you know, some of these things, like the agents I think are going to allow like quicker progression in careers, like you're going to learn stuff, it's going to give people kind of higher value in the roles they play over time as well.

That's a really interesting concept that I really had not thought much about. You know, it's the whole notion that in the future, perhaps the people who are best of, in taking advantage of agents, taking advantage of AI, will actually have a bit of an advantage over other candidates who have perhaps resisted the use of this technology. And one of the key messages that I think it's really important for us as CIOs to get in front of our organizations is that, hey, first of all, don't be afraid of AI. Embrace AI and think of AI. Think about these agents as a competitive advantage for you to be able to do more for the company and find opportunities in the future. Now, all of this has to be managed effectively. So I'm going to take you to now, let's say we have this conversation five years from now on and we're thinking about all the good work that our companies have done, all the good work that our companies have done together in advancing agents, in advancing AI in the enterprise. And now we find ourselves with thousands, hundreds of thousands of agents running multiple tasks and work being done through the agents in the organization. What do you think are some of the things that today and in the future we need to be thinking about to ensure that we have good governance around all of these agents?

So, yeah, I'm going to start with one that I've seen it come up repeatedly as a challenge across the entire CIO space. I think one of the big things we're going to need to keep an eye on. I think we've laid the groundwork data knowing your underlying systems. But one thing that connects the systems to the AI, to the agents is the level of privilege you have in a system. So are you an administrator, are you an end user, are you a customer? And I think there's going to be, I'll say it's everything old is new again. Thinking about like, you know, not accidentally doing privilege escalation, I think in AI, and I know that's a very like fine point, but I think some of the use cases we're seeing, you know, gonna have to be very careful in the shaping of AI and agents going forward. You know, as you think about things like again, let me go back to crossing multiple systems. You know, there's very specific rules around Sarbanes Oxley requirements like that. So this notion where we think about access control at a system level, my instinct says there's something coming with some access control at an AI at an agentic level. Because, you know, right now, as we're evolving on this curve, I think there's always like one of the big risks. If I look out five years from now, I hope we've solved this. But you know, there's the agents act in different ways versus your privilege level within a system and that's how we want them to react and act. So for example, me and my seats, let me go to an hr. Skip over to an HR example. Like there's information I would see in an HR system about my whole organization that, you know, a manager who's a layer or two down from me, you know, shouldn't see or you know, our heads of HR should see that information. How do you start to have your agents think and train around that? Like what would be in a, in a system of record access control versus understanding the user, the person that's interacting with the agent and the AI.

Yeah. You know, you made me think of another important concept that I think as CIOs we're going to have to continue to evolve and especially as we continue to deploy and use more agents in the enterprise. And it's that whole notion of addressing governance at the start. What does that mean to me based on what you just described? It means ensuring that in your architecture practice, where you're actually building and creating the solutions that support the enterprise, your architecture is also agent centric. And that all these different elements that you mentioned are taking into account in the build out of the architecture that you're supporting.

Yep. And I remember one of the demos at Dreamforce a little bit ago was a customer support, a calling, customer support one. And like some of those are very obvious, like, you know, you're talking to a customer. But again, think five years down the road of, you know, I've got my ERP system talking to my CRM system, I'm getting a whole different set of insights from an agent. Again that, that the roles and understanding, I'll call it a more directed AI understanding who and what Persona is on the other side of that AI, I think is going to, I think that's a really cool evolution that could be ahead of us.

And one of the things that I have learned over the years as a CIO is that partnerships matter. And like I mentioned at the start, I've had a great partnership with IBM for a long, long time. It's really important that in this AI world we also build strong partnerships in AI development. IBM's done that really well. Do you have some examples for our audience in terms of strategical, strategic and technical advantages that CIOs can gain from building these partnerships with the right partnerships. I would like to think like IBM and Salesforce.

Yeah, I love that. And that's actually something and I was hinting at it a little in, in the last chunk of the conversation with you know, your ERP and your CRM system talking to each other. But you know, one of the things I' loved I'll talk about this as a career evolution is, you know, coming out of College in the 90s all fresh faced, there was this notion of like no, you have to be all one this shop or all that shop. And then this kind of notion of coopetition was used. Like, you know, you're cooperating but you're competing. That kind of fell by the wayside. I feel like we're moving back. AI is bringing us back to this world of coopetition again. So you know, Salesforce and IBM, there's some areas we, there's, there's a few areas we compete in but a lot more we cooperate in our services arm, you know, we're delivering Salesforce services, some other very large vendors and partnerships that we do. It's same thing. There might be a little area we compete but a lot we're trying to partner in. And going forward I think the value of, you know, I feel like this is, you know, bringing all the pieces together. It's like the value of these partnerships. Understanding what it takes to run your company, understanding the business process, understanding where the data needs to flow to, you know, allow your company to be the most nimble, fast moving, you know, time to revenue tied to product out the door, whatever your industry. It's like these systems have to talk to each other. There have to be simple interconnections. And I think maybe it was a little more hush hush 25 years ago when I was getting started, but now it's like I think we need to talk openly as no one of us can go at this alone. If I hearken back to like I think about freshman year economics, they talk about, you know, you outsource, what are your less important things. That was kind of an outsourcing talk, but really I kind of internalize this in my role now of like IBM should do what it does best, partner where we don't like why would I want to go build a whole other CRM system like that? That's not IBM's core strength, that's not the AI and hybrid cloud company that we want to be. And so again I think going forward you cannot minimize the importance of, you know, really taking that mirror Check and saying, no one goes at this alone.

I like that so much. And, you know, for now, a few years, we've been working diligently as IT shops to bring our business and IT together to ensuring that there's no separation anymore between the two. I really believe with what you just said, that as we deploy more AI, as we implement more agents, that proximity between business and it has to be closer than ever, ever before. So I have one last question for you. And, you know, one of the things that I always appreciate about you is the energy, the enthusiasm that you have in describing how technology can solve big business problems. Now, when we talk about AI governance, that doesn't sound that exciting. How do you maintain the attention? How do you make AI governance something that is not viewed as a burden, but actually something that it's important to do to gain the most value from the implementation of AI, the implementation of agents in the enterprise.

Governance isn't sexy. It is not what people get excited about. Like, I mean, no, don't get me wrong, there's probably a lot of people who do, you know, dig into that, but, like, that isn't the sexy stuff. And one of the things I've said to my team, you know, to partners that we work with, is like, look, in this world going forward, I fundamentally believe the CIO's role is to make the users inside the company's job frictionless, simple, and as easy as possible so they can spend their time delivering value to the customer that your company is serving. And that is really a guiding principle. And through the evolution of technology and where we're going, even with where AI and agents are going now, I think that principle becomes, I mean, even more of kind of a guiding principle of how I want. I, as a cio, want to serve the IBM Corporation. If, If I'm not providing a simple way to close the sale, if I'm not providing a simple way for our accountants to, you know, close the ledger at that close of a quarter, if I don't, If I can't help our hardware team, you know, get that hardware out the door and to the customers, like, you know, I. I should always be looking and my team should always be looking for it. And I, again, I do believe where we are going with AI, get the data clean, get your systems clean, understand your flows, think about your partnerships, get all those things constantly refined, constantly turn the crank. And I think that's really fundamental to what I believe my job and my team's job as a CIO are.

You made me reflect in Terms of AI governance, we cannot articulate AI governance as a barrier to progress when it comes to the implementation of AI and agents in the enterprise. We should be thinking just CIOs, that AI governance is an enabler to actually deploy responsible and effective agents and AI in the enterprise without causing significant stress to the people who want to use these technologies to improve the overall performance of the organization to create more values. I'm thinking that in a world with many, many, many, many agents, you still have to maintain some level of visibility to understand what these agents are doing, what data they're using, what systems these agents are interacting with, and ultimately what systems perhaps they're updating automatically in the background. So the way I see it, Ann, there's a spectrum of agents. There are very basic agents that just pull information, interact with a user, respond back. There are other agents that do a lot of things, a lot of actions for the user in the background. The way that I think we're going to end up managing these and putting guardrails is that for those low complexity agents with low impact or low risk, we'll just simply maintain a good list of what those agents are, what they're doing in the enterprise. For the more complex, more impactful agents, we will definitely have some level of IT governance that will support the management of those agents. Well, Ann, thank you so much for joining me today. It was awesome.

Thank you. No, always a pleasure. This was really fun.

Please check out the links in the description below and be sure to stay tuned for more episodes of CIO Corner.