Rise and Risks of AI in Business Leadership

Welcome to the Cybersecurity Vault. I'm your host, Matthew Rosenquist, CISO, industry cybersecurity strategist and advisor. And today we're going to talk about the rise in risks of AI as innovators look to put it in business leadership roles. And I'm going to be talking with Stuart Evans. Stuart is a distinguished service professor at Carnegie Mellon University. With a finger on the pulse of technology innovation and adoption in Silicon Valley. Welcome, Stuart. Thank you for coming and talking with us.

Thank you, Matthew. It's a real pleasure to be here, and I'm just super excited to be able to share some insights. I think that the rest of the world is now being— had this sort of Silicon Valley mindset imposed on them, and I think it's a shock to many, especially with the technology changing as rapidly as it is. I'm delighted to be here and I'm happy to share any insights with you.

Well, this whole topic kind of blows my mind a little bit, right? When I'm trying to think about this, could AI really replace leadership roles, executive leadership roles like a chief financial officer or chief information officer or chief marketing officer? You know, I mean, what are you seeing when it comes to AI either supporting or taking responsibility for business leadership function?

I have a sort of different view on this, Matthew, because, you know, I see my students are all software engineers and I see the impact of Cursor and Lovable and Windsurfer on how the market shifted. And certainly at the entry-level stage, AI is taking the dominant role and it's working really well. For senior management roles though, and I think the same is true for senior software engineers, while 90% of stuff can get done with AI on a very consistent basis and very comprehensively, it's the little tweaks that make the difference. And you've seen that particularly with open source software. If you look at the data on open source software, AI is actually slowing down some of the really kind of clever software engineers to do things that need a slight tweak here and there. And I think the same is true with leadership. I mean, leadership is about judgment. And intuition as much as it is about kind of patterns and regularity. And my good friend Mark Settle, who I mentioned before, just wrote a white paper called Time and Attention Engineering. And I think, you know, the thesis is, look, AI is inevitably going to automate standardized processes. We've seen that happen all the time now. It's, it's, it's having massive impact on every organization. But what is it we uniquely do outside of those repetitive processes. And it's the art of judgment. And that's really what leadership is all about. It's not just about making like the right decisions because the problem right now is you don't know if a decision's gonna be right for 6 months or 9 months cuz the world is changing so quickly. So that ability of, of leaders to adapt and adapt on the fly, I think is gonna become the most sort of defining characteristics of the leaders of the future.

It's interesting, right? Because AI evolves so quickly. When we saw the first instances of generative AI and the text that it was writing or the stories it was creating or the videos, I mean, it was obvious. People were saying, oh, you can tell this is AI. This is very bland. This is horrible. No, it can't. It can't be creative. It can't sound like a human. And yet very shortly, just a few generations, and when we're talking generations, not human generations, AI generations. So weeks to months, not decades. A few generations later, it was getting better and better and better. And now some of the creativity capabilities, some of the insights that these systems are having is phenomenal. So is the concern that AI, oh, it won't have that intuitiveness, it won't have that judgment. Is that a point in time perspective? Or do you think that also will be overcome? With new data feeds, new insights, new learnings that AI may be able to incorporate, deftly incorporate to be in those roles better than the average type of leader?

Oh, that's a great question, Matthew. And it's hard to unpack all of the things that you brought up. I think I'd like to kick it off with a little story. I've just finished teaching a class on next-generation CIOs. And I started the class with a picture from a magazine called Upside that I'm sure you recall from Silicon Valley days. And it had a picture of Gil Amelio when he was at National Semi on a rocket ship doing a great job. But the headline underneath was AI enters the mainstream enterprise. And this was like 1990. So, you know, I think AI has been around for a while. And we have this theory about innovation. We call it flexible recycling where something comes around that almost works and doesn't quite work. And then it kind, and it goes away again and then comes back. And AI had been around for, for a long time. I, I studied operations research and came to work in the decision analysis group here at Stanford. And I think the precursors were, the signs were always there. And I think when you look at what Google was doing, what the hyperscalers were doing with the billions of interactions they were doing, they're all using AI. But what came together was what I call a, a, a kaleidoscopic shift. And it's just like, you know, when you're a kid and you're playing with a kaleidoscope and you just twist it and the whole world changes. And that was that ChatGPT moment. It was, you know, it was the thing that made it popular. It was like, you and I go back long enough to know, it was like Lotus 1-2-3 for the IBM PC.

That kind of, that was that kind of a mind-shattering.

And so these things are what economists call punctuated equilibria. You know, the equilibrium of the old day was just shifted and, and it changed and it changed when Deepsea came out too. That was another kind of, um, you know, kaleidoscopic shift that people kind of became aware of. But the impact that AI, uh, since GPT came out and the impact AI has had on, on every enterprise has been earth-shattering. I mean, it's changed productivity. Beyond all recognition. And it's strained organizations massively because the people who embrace the technology are reaping major benefits. And in most enterprises, it's kind of happened at 5 different levels. You know, you and I talked about the shadow IT, the shadow AI that's going on where every individual is, is like going in their VPN before they go for a meeting with a CFO to figure out, get more data, get more data, or figure out what's going on. And then you have this sort of, the sort of citizen development where you'll have groups of people in marketing or R&D or in engineering where they get together and then they can use AI to increase their productivity. Then you have this sort of, this sort of formal IT-driven activity with, you know, the IT service desk, all of the service desks, all the stuff that that, that Moveworks kind of pioneered. And, you know, Moveworks was around before COVID They were the first to use AI really substantially in the IT organization. And they did a fantastic job. And then you've got the sort of C-suite where AI is being used at a number of different levels. I mean, clearly the office of the CFO is all numerical. It's, there's a lot of stuff that can be done with contracts and big impact there. But strategically for the CEO and the board, there are other things that can happen that really collectively have just increased the speed of action. You no longer have to wait to go to an analyst to get a view of, you know, what's going on. So now I think many corporations are building digital twins of their enterprise instead of like doing a scenario planning exercise once every 5 years. Doing it every 5 weeks because they can do it now. And so when they see a strategic shift, they can examine, you know, what are the possible scenarios that might come from this? I'm seeing, for example, onboarding of new board members to try and brief them on previous earnings calls, on all kinds of stuff, competitors, what people are doing. That kind of information is really at your fingertips. It's moving on. So I think the organizational impact at those different levels is really quite profound. And we don't really have an organizational sort of structure to match this new kind of world because essentially much of white-collar work is, is vaporizing. We see it in the employment numbers. It's having a profound impact on the number of people that are working in companies. And more importantly, it kind of represents a real shift. So historically, we've always thought the smart people in the organization and strategic planning can, can figure out what's going on and tell us what to do and things like that. Now it's the operators that need instant access to data to make their decisions. If you're in the middle of a negotiation, for example, you can't say, hang on a sec, I need to go to my FP&A people, figure out what the implications of this are, and then, and then move on. So I think those sort of decision-aiding platforms are being very helpful. But the real dilemma that I think we're facing right now, which is a conundrum, is that the innovation cycle is moving quicker than the deployment cycle. So the real big issue that I think people are having to confront is we don't want to invest in something that looks really good at first, but 6 months later it's going to be obsolete. So how do we figure out ways of flexibly deploying are assets that can be reconfigured and sort of changed as the world kind of unfolds. That's, I think, the central problem.

So you bring up a lot of interesting points here that right now the systems are great at advisement because they can go off and look at multiple sources of information, aggregate that, bring that back, present it in a way that's starting to be more contextual based on your prompts and be able to provide those decision makers with a lot more insights, not just data, but information. And now we're seeing that that bubble reached a little higher where it's not just information, it can also be recommendations. Yes. So instead of saying, hey, tell me about or teach me about X, you know, teach me about the market I'm in, You can start to ask, well, how do I move that market? How do I, you know, what should I do next? And now that's a leadership decision. And for these early adopters, it seems like they're moving in sync with AI. They're driving AI to be able to not only gather information, but be able to help make decisions. And they're adopting that. Then they're asking for the next thing. Well, if you can, help me make a decision. This is what I need to do as a CFO or even a CEO. Right. Well, the next logical step is to say, well, can you do that for me? And now we've got Agentic AI that, yeah, that's part of what the selling point of Agentic AI is. Yeah, I can go off and do that. Let me go create that PO for you. Let me go create that marketing material or that marketing strategy for you. And so we see in lockstep where the capabilities are matching the demands and the demands now are getting higher once the capability is matured. So do we get to a point where that CEO who maybe is really good at innovation, but not so good at running a business, right? And with 90% of businesses failing because of poor management, not necessarily poor ideas, Does this AI capability now, you know, looking forward, does it elevate to really doing a lot of the functions of that C-suite officer so the officer can go off and continue to do their strengths, their innovation, their ideas, and so forth? Where do you see that?

100%, Matthew. I couldn't agree with you more. The biggest impact I'm seeing now in terms of moving the needle is with kind of rockstar CEOs that understand these small, old-school kind of professional services kind of businesses running things like dialysis clinics or movie theaters or all kinds of things like that. And what I'm seeing is the savvy ones are saying, hey, we didn't really have much of a software footprint in our organization. We might use QuickBooks or something like that, but now we've got access to stuff that was previously only available to Fortune 500 companies. And so we've got a rocket ship that we previously had a little bicycle. And so, so the ones that are adept at doing it are able to acquire rapidly and roll out that technology. And in a way that our private equity friends are seeing significant returns. So I think that's the first indication to validate the point that you were trying to make earlier, that AI can take away the everyday drudgery of what people have to do and release them to do what they uniquely do best. And I think that's what we all have to decide what we do. What is it we uniquely do best? And it's in a large company, right? Innovation is one of the key roles. But I think at the end of the day, the speed of change is bringing out the importance of the leadership role of aligning everybody together. Because one of the things that AI has done, it's kind of, it's both fragmented and sort of integrated this sort of alignment at the apex of an organization. You know, the C-suite has to be together on what it's doing. And, you know, you can imagine a situation now where you've got a decision to make in a board meeting and AI is kind of being involved and it says, hey, look, you know, those two board members, it looks like they had a bad night last night. I'm not sure they're going to be at their best to make a decision. So why don't we weight the decision process such that the ones that look sharp and look like they're really into this are the ones that are going to be like making decisions. So I think that the change is just so profound that we're still in those kind of early stages. You know, in evolutionary biology, there's a term called phenotype plasticity, which refers to the ability to like rewire on the fly when things change. And it's abundant in nature. The best example, I'm from the UK and in Scotland, there's a bird called a ptarmigan. Which is a big kind of bird that's in the highlands. And in the wintertime, they take a white plumage on, they molt, and in the summer it's brown. And now with global warming that they're experiencing over there right now, the snow melts earlier. So some of the ptarmigans are smart enough when they've got like a white plumage against a black background, they're pretty exposed to their predators. Some of them have taken to like rolling in the mud to darken themselves as protection. And then they're the ones that survive and they're the ones that then become the sort of dominant DNA. And I think that, that, that, that's kind of where things are going to go with these large companies and how you align everybody to adapt on the fly and continuously adapt is going to be key. So I think that our paradigm shift is in its early stages. And we still bring a lot of baggage on the old way of managing. I see, for example, there's rumblings about the curriculum at business schools now. Are we teaching our prospective future leaders the right kind of skills to have? Do you need to know a Black and Scholes model to figure out how to value a company? I know you don't anymore. You know, you can just ask Claude or ChatGPT and it tells you these things. So I think the You know, we're really in a very, I think we're probably at the end of the first innings. I think that's probably where we are right now. And I think there's still a lot more innovation to come. I mean, the money that's been invested by the venture community is at a record. The valuations are crazy. The amount of money that people are paying for talent right now is just insane. We're back in the old acquihire days where you buy companies for the talent again. So I do think that what you, what you. What you alluded to before is something that, you know, the idea that AGI is going to be running the company is probably a stretch, but the idea that we'll be running a company with significantly fewer people than we have today, I think, is the reality that we're going to have to deal with. As you say, as we have, and I think this is where we are right now, I think this is the beginning of the second innings, is that every CISO and CIO has had 6 months pilots and playing around with startups and spending a lot of money on the bigger infrastructure plays and things like that. And now it's time for the rubber to hit the road. I think people are just literally starting to deploy multiple agents across the enterprises to realize the dreams that have been sold to them. And now I think that historically in large enterprises, you know, we view uncertainty in a very different way. So, I don't know, 40 to 200 use cases is probably the norm that people are working with right now. But in Silicon Valley, 1 in 1,000 companies make it. So I think we've got to embrace a much bigger portfolio of activities that we can redeploy and remerge and have sort of capabilities that can be dynamically applied rather than sort of fixed and the way things are going. I think that's the mindset that we need to to build right now. It's a different mindset. And then it's kind of natural in entrepreneurs. That's the way it works. You have to shift gears 4 or 5 times a day at a minimum, probably more than that. And I think that ability to be versatile is really key. And we have this notion called adaptive DNA where we kind of look at the ability, the different capabilities of people. And when, if you look at, you know, I'll not bore you with it, but if If you look at how animals adapt, it's really interesting to learn from nature. So if you think about versatility, the chameleon comes to mind. The chameleon changes color instantly against its background. It's doing that to both protect itself by camouflaging itself from predators, but also to hunt for prey, to be able to hunt for prey. So you've got that duality going on. It's like the explore-exploit notion that's prevalent in management thinking right now. Jim March's notion is strong. You also have different kind of capabilities like resilience. So if you look at a starfish, a starfish can lose a limb and grow it back. And, you know, resilient organizations are really key. And then sometimes we need to be really robust. We've got to withstand change. I mean, I think that there are, there are some people right now who are smart enough to realize that breaking glass and doing all this innovation is a good idea to rekindle, forge the way a company works. But there are other companies that say, hey, we're not quite sure where it's going to go right now. Let's go into a bunker for a little while and just wait.

Wait this out.

See what's happening. And that's more like a crocodile. Crocodiles like sitting there with eyes above the water. Let's see what's going on. But the crocodile is robust because it can eat anything. It's the one creature whose gastric intestinal processes have evolved to the point that anything that moves is a good meal. It doesn't really care. So you've got that robustness and resilience, and then you've got the versatility of the chameleon. But then you have the agility like a cheetah. You know, the cheetah is a really good case in point where You know, they can accelerate very rapidly when they see their prey, but typically it takes 8 engagements to get a, to capture a deer, the antelope that they feed off. If it goes beyond number 8, then they're in trouble. And it's a little bit like the way in our software organizations, we just keep people on these agile sprints time after time after time, and they burn out. The sad thing about the cheetah is it's dependent upon one type of antelope for its food. So it can't move. And then we have this kind of modularity concept, like a bee, you know, a bee's nest. It's got all the components in where it makes, and modularity is another capability. So I think, you know, going forward, we're going to see a much stronger emphasis placed on how much flexibility we have to adapt as a— because we can't know what the future is. It's a complex situation. If you can't know what the future is, you need to be making investment decisions, not on a rifle shot ROI where you know everything, but on how malleable is what we do. How can we shift resources quickly from failed experiments into the ones that are working? We've got to view these kind of AI development activities like a portfolio of activities of which maybe 5% work out. So we've got to look, you know, I mentioned before the explore-exploit kind of part. Which is pretty dominant now from, from what Jim March at Stanford Business School put out. We'd like to add to that saying you also, in, in addition to, to exploring for new things and exploiting existing opportunities, you've got to correct mistakes very quickly. And you've also got to be putting protection in place. And this is absolutely crucial for cybersecurity. You've got to be able to quickly figure out where you've got vulnerabilities. And fix them if they're not right. But by the same token, you've got to inoculate yourself for future problems to protect yourself. So I think that's the kind of paradigm we need to go moving forward. It's a complex, different way of thinking about the world.

So we're at the beginning of the second inning, and we know the capabilities of AI are going up. We know the value allows organizations to hit above their weight class, right, when they successfully adopt it and use it for the technology. And again, that's step that technology then gets better. That gives an advantage to early adopters. That promotes more funding and innovation for a whole bunch more AI types of solutions and services. And yeah, maybe only 5% or 10% survive and become viable, but still you've got a massive number. So at the end of the day, you've got many more use cases and potential value propositions. So you've got value, you've got adoption, you've got rise of innovation, and the winners are the ones who rush to integrate this for a benefit. That tells me we're going to see an increase in velocity in the use cases, in the amount of adoption for organizations, small, medium, and large. That paints the future. But my question is, as we rush towards this, specifically for AI leadership support or functions, what are the risks that we now have to accept or embrace or prepare for, mitigate? I mean, what are the risks for this velocity moving forward and adoption?

Yeah, you know, that's the key question that everybody is confronting right now, I think, Matthew. It's difficult. For the last 18 years, I've taught a class on innovation and entrepreneurship in Silicon Valley, and it's different. It's different every year. But, you know, the one of the things I always get from my students is, oh, Professor, I've got a great idea. I've got a great idea. And, and, and, and ideas are really important DNA of what comes afterwards. But the real issue is how you scale something and how you get it to be in industrial strength. And I think scaling requires a different kind of flexibility. Than coming up with ideas and things like that. So I think the risks of high velocity organizations are the reason we have so few public offerings, because to go from like a Series C funding to becoming public is a super complex activity. And then when you're a public company, it's a whole new world. It's a different game out there. You know, you have to manage the quarterly earnings calls. That's the name of the game. You've got no choice. So you've got different kinds of constraints at different stages of evolution. So I think that the risks of moving at high velocity are really different from stable organizations where, you know, you 5% every year and you're just doing this and you're doing that. That's a different kind of game. So the risks are really that when you hit a wall at high speed, you do a lot more damage than you do if you're just walking along. So that the biggest risk is, is the potential catastrophe of of what's going on. And I think that's something that we, you know, when we drive a car, we don't think about what happens if we go off the road. We continue to be safe. We know that we check the brakes. We make sure everything's working right. So I think that's what large companies are kind of doing right now. The risks from the cybersecurity point of view are really the ones that are just almost unthinkable right now because the potential that we can imagine of what's going to happen when all these agents are running around interacting with each other autonomously is something that humans can't really begin to understand. And we all know that nothing's perfect. And quite honestly, when we bring on, when we onboard a new employee, we give them 6 months before we expect them to be like fully operational. And now when you hear the sort of resistors saying, oh, This thing's only 90% accurate. What, 90% is pretty good. It's a lot better than a new employee that's, you know, is trying out and trying out and it's maybe 50% the first month, 60%. So I think we've got to, we've got to, we've got to change the way that we think about these things. But I think the cybersecurity risks are really just one of the biggest problems. The other problem is that it seems like a lot of these models tell us what we want to hear. So I, I, you know, my, my experience has been the, the, the best leaders are the ones that people that want to give them the bad news. They want to tell them what's really going on. They don't want to kind of paint the picture in a, in a, in a rosy way when it's not. So I, I think having AI make it look like things are going well when they're not is probably a big risk that we, we facing right now, uh, the environmental impact. What are we going to do with all these data centers the size of Manhattan? The energy costs are going up and the costs of infrastructure are going up. I think there'll be some innovation there too. I think with these constraints on leadership and management, what I suspect is going on right now, and I've seen this quite a bit, is we're going to see AI move into physical systems. The operations side of enterprises. And I think they will make organizations much more efficient, much more adaptable, much less sort of machine age kind of organizations where you've got a huge production line that, you know, you can't like, once you set it going, you can't turn it off. I think those days are changing. I mean, I've seen one company in contract manufacturing where they were able to use computer vision to completely automate their test and inspection function. And, you know, they had 1,000 people there. So I think the productivity impacts are big. And that leads me to the, I think the most important risk right now is that our organizational infrastructure, the way we think about organizational structure is at odds with the kind of speed and velocity that you can have with AI. And I think the C-suites and the boards need to increase their clock speed. And that isn't something that everybody can do.

That's okay. They'll use AI to do that. So it's, it's fine.

Hopefully.

I don't know.

I think that, you know, oftentimes we, we, you know, we think that everything runs in this sort of regular, normal way, but I think irregular operations, the way most people view the world, things don't work out the way you expect them to do. And people don't always do what you want them to do. And something you think is a certainty doesn't turn out to be that way because something changes. And so this ability to adapt on the fly, I think, is going to become probably the strongest capability that we're going to see in our future leaders, this ability to be nimble, to be dexterous, to be what we call super flexible. I think that's going to be the key going forward. But you have to have some foundation. You can't just be blowing in the wind as everything starts to come along. You've got to have a vision that's driving it. What, what, what's the core business we're in? What are we really trying to do? To do and how do we connect with customers? Uh, you know, we talked a lot about AI in, in, in services, but, you know, I'm seeing oftentimes I'm seeing marketing and sales organization just merge now. There's no need for the two to be separate. It's changing that kind of organizational structure. And again, I think it goes back to the, we're seeing structural innovation unfold organically right now. And I think we need to move away from the models of the past and figure out how, I mean, to go back to your earlier thesis, it's all about dynamic execution under extreme uncertainty. When we have, we've lost that stability that we had pre-COVID and the world is just very turbulent now. The VUCA world is upon us. So every day is a different chapter and we've got to be able to, we've got to be able to switch gears. You know what, when I, when I talk about innovation, I start off my classes with innovation and I put FAD, and they say, what's— it's a fad? No, no, innovation's real. I said, no, no, what do you think FAD stands for? And innovation is either forced most of the time, accidental some of the time, rarely deliberate. And I think that as we deploy AI at scale, we need to think about how to deliberately adapt and not be at the whims of what's going on outside. We have to have core capabilities to be able to do that.

I like this idea, right? We want to be deliberate. We want to have oversight, control, use it, optimize it. But do you think, given the lure of the value propositions, that many organizations have blinders on right now when it comes to those risks? They see how this can get you to the pot of gold better, faster, and cheaper. And they're not thinking about some of those risks. And specifically, you know, in my world, the privacy risks, the regulatory risks, the ethical risks, all those kinds of things. And we're just rushing. And that velocity we talked about is good from a business sense if it's successful, but really, really bad from a risk acceptance and awareness sense. Are you right?

Yeah, I agree with you. Earlier on before we were discussing like mergers and acquisitions, positions. And I, you know, you know what it's like when there's two— where there's, you know, the, the investment bankers have created like this capacity that, you know, merge together, we're going to be, you know, we're going to be ruling the world, two products are going to be this, that, and the other, let's get our customers kind of merged so we can do this. And security just takes a back door. It's like, you know, all of the, all of the—

nobody loves us, we're a barrier.

These checks and balances we put in place about, you know, are they, you know, is it, is it SOC compatible? All of this kind of stuff goes out the window. And I think to a certain extent, we're kind of lowering our threshold in these experiments that are going on with AI right now. We're working with startups that really haven't, you know, they're young people. And, you know, that's my advice to board members right now. Find somebody in their late 20s, early 30s that really has been playing with this stuff. For like 5 years and, and, and, and bring the maturity of business experience together with this new outlook on the world and forge that, that, that viewpoint. Because letting it go unbridled is not really an option. I think going at it aggressively is not an option either. So you're kind of caught in the middle of this conundrum. Is we know we've got to use it, we know we've got to move on. And I think that's what's happening in large organizations right now, people are really frightened that they're going to lose their jobs with good reason. And that there's a lot of people that, that, that if they don't adjust and become more productive, that's probably what's going to happen. I think that the employment numbers that came out, I haven't heard anybody say, well, isn't AI kind of partly responsible for this? So how are you going to adapt to that kind of world? I, you know, I honestly don't know, but I think that there's an increasingly strategic role that the CIO and the CISO are going to have to play in this new organization, Matthew. I think historically, you know, we've been viewed as a staff function that kind of slows things down, and rightly so. I mean, we, we know the risks are entertaining. We've seen it before, we've seen the movie, you know. And it's like, and if the people we work with don't have that kind of background, they're not aware of it. So I think what's happening now is the, the The leaders that I see being very effectively adapting to this new world are leaning way more heavily on their CIO and their CISO because they know at the end of the day, there's going to have to be some sort of experienced people that know how to create essentially a software-driven enterprise. That's what we're going to have at the end of the day. You know, we, you look at how quickly the software is being developed. I mean, most big companies had like an 18-month backlog of software projects. Boom, done. And done maybe not perfectly, but 70% good enough. When you didn't have it before, is it way better than not having it at all?

Not from a security perspective. That just introduced a whole bunch of vulnerabilities.

But if it's been done automatically, how do we know this? What's the kind of tests that we put on them? You know, I was talking to a chief operating officer of a big manufacturing company, and he was saying that, look, we have to speed things up and get cost out because, you know, the world's changing, we've got tariffs to deal with, all this kind of stuff. So I was sitting down with my people and I was thinking, well, what, what can we do to make things quicker and better? And it was like, well, you know, when we look at the, the list of all these components we use, 60% are pretty standard components. So why do we need to do 5-day burn-in of these things we've been using for a long time? It's the newer stuff that's coming in that we need to be subjecting it to more rigorous scrutiny. And I think that's, that's where things are going right now. There's another big issue going on, which you and I talked about before, which is kind of shifting, punctuating the equilibrium. In most big companies, they have a stack. They have systems of record. They have security guidelines. They have ways of doing things. What's happening in AI right now is all these vertical applications are taking a piece of those SaaS kind of things and unbundling. How do you trade off the value of a slice of that SaaS stack working way more effectively against the risks of unbundling it all and just letting it fall apart? And only the CISO and the CIO are really able to answer those questions. And that's why I think the strategic impact is, is really enhanced. And I think the role has a, not every CIO or CISO is going to be able to cope with this modern world, but the ones that are embracing it and can understand the old and the new together, I think are the ones that are going to make, because you're right to say that there are security implications, which people are just unaware of. And on the positive side, we can use AI for security more effectively too. You know, so it's not all downside. And I think we need to understand the risks and the upside can be, we can protect ourselves. We can deploy agents at points of vulnerability to scan what's going on. And that, that's probably where the best value is going to be using AI as an intelligence network to see where our points of vulnerability are and how that changes. You know, if you bring some software in, it opens up a whole set of other things. How does that vector sort of opening up change what we're doing? And I think nowhere is that better illustrated in operations. You've got machine-to-machine communication, which is, you know, you don't have the same people issues involved. But how do you, how do you make sure that there's no points of vulnerability there? If somebody can get access to you messing up your production line, they can cause serious damage, way more than just a privacy leak.

And unfortunately, there's always a lag for justification or viewing of security. If you've got new tools, new processes, whatnot, and it's making your productivity better and making money and reducing costs, that's what your focus is on, right? It goes back to the first axiom of cybersecurity. Cybersecurity isn't relevant until it fails. Exactly. And so, yeah, hey, we got this new AI system and it automates everything. That's great. Great and it's making you money until somebody takes it over and destroys everything because you've made a single point of failure. Oh, hey, we need security now. Well, okay, well now we need security tools. So those have to be developed and tested and integrated. And there's a lag there. So innovation's great, but there's a lag before security can catch up. So what are your predictions? Next 12 months, what are the undesirable risks that you see manifesting? In the next year, given the trajectory of AI, given the value and, you know, of, you know, and everybody's going to rush to it and potentially making more management decisions. What are those risks next 12 months?

I think the biggest risk is probably that we don't have the data to be able to take advantage of the models that are coming out. I think that people overlook the importance of data and securing that data. I think that there are some innovations coming up on using encrypted data to train models, which I think is a big issue because, you know, when you look at most enterprises, there's a point, even though everything's encrypted, that it's got to be decrypted to get outside of the firewall. So you're making yourself really vulnerable there. So I think there are some advances there, and I think that's probably one of the risks is people don't realize that they're exposing their data. I think increasingly models are going to be vulnerable. If we're depending upon, dependent upon these models, what happens if somebody, we get sort of, you know, people kind of messing with the model to change things around? How, who, who's going to understand that? Is the C-suite going to understand what's happened if the model parameters get changed or something, you know, pernicious is, is, is injected in there? I think that's, that's a big, big problem. And so dependency is going to be the biggest problem there. Once we get totally dependent, then we can't go back to the way it was before. So I think that's a big risk, is leaping into the new world and not being able to be put into reverse gear to go back to the way it was before if we need to. So I think there, there's a term some economists that have specialized this a long time called the irreversibility effect. I guess it's like Amazon's, you know, two-door, one-door kind of analogy. And I think we need to think more in the two-door so we can get back if we need to. I think that's probably the biggest risk of all is, you know, and I think the, the third one is losing our ability to value judgment for these sort of irregular activities that need to be dealt with. I think the models are fine in a probabilistic way, but there are, there are management skills that are required to, to make things work. You know, like in a negotiation, you've got to be sort of, you've got to be able to give and take. You've got to have some malleability. And so we need to trade off like tighter security against speed of movement. I mean, who makes that kind of trade-off? And I think we've got to make those risks pretty open and let people know what they're doing. And, you know, they're going to— the buzz around right now is really hard for people to keep their feet on the ground. Everybody gets super excited and it's going to be the next big thing. Really, if you think about the history of how we got to where we are today, you know, it started with a PC and then it moved to the network, then it moved to the mobile device and it It was like a nightmare bringing your own device to your company. That was a big conundrum that people were kind of grappling with for a couple of years. And then they realized, then we went to the cloud and, you know, well, we went to the internet, then that was kind of, you know, how do we keep websites safe and things like that? Then we went to the cloud and now we've got AI, which is kind of like the endpoint of, you know, where we started out with the mainframe. It's really how we automate administration. And so I think there's going to become a point in this next year where we'll see kind of the base administration system get kind of locked down. And I think there's going to be some failed experiments on the way with that, but I think that's where we're going to get to.

Yeah, I love the predictions, right? Data security, you've got model integrity, you've got over-dependence, which really comes into play, especially if something breaks. How do you recover from that if you don't understand how it works and what bad things have happened? And then still preserving that decision adaptability, that innovation, and making sure you've got that balance, that judgment balance for the trade-offs between security and productivity and innovation and all these things. I think it's a good list. Unfortunately, it's a really big broad, deep, powerful list just for the next year.

Totally.

Given that, and you know, a lot of the audience here are cybersecurity leaders, what should they be doing now to prepare for these challenges in the next year? Do they need to be adjusting their security programs or priorities? Do they need to be working more closely with the boards and the CEOs? Do they need to be working with the product teams to go, I want to be part of your next AI project? I don't want to find out at the end or when it breaks or when you turn it on. What should cybersecurity leaders do today to start addressing these issues in the next 12 months?

Well, as we talked about earlier on, I think they need to realize that we need a flexible posture. We need to be able to We can't predict what's going to happen. We have a core basis of kind of vulnerabilities that we can kind of work on, that we know about. There are things that we don't know about. And these unknowns, I think, are the things that require experimentation. So I think that there's a lot that's been said about the role of the CISO. Every quarter, the board meets and the audit committee has a CISO to show where they're vulnerable. Increasingly now, the data architecture, the systems architecture people are brought in. I think that role is expanded at the board. I think the CISO needs to be not just in the board meeting, it needs to be an active part of certainly the independent and the governance committee need to be involved in all that too. They can't just leave it. I think that When you're dealing with something like this, that's so impactful and yet so potentially harmful, as well as being potentially kind of cost-saving and looking at opportunities of moving too quickly, you've got to be able to experiment and play. So I think the one thing that I would urge cybersecurity leaders to do is to put aside maybe 5% of your budget and just experiment massively. Things that you don't know about. You've got to learn about things that you've got no prior knowledge of right now, and you can't just wade into those and make decisions without having some firsthand knowledge. You know, we spoke earlier on, Matthew, about the value of seeing things through your own eyes. I think cybersecurity leaders need to see things with their own eyes so that they can inform the leadership of the enterprise, the C-suite and the board, not just what they read about in Fortune magazine or pitches by consulting companies that want to come in there with a team of 500 data scientists, so they're going to like revolutionize the way that you do manage the company. But the cybersecurity leaders need to be able to give the benefit of their experience to leaders in a way that they can see what's going on, they understand it because historically we've always been viewed as naysayers and, you know, the last people you want as a security people because they'll put a stop to everything. We need to shift away. We need to be enablers. We need to say, okay, like you said, we want to be in not at the end when something's finished, but at the beginning when we can have, it's like quality. You know, a friend of mine was involved in creating the first offshore manufacturing looking at really high-quality production. And I said to, you know, and he said, you know, quality's got to be in at the ab initio. It's not something that you fix afterwards. And I think security is the same thing. I think cybersecurity particularly needs to be put in at the beginning of every activity, not at the end. And I think that the smart leaders understand the need to pay for insurance. And cybersecurity is really the insurer's protection to enable these risky decisions that could have huge payoff to be made. But, you know, we talked about Napoleon earlier on, and Napoleon, um, had wrote, wrote quite a bit, and he had his son as part of the army. And one of the pieces of advice that he gave his son, he said, Without strategical security, there's strategic surprise. And I think that is something that we all need to keep in mind going forward. We need to avoid strategic surprises, and that's what the cybersecurity experts can assist and help.

I love it, right? Having that strategic mindset. What, what can leaders do now realizing that The velocity of technology change and the rapid adoption from a business perspective changes our environment. And we cannot be entrenched in the way we've always done things and think that we're going to continue to be successful. We have to adapt. We have to learn. We have to intentionally set aside that time, that resources to become experts in areas that we wouldn't normally be, to learn more about AI, to be more in tune. With the business changes that are coming and then be that advocate, right? Yeah. You know, and we have to work with the CEO and the boards to be able to advocate for the security as you're advocating for the business benefits as well. We can't be the department of no anymore. We can't. We have to be a business partner. Yes, let's go after this. Yes, let's adopt these AIs. And let's make sure they don't bite us in the backside, right? Let's put those controls in place, but having those discussions early on.

And I think you've hit on, I think, probably the key inhibitor of people embracing cybersecurity is that we've always been viewed as the naysayers. And I think that part of that is because of the direct approach that we have. One of the things about flexibility that I think is really important that it It amplifies the strategy of the indirect approach. I think cybersecurity experts don't need to be that stop-before-you-do-it kind of approach. I think they need to be able to show people with the evidence of their own eyes how it's better to do things this way, how it's less risky, how while we might be paying, might be hedging our bets in terms of paying a premium for that, The value of that is immensely greater than putting everything into one thing and it failing. You need to be able to do that. And I think that's the kind of new mindset that we're moving into. It's the topic of a paper we're writing right now called Adapt or Perish. I think it's all about adaptation.

Yeah, it's, you know, nature, right? It's, it's, you see it in nature and you see it in business. Those who don't adapt, they're not going to survive. So Stuart, thank you so much. Distinguished Service Professor at Carnegie Mellon University. Again, thank you for sharing your cybersecurity insights with us. Very much appreciated, sir.

Thank you, Matthew. It's been a pleasure as always to talk. And I think we've explored ideas that were certainly unscripted. And I think that the more conversations people have, the more people realize that this is a new world we need to embrace and it's gonna change things in ways that we can't even begin to imagine, right? Now, but the upsides are way, way better, and we need to embrace those wholeheartedly. So thank you. It's been a pleasure and a privilege to join you today, Matthew. I wish you well going forward. And again, thank you very much for having me.

And thank you all for watching. Be sure to subscribe and catch all the Cybersecurity Vault episodes where we chat with industry leaders to dive into the most relevant and interesting cybersecurity challenges, perspectives, and best practices. We'll see you next time.