AI as a Business Disruptor: What Your Board Needs to Know (and Do) on AI

Next, I'd love to introduce our next moderator leading the panel, AI as a business disruptor. And our panelists, if you can come to the stage as you already are. Michael Nisit is a partner at Hedrick and Struggles in their Boston office and his partner and is part of the Global Technology Practice Practices leadership team conducting CEO, coo, President and Board Director searches for technology and technology enabled companies. He also leads many of the firm's global client relationships. Michael, take it away.

Ah, great.

Actually, I'm going to take this. So first of all, thank you on behalf of myself and the panelists for inviting us to be here today and for the online audience that's watching. Hopefully they're still engaged and participating. We're honored to be here and I think the Institute did a really interesting job putting together this mix of people for what we think is a really interesting topic. A little bit of a change in tone from what we've been hearing throughout the day here in the sessions that we've been in, there's a lot of excitement, there's a lot of enthusiasm for what is AI, what can IT do, What are we doing, where can it go? I mean, that's all really kind of positive, exciting things. We're going to put a little bit of a different slant on that today because we're talking about the impact of AI in corporate boardrooms, what they know, what they don't know. And there's a lot of the latter, not much of the former. So the panelists that we have here today, Linda Avery, who spent the last nine years of her career as the Chief Data Officer in two very different organizations. First at the Federal Reserve bank of New York, where she stood up the inaugural data sciences team, deploying the use of AI ML and economics, management of the financial markets and the supervision of financial firms. She, she then went on to Verizon, serving as the first Chief Data and Analytics Officer in the organization globally, where she built a team of 1,000 person AI and data scientists that in 22 delivered over $3 billion of value back to Verizon through the efforts of her, her team and the solutions they put together. Her previous roles include 20 years at Goldman Sachs where she was Managing Director in technology, driving the integration of cutting edge technologies across their capital markets, business businesses. And she and her husband sail out of New York City. Yes, David Kremer, who many of you probably know currently the Dutton Family Dean, Damore McKim School of Business and professor of Management and Technology at Northeastern University. Couple cheers in the audience for him there you go and a Fellow at Cambridge Judge business school in St Edmund's College, Cambridge University. Before moving to Boston, he was Provost, Chair and Professor in Management at National University of Singapore and the KPMG Endowed professor in Management Studies at Cambridge University. He's the founder and Director of the center on AI Technology for Humankind at NUS Business School, which was hailed by the Higher Education Times as an example of interdisciplinary approaches to AI challenges in society. He's one of the most prolific behavioral scientists in his generation, a recognized global thought leader by thinkers 50 and a best selling author including Leadership by Algorithm who Leads and who Follows in the AI era and you have a new book coming out on Amazon in a couple of weeks. No Mid 2024 okay, the AI Savvy Leader 9 Ways to Take Back Control and Make AI Work, which will be published by Harvard Business Review. We also have an experienced city and CEO with us today. Steve Johnson is a seasoned executive with a track record of building innovative technologies into successful enterprises. Steve started his career by founding the Johnson Grace Company with compression technology that he had patented enabling fast, easy distribution of images, sound and video over narrowband dial up. Going back to a couple of those 1990 videos that we just saw from Michel Johnson Grace was purchased by aol where Steve was VP of Technology Development, responsible for continuing to bring in leading edge media, telephony and broadcast products to the market. Over the last two decades he's founded several technology companies. Most recently he is CEO of Notable Systems, an AI based document solutions company. Steve is also a Fellow at the Belfer center at Harvard's Kennedy School, writing about Internet regulation and a Senior Fellow and Advisor at Northeastern Institute of eai. My background, as I was just previously introduced, I'm a Senior partner with Heidrick and Struggles. We're a global executive search firm, about 2,000 employees around the world operating in the major markets around the world. We recruit board directors, CEOs and chief CxO. Kind of the direct reports into CEOs across all industries and across all functions. I'm based out of New York and I live in Cleveland. Just one couple of comments that I wanted to make before we dive in with our questions as the panel and I have had a number of discussions over the last several days. There's really some interesting kind of thinking and energy between the group here as we think about and I deal with corporate boards every day. My job, my responsibility is boards come to me, typically Fortune 1000 boards and call it Mid Cap, Large Cap Private equity owned boards. They come to me for Board directors and I work closely with them on board composition, board experience, and then of course, board recruitment where there are experiential gaps that we're looking to fill. Experiential meaning what they've done in their career, not experiential AI. And what's interesting is that the boards today that I'm dealing with, I would say 80% of them aren't engaged in this topic at all. And when I say at all, I mean at all. There was a slide that Michelle had put up that evidence kind of where they're at at McDonald's with the crawl walk run. I think what I would say about corporate boards today and my experience with them, working very closely with them, is that we're probably still in the cradle. I think we'll be crawling. Boards will be crawling next year because there's probably going to be some things that are going to really rock their world in terms of attention getting. But there's a lot of work to do at the corporate board level. We're going to talk about that today. So there's just one other thing I wanted to share and we as a panel talked about it. Deloitte just recently published a study in August where they had surveyed a couple of hundred of the top public company boards asking about their knowledge of engagement with AI at the board level in the corporate boardroom. And a couple of interesting statistics from that. The frequency of AI related topics on board agendas. 86%. None. Not on the board agenda. Does your company permit the use of AI by employees? 74% of companies today don't have an AI policy. So it's kind of the Wild west. Employees across organizations are being allowed to do what they want to do. And that's where I think maybe in a year we're going to find some things that are going to require some attention. And lastly, does your company provide training and education in AI? 83%. No. No training, no education in AI right now. So there's a lot to be done and we're going to talk about that today with the panel and maybe a good place to start, Linda, given your really remarkable experiences as a chief data and analytics officer and some very large, very sophisticated organizations. Tell us how prepared are organizations to engage and leverage AI as a business disruptor today and what challenges do you see on the radar screen for the boards?

So Michael, I think that survey that you quoted from, excuse me, and you had the courtesy of sharing it with us before and many, many frightening statistics in that. I think that is just a leading Indicator in terms of the lack of preparedness right now in organizations. But I'm going to speak from my nine years as a CDAO in terms of the challenges that I've seen, and I'm going to start very much with the organizational challenges. To be a successful chief Data officer, you have to be one of the nosiest people in the world. And I'm not kidding you, because what you have to do is that you basically need to put a roadmap together that is basically mapping out the most important decisions that a company is making. And you're trying to figure out how you can optimize those decisions. And that's at the core, that's how you drive value, is the optimization. And so to find those decisions, whether they be macro decisions around business strategy, capital expenditure, supply chain or micro decisions like, you know, around customer experience, what you're really looking at here is you're putting your nose in other people's business and they have to be willing to open up to you. And without the accountability back to the board and without a tremendous amount of interest coming down from the CEO, you're going to find that's very, very difficult to do. You need to have the support of the board and you need to have the support of the CEO. I believe to make any real progress organizationally, technically speaking, lots of challenges there. If you think about the story of data, it has ended up everywhere. You're going to find data in heaps like refuse coming out of it applications. And now you are basically faced with all this demand and interest coming at you and you are going to have to bring together what I call a team of data tamers. You think about data taming, what does it involve? Involves a lot of different specializations. You've got data governance, data architecture, data engineering. Now you get on the other side of the equation. You need to have of course your data scientists, but you also need to have your ML engineers, your platform engineers. And I will guarantee you along the way you're going to find that you need to stand up a cloud strategy so you can actually include a cloud migration in terms of what is required. The path from razzle dazzle experiments with people sitting around a conference room table oohing and ahhing about what is possible to actually having industrial strength, reliable, good AI. It is a big journey. And so I think boards need to understand that what you're really doing here is that you're building out what I've called a data factory and an AI factory in all of their glory. And so I Think there's a lot of learning that needs to happen in terms of what the actual path to value to get that value out of AI is really about.

And building on that, Steven, you're working with clients every day with Notable Systems and delivering AI solutions into those. As you're engaging with your clients, what's your sense as to their level of preparedness? Do they know what they know? Do they know what they don't know?

So my company, Notable Systems extracts data from documents and we use a whole variety of techniques, including AI techniques. We're starting to use large language models to do things like concept extraction and even second opinion auditing on the results that we generate. And so I like to give kind of of the advice that I would give to a board is the same that I give to my customers. Really. Three pieces of advice. First, don't trust AI too much. One of the things I don't like about the term artificial intelligence is it's very intimidating and people don't ask enough questions about it. It sometimes gets more reverence than it deserves. And you know, I think that.

It'S.

Good habit to swap the word software for AI as a start. And if you're a boarder, if you're a customer, ask all the same questions about the AI system that you would about a software system. And you don't want to just plug it in and then everyone goes home or everyone goes on to another task. You need to treat it like a software system and you need to test it. So don't trust, test and verify and keep verifying and over time, keep verifying. The thing I like about the term artificial intelligence is it signals that there's something different going on. So it's not just software, it is new. Some of the capabilities we've been talking about all day are brand new capabilities. One of them, and this is a capability that we're developing in my company, is the capability of identifying what the system is not certain of. All AI systems make mistakes. If you leave it up to the customer to find the mistakes, you're doing your customer a disservice. When it comes to our job, which is to find information in documents and transfer it to a computer system, it could. Unless you're doing a nearly perfect job, it could actually take the customer more time to find your mistakes than it would have taken them if they just hunted for the information to begin with. So all AI systems, this goes to the board and this goes to anyone shopping for AI. Ask the vendor, the supplier, can your system tell me where the vulnerabilities are where the weaknesses are. And in particular specifically in my case is which pieces of information aren't you so sure of? And that is something that when you see AI is a legitimate question to ask. Software in the past may not have been so good at identifying the level of certainty that it was doing its job, with which it was doing its job. AI systems, you can ask for that. The other thing about AI systems, this is all under piece of advice number one, by the way. The other two are a little shorter. The other thing that AI systems can do is learn. So when the client or the vendor fixes the problem, whether it's an error or whether it's we made the wrong choice, we need to adjust the trade off calculator. We need to make sure you should. When you're shopping for AI, when you're on the board and you're asking questions of the CEO of your company, are you closing the loop? So the fixing of the problem needs to be fed back into the system so the system learns and that's key. That's what AI should do. And if it doesn't do that, then the vendor is missing something. Okay, so the second piece of advice is don't mistrust AI too much. There is something really remarkable going on and most of my customers had given up looking for a technology like mine until we knocked on their door and said, hey, it really works. Now they have been doing moving information from documents to computers the old fashioned way for decades. So it is time to go shopping for this new thing that's happening. And it's not just generative AI. There are all sorts of techniques that are maturing. Finally, the third piece of advice is devote some resources to this. Don't just board or CEO or the IT department of any company, don't just think. You could just read about it in magazines. Really, it's time to hire people that know about AI or make sure that your staff learns about AI so that they can ask the questions that need to be asked. Great.

Which is a great segue into a question for you, David, given your experience in the Institute, working with enterprise clients that come to the Institute to engage with you and with the team on solutions, things that they're trying to solve every day. Tell us a little bit about what you're observing from the board's perspective. They've got to understand there has to be a basic knowledge because they're going to be called upon to make decisions and approve budgets. In some cases, pretty big budgets for AI projects. Do they have that level of knowledge today from what you're seeing. And what do they need to do to be prepared?

Well, like I said, from my experience as a business school dean and also business school professional professor who's doing consultancy with these companies. So, in fact, yesterday evening I was advising a board again, and the first question was, can you just explain what AI is? Oh, a computer acting in ways that seems intelligent. Oh, okay, so it can learn, like Steve said. Yes. Okay, it can learn. What is it that we need to do? So that's the level that I run into quite a lot. And I think there's a big change happened in the last 10, 15, 15 years where business schools usually when working with board CEOs, we focus on, look, you don't need to write the strategy or you don't need to lead the company, but you need to make sure that, you know, do they have the right kind of leadership and do they have the right kind of strategy? That's your job.

Yeah.

So today that's been extended by also looking at what are your obligations Now AI is being integrated in the organization that you're evaluating, governing. So, yeah, you need to know about AI to a certain level to understand how you evaluate a budget, for example, because that's where I see a lot of mistakes happening to say so, because most companies, as you probably know as well, like when we talk about AI adoption, most people think right away, oh, AI adoption is automation because we're replacing people in tasks. They're not thinking about augmentation. So I see most often as a consultant that 70 to even 90% of technology budgets go to AI adoption, leaving 30 to 10% for other things. And what are these other things, you may say? And a board has those questions as well because they think that looks good. Because isn't it? We plug it in, immediate results. We have AI. So you have an operations culture. Work is done. No, it isn't. Because there's another problem. Most companies don't upscale AI. They can't reveal value creation to the extent that a board should help them to receive. So if a board does no AI sevenness, how are they going to help you out in deciding this is the right kind of budget? And if not, if they don't know about AI, they don't see the issue of augmentation requires more than simply buying the technology. So when I tell them, hey, you need to push down the technology budget and you need to increase the budget for job enrichment and more support of making sure your leaders are being trained to create the right conditions so people will work with AI and will leverage it so that you can augment meaning enhancing human intelligence and performance. Oh, then they're looking at. What do you mean? But that was in the past. We were looking at that, but now it's AI. No, again this shows where they need to know and understand AI in terms of its limitations and its potential. Just one example, I mean some of you may have seen last week a new paper from Harvard came out on testing ChatGPT for Boston consultancy and it showed like whoa, ChatGPT makes those consultants so much more productive.

Great.

So boards are also referring to this now Yesterday again they referred to the article and said, well, I mean it helps. I mean, why are we even investing? Why do you say push it down? I'm not going to push it down. No people budget said no. But there's a problem there. The study shows we make consultants more productive, meaning they can produce more PPDs. Consultancy is not about productivity, it's about de risking the situation and making sure that you can create that value. So the study says nothing basically because if you don't have the climate, you don't have the leadership to empower in the people management. AI is not going to be used carefully. And so those are that understanding. Maybe it's even more holistic point of view, even on AI that's completely missing in my view.

I mean consulting is a great, I mean it's a great example. It's one of the areas sectors that is really being impacted by this. I happen to be in Boston today. We had a global partners meeting the last two days, a couple hundred partners from around the world that we're in and we had the, the pleasure of having the CEO of Bain come in and do a talk with us for about an hour yesterday afternoon. One of the interesting comments that he made, and he made it in a public setting, so I'm going to go with it Here they have a Genai practice that they're leading that they started in March of this year and it's not even the end of October. And that practice area has gone from zero to, I'll just round about it, several hundred million dollars in six, seven months. I mean just remarkable what it can do in some of these sectors, consulting and professional services in particular. Linda, building on that, I mean from your experiences as a chief data officer, chief analytics officer or running technology organizations where you're routinely going to the CEO and to the board asking for major capital commitments and investments, millions of dollars, tens of millions of dollars, these are Going to be big checks that have to be written. So what does a, what does a board. What does a CEO and what does a board need to know in order to be able to make these decisions?

Well, we're kind of lurking around this topic, but that the necessary controls are going to be in place. That's absolutely huge. And there need to be basically estimates in terms of the return on investment. So these are things that you need to be able to figure out a priority. It's a matter of controls and it's a matter of opportunity and that there's a healthy roadmap. There's going to need to be investments in infrastructure, there's going to need to be investments in, in responsible AI. And how do you know that you are not going to be in the headlines at any given moment? Because you don't have the right testing and controls. And so I think one of the key things that needs to happen is that there has to be a control mindset. And I don't necessarily see that as developing as yet. This is not, you know, these are the glory days right now. Everybody is a bit naive in terms of what's possible. And yet there's so much risk, you know, if this new capability is not appropriately tested and not appropriately controlled. And the investments have to go there.

And on that point, the board has many responsibilities. A corporate board, let's say a public company board in particular. But hiring and firing the CEO is number one. That's job number one for board. Number two is strategy, making certain that the team is coming up with the right strategy, they're presenting it, and the board signs off on that. Number three is risk. Yeah, and risk management. And something like Gen AI. In an organization without policies, without training and education, where people are just freewheeling it, that creates enormous enterprise risk. And so for those of you in the audience that have startups that are focused on this area, you're developing products and solutions that you're bringing to your enterprise customers. You may be bringing it into a level in the organization, in the sales and marketing team, in the finance team, wherever it might naturally sit. But understand where you're selling it. In an organization at a manager, director, VP level, where they're dabbling, maybe going rogue, maybe doing it sanctioned, but they're dabbling. Understand that your solution is ultimately or can ultimately get visibility by audit, by compliance and by boards and help your customers out. Right? Bring them the solution that's going to do whatever cool thing it is that you want it to do. But also understand that maybe Even if they're not asking about how it's going to manage or how you mitigate operational risk, financial risk, compliance risk. Like be proactive and answer those questions for them. Because at some point in time, whoever you're dealing with in a company is probably going to get asked that question. It'd be nice if they had an answer in advance.

Absolutely.

Steven, from your experience, what are the fail points? Where is AI going to break first in an enterprise? And how can boards monitor it to identify it and then mitigate it if and when it pops up?

I'd say two things about that. First, make sure. I think I heard this mentioned on a previous panel. I don't think it was David Bowie, but David Bowie I think pretty much covered the terrain, which is make sure you're not just trying to figure out a place for this solution to go. Make sure you've got the problem that it solves. So there's a fail point. This cool new technology came out not being sufficiently, as the board, not being sufficiently skeptical or questioning of the urge to use it, make sure it solves a business problem. The other thing, just sort of more into the interior of the technology. You start lashing LLMs into whatever application it is and the vendor changes the model, then you haven't tested it. And so you need to make sure that the vendor flags any change like that. Because actually, if you follow the rules and you test your system in its first version, it if it's ChatGPT 4.0 and then it needs to revert to something else or the company loses a legal challenge and has to subtract a model from it, suddenly your system doesn't work as tested. So make sure that the vendor flags things like that. And it's a very unusual type of issue in software development, but that is an issue today.

And building on that, David, and the clients, the enterprise clients, the businesses that you're working with, the boards, what's your sense from what you're engaging with and observing, how can they identify the risks and mitigate them? Because they're going to come.

Yeah, obviously, like, I mean, as everyone has said already, it is a very important task, responsibility of a board to do the risk management, because it's governance. So, and one of the things that I'm really interested in, when I look at that governance aspect, when AI enters the equation and they're not really savvy enough to understand, I'm holding my heart then and saying, like, wow, you're evaluating the risk here, not being AI savvy enough to understand what AI means here is that I see a lot of boards moving into a very techno solutionism mindset. Oh, you have AI. AI can actually make governance decisions because you have literally boards today where AI becomes part of the board. So they seem to accept that to some extent. But why is this important to realize is that at the end of the day, again, they still need to evaluate whether the strategy of the organization that they're scrutinizing that they have to be critical about, is that company acting in ways that serves our stakeholders. And in my book that's like, is it responsible? Is it ethical? And if you don't understand ethics in terms of AI, we have another problem here when it comes down to risk evaluation. Because I mean, let's take a look at, for example, Microsoft and Google and their. It's an arms race. When they brought down their large language models, they jump on it. It's not a purpose driven enterprise, it's a profit driven enterprise. I mean, this relates very much to the discussion where people say, oh, you know, we're going too fast with AI. I always say, no, we're not going too fast. Development of knowledge about AI is not too fast. As long as you don't commercialize it, because that's the thing, then purpose becomes profit and then that's a threat in itself. And if boards don't really realize that, look, okay, you need to be careful. Look at Apple probably has a completely different board looking at it differently and saying like, hey, hey, hey, privacy is important. Google and Microsoft left it behind. Their board missed some points there as well. In terms of this is going to harm your reputation. You're moving too fast. This is the risk because it's related to ethical, reputational risks. And if they don't understand that, if they think AI will take care of that, yeah, we have a big problem.

We'll close here in a minute. We may take one question, but just a thought to share. Spent the last 25 years of my career in executive search, recruiting CEOs and board directors. I guarantee you, if you were on a board of directors, you would not want me on your audit committee. There's no. I have no business being on an audit committee. I could serve other purposes. Not that one. Years ago, when, when cyber security, let's say 10 years ago, when that was starting to bubble up and become more and more of an issue. Linda, you saw this in your organizations. You know, you had like a manager or director level of security, network security, and all of a sudden it became a VP of Information Security. And all of a sudden, five years ago, it became a ciso, Chief Information Security Officer. And that position, because of the importance of what was going on, it got moved up and now there's Chief Information Security Officer, some of which, not all, but some of which report directly to CEOs. What we're going to find in, in AI is the same thing. We've got, you know, organizations that are developing right now, but the responsibility, the awesome responsibility for managing this and the risk and the opportunities inside of an organization, the, the executive that's going to be responsible for this in an enterprise, boy, the, the positions are going to grow pretty quickly. And you had mentioned like a Chief AI officer or yeah, ethical AI officer. Like, like when she said that, I was thinking, okay, like 12 months from now, I'm going to start getting phone calls for them. By the way, they pay really well. We've got a, we have an AI practice that Ryan Bukowski, who Usama knows really well, he leads it. He was giving a presentation that our partners meeting yesterday and he was talking about, you know, some of the amounts of money, I mean, individual contributors in organizations, typically PhDs, with some, some outside corporate experience already. I mean, two, three million dollars a year for individual contributors. So it's pretty remarkable what's going on. What does that mean for the boards? Because nobody's going to put a chief AI officer on a board, just like nobody wants a CISO on a board. Right. Unless you're like a security software company. I've actually done a few of those. But we're going to have to figure out a way that boards get educated and that somehow we, we bolt onto them the needed experience and capability so that they can manage the risk and hopefully unlock some of these opportunities. Do we have time for a question?

We have time for one question.

Who's the lucky person all the way in the back? No, we'll get you a mic.

We'll get you a mic. They won't hear you online.

Okay. Can you hear me? Yes. So specifically to your last point about.

We need to bolt on expertise.

Do you think that that expertise exists at the scale that we need in the demand at this point? No.

It's terrifying.

The talent pool is very thin at the board level, CXO level. It's very thin. We're gonna have to put it together in different ways. Like executives like Linda that have, growing up through data and analytics, big operational, big technology roles, that's had experience in and around machine learning that now look, I mean, how old is AI Right. It's like two or three years old. We're not going to find somebody. If the spec for a position is, I need somebody with 10 years of experience in AI, that's a null set. So the talent pool isn't there. It's going to be developing very quickly. We see people moving over from kind of other functions and other functions just kind of morphing into taking on responsibility for AI. So I guess all that's good news for me.

Michael. I'd also add that that was a pretty good commercial message for the Institute. The Institute is training people that will enter the workforce knowing a lot more about practical use of AI. So that's really important.

And what the Institute is doing with the Solutions group, engaging teams of people, teams of students with organizations to solve real problems that organizations are trying to solve. Look, I had Usama on the. On a video conference with the Chief Information officer of Heidrick and Struggles last week. Like, we want to engage with one of those teams. When I heard about the capability, I know from all of my clients how scarce it is. It was something that we wanted to latch onto right away.

Well, thank you.

Thank you panelists so much. Absolutely. Thank you, everybody.