#170 What Fortune 1000 Executives Believe about Data & AI in 2024 with Randy Bean, Innovation Fellow, Data Strategy, Wavestone
4 Jan 2024 · 46 min
Randy Bean presents findings from Wavestone's 12th annual survey of Fortune 1000 executives. Two-thirds now list generative AI as a top priority, 89.6% are increasing investment, but only 5% have deployed at scale. The biggest barrier isn't technology: 77.6% cite cultural and skill gaps as the main obstacle.
Chapters
Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com.
The question of the discussion was artificial general intelligence, this notion where basically computers could perform any cognitive task better than a human being. Is this like five years out or seven years out? And the answer was last month we thought it was like five or three months ago we thought it was five to seven years out. Last month we thought it was like three years out. We think it may be in the lab today.
Welcome to Data Framed.
This is Ritchie.
It's very easy for managers to become solipsistic. That is, it's all too common to become focused on internal company issues and not think about what your competition might be doing. You know you're making progress, but you aren't quite sure if you're progressing fast enough. That data transformation program seems to have been going on a long time, but you don't seem to have made money from it yet. Does everyone else really have their generative AI projects in production already to clear your mind? Today we're going to talk about the results of the Wavestone Data and AI Leadership Executive Survey. This is one of the longest running surveys of Chief Data officers and other blue chip executives so you'll get a chance to hear what other companies are thinking and doing about data and AI. Our guest is Randy Bean, the creator of the survey. Randy is the Innovation Fellow for Data Strategy at Wavestone and he was previously the founder and CEO of NuVantage Partners before Wavestone acquired it. He's the best selling author of Fail Fast, Learn Faster and he regularly writes in the Harvard Business Review, forbes, and the MIT Sloan Management Review. He's also incredibly well connected amongst Fortune 1000 executives, so hopefully we'll get some personal insights out of him as well.
As the latest survey results.
Let's find out what all these Chief Data Officers are up to.
Hi Randy, thank you for joining me on the show.
Hi Richie, how are you today?
Doing all right, thank you. I'd love to dive straight in. Can you just explain to me what Are the goals of your data and AI leadership executive survey and who gets surveyed?
Yeah, so this is the 12th year that the survey has been conducted. It first started in 2012, and at that time it was at the behest of a series of Fortune 1000 data and CIO executives. And they basically wanted to understand data had reached that point. If you remember back the term big data had started to come into use and data had been elevated from something that was hidden off in the corners of the organization to now something that was a C suite concern. So these executives want to understand whether basically it was just their organization or whether it was pretty common across the industry. So we start conducting the survey. The survey is aimed at Fortune 1000c executives, for example, to be precise. I'll just look up the numbers here. This year, 89.8% of the respondents held the role of chief data officer. Chief data and analytics officer are basically the equivalent role in their organization, like head of data and analytics. And of the respondents, roughly 94% were C Suite executives. So that's the audience, that's the respondents, and it's four to 1,000 companies and global leaders.
Okay, so it's a pretty comprehensive survey of all these sort of leaders of big companies. I'd say when you said like 2012 was when sort of big data hit the mainstream and data grew popular, I suppose it seems like a long time ago, but it's a decade. I guess the change this year has been generative AI has hit the mainstream. So do you want to tell me how that's changed things?
It's really changed everything. And in the forward to the survey, which is co authored by Tom Davenport and myself, basically what we say is generative AI has transformed everything. It's really elevated the whole discussion, the whole urgency around the importance of data and analytics in organizations. We dedicated a major portion of the survey to generative AI and I could share some of the data. For example, we asked, are investments in generative AI a top organizational priority? And roughly 2/3 said that it was, which is pretty good considering it's only been a year. Is your firm increasing investment in generative AI? 89.6% said that they were. Do you believe generative AI has the potential to be the most transformative technology in a generation? Roughly 2/3 the answer that it was. So it's, it's definitely getting C suite and board level attention. Here's a good one. Is there a need for safeguards and guardrails for governing generative AI? 99.0% said yes. I'm wondering who the 1% was. Oh, no, just like, let it rip. And then we asked, you know, are the safeguards and guardrails for generative AI in place in your organization today? 62.9% said yes. There's a pretty good gap. And some people, initially I've shared this finding and some people said it's not Even close to 62.9% of people either lying or overly optimistic. And then we asked, is the talent in place to responsibly implement generative AI? And this is a nice answer because 50.5% said yes and 49.5% said no. So clearly, generative AI is top of mind. And the other thing that I'd add is, is each year we ask a number of questions about the progress of data and analytics within your organization. For the first time in the past five or six years, the numbers jumped pretty significantly. And we attribute it to generative AI and the elevated attention that all data and analytics initiatives are receiving now.
That's pretty impressive. Just to shift just from 2022 to 2023. And one thing I'm wondering is, basically everyone has said, yeah, generative AI is really important. We're thinking about it. It's top of mind. I'm not sure who's or how many people are actually implementing things yet, like who's starting working on this.
So we asked about the state of generative AI implementation efforts. 60.4% said that they were in the experimentation and testing stage. 24.5 said that they had implemented in limited production. So a case here, a case there. 6.6 said they were in the planning and design stage. So the earliest stage, only 4.7%, so less than 5% said they had implemented in production at scale. So that's the highest. And then 3.8% said they're doing absolutely nothing at all.
Okay, so two extremes, but it seems like the prototyping area is maybe the most common thing. So people are toying around with what's possible at the moment, but not necessarily doing things at scale yet.
Yeah, a lot of talk, a lot of experimentation. But I published an article in Forbes about Ally Financial and they released some metrics, quantitative metrics, about how they're using generative AI to increase the productivity of a series of their customer service tasks. And I wrote about the 66% improvement, 88% improvement in various efficiencies that they're getting. So. So they went public very quickly with some of the results to show the industry what they're doing. And they're purely a online bank. So they're innovative by nature. But it was really some of the first data that I'd seen that organizations were quantifying or at least sharing their quantified results.
That's pretty impressive that they're already. They've made that progress just in the last few months. So you gave an example of an online bank. Are there any areas or projects that you see a lot of organizations investing in around data and AI?
At the moment, I see most of them focused around productivity improvement. And we did ask the question about primary business opportunity Created by Generative AI. And 49.1%, just about half said achieve exponential productivity gains. The other two areas were liberate knowledge workers from mundane tasks. And that was another thing that Ally Financial was that they could free a lot of their workers to do higher order type of activities, more creative or more synthesis type of activities. And then 22.6% said improve customer service and experience. And then 4.7% said other. I'm not sure what the other are, but a lot of the activities around customer service, particularly in terms of activities that are pretty standard in terms of communication with the customer and how they can be improved and made more efficient and streamlined. Okay.
I mean, I like those three areas. They seem pretty sensible. So improve your productivity, automate boring stuff, and then give a better experience for your customers.
Yeah. And by the way, I happened to go to this event, I live in Boston at Harvard University in October, and it was with leaders from Harvard Medical School. And they said that they were experimenting with generative AI and they had two pieces of data to share. And I, as they were sharing them, I texted this data to my wife, who was in the healthcare industry. So the first one was that it said that generative AI did a better job of diagnosing patient illnesses than Doctors did in 90% of cases. And they related an instance where there was a certain set of symptoms and the person went to 10 different doctors and there was slightly different diagnosis. And then they asked them generative AI and of course produced a result. And I went back to those 10 doctors and they said, ah, you know, actually that's the correct diagnosis. And so I texted that to my wife and she goes, yep, makes sense. And then the second thing was they said that in 80% of the time, generative AI was reported by the patients as being more empathetic than the doctors, which I thought was amazing. And I sent that to my wife and she said, yep, definitely. And a week later I was hosting a Chief Data Officer panel. And I had the Chief Data Officer from Mayo Clinic. And so I said, ah, you know, I basically surprised him on stage and said, hey, you know, what about these results from this study at Harvard Medical School? He said, absolutely. He said, that's exactly what we see in terms of efficiency of diagnosis and empathy of the doctors.
I have to say the empathy one surprised me a lot. But now I think about it, yeah, you can, you can set the tone of voice for these generative AI tools. Yeah, it can be consistently nice. So one thing that we talked about on a lot on this podcast in the last year is the idea that in the near term it's probably going to be humans and AI working together. But in this case you were saying that actually AI is just outperforming humans a lot of the time. So do you see that as being AI replacing humans rather than augmenting them?
Probably. The key thing is that I was at an event in Boston last week, a Chief Data Officer Summit, and organized the kickoff keynote panel for Chief Data Officers. And the second day was Chief AI Officers. It was billed as the first event bringing together Chief AI Officers. And it was held at Northeastern University at a new center they've established, the Institute for Experiential AI. And was speaking with the director of that institute yesterday. And the main. And he's done. He has his PhD, he's been doing work in the space for 30 years. He led AI research for Microsoft for a number of years. And at Yahoo, he said the single most important thing is the human in the loop. Because he said that basically, and this is interesting, so he's a leading international expert and he said it was really that contextual human experience that could guide the AI process. Now in October, I went to the Wall Street Journal Tech Live event, which was interesting because it's an invitation only event and people like Vinod Khosler and others. But the keynote speaker was Sam Altman. So he's there, there's about a hundred people and he shared some things. A week later he was fired. Two weeks later he was rehabbing it. So I had written about the event and all of a sudden it became like viral. But what he was talking about was artificial general intelligence, this notion where basically computers could perform any cognitive task better than a human being. And the question of the discussion was, is this like five years out or seven years out? And the answer was last month we thought it was like five, three months ago we thought it was five to seven years out. Last month we thought it was like Three years out. We think it may be in the lab today. So again, there's that importance of the human in the loop to guide the process. But there's also this notion of artificial general intelligence where a machine can perform any cognitive task better than a human being.
That would be an absolutely revolutionary occurrence if we do get to that point, and especially, like you said, could be in the lab already. That's a fascinating and also mildly terrifying thought. Before we carry on with the artificial general intelligence, let's get back to the survey. So we're talking a bit about what organizations are actually doing with generative AI at the moment. Do you see a difference between industries? What's going on there?
That's a good question. The survey respondents, I actually have a breakdown of that, so I can give you an idea. This year, 50.9% were from financial services. And Financial Services have always been the largest respondent in these surveys because they are very data and analytics driven and have invested in data and analytics initiatives for over a generation. But what's really exploding is healthcare and life sciences. So this year, 15% of the respondents were healthcare and life sciences company. And a lot of that was fueled during the COVID period, the race to develop vaccines and sharing data across clinical groups. We also had 11% from retail and consumer goods. And they've been very aggressive because whereas financial services firms and to a certain degree, healthcare and life sciences, those are industries that are heavily regulated. So a lot of the work is defensive. For consumer and retail, it's basically offensive. I moderated the keynote panel in Boston a year ago in October, and I had a woman, Diana Schuldhouse, who's the chief data officer for Colgate Bomb all over. Actually our title is like chief analytics and insight officer. And I had three banking chief data officers. And I asked, how much of your time do you spend on offensive activities? Revenue generation, business growth, and how much time do you spend on defensive activities in terms of regulatory and risk management and compliance? And the bankers were all like, you know, we're aspiring to spend maybe 30, 40, if we could get to 50%, that would be great in terms of offensive activities. And it came to her, she goes, 100% offensive. We started like cheering. So it's, you know, it depends what industry that you're in. I, I, I wrote a piece last year on Levi's. They're on the fashion business. So they're in the business of change and trying to anticipate trends and needs as opposed to when you're in financial services, it's about safety, security of the assets, basically being able to instill the same level of confidence that for 150 years you're American Express or Wells Fargo or Banco America or Citigroup, all of which have existed for basically 150 years.
I suppose it's going to be the more regulated industries that have to do a lot more defensive work because otherwise they're just going to get sued into oblivion or fined. So can you tell me a bit more about what some of these business risks companies are worried about in terms of using AI?
Yeah, I can answer that in a couple different ways. First, in terms of the defensive activities, Remarkably, this year, particularly in financial services, there was much more time spent on defense, more than they had been the past five or six years, which was prompted by earlier this year, the failures of Silicon Valley bank and First Republic Bank. So now all of a sudden, the regulators came back in, particularly for the kind of tier 2 and tier 3 banks, the really large banks had to undergo that level of regulatory scrutiny. So a lot of the CEOs that I work for, for the large regional banks and so forth were like, yeah, no, we're not focused on business expansion activities. We're spending all of our time with the regulators. So that's part of the answer. In terms of defense versus offense, in terms of. Could you just share the second part of that question about AI?
Yeah, it was just about what are the risks that executives are most worried about with respect to AI and generative AI in particular?
Yeah, there's huge risks. We asked the question about what is the primary business risk posed by generative AI? An answer for every question. By far the largest, 44.3% said, spread of misinformation or disinformation, particularly when you look at social media or media of any kind, is the opportunity to basically present anything that fits your audience. The second one was at 23.6% was ethical bias. At 45.7% was job loss and job displacement. And then the other things were everything else. The other thing I should note was that we also asked about the state of data and AI responsibility and ethics. For example, we said, is data and AI ethics the top priority for your organization? And roughly 3/4, 73.8% said it was. We asked, is the board of directors well versed in data and AI issues and responsibilities? 51.4% said yes and 49.6% said no. I will add there that on November 1, I was in New York and I did a panel at the national association of Corporate directors, which is basically Fortune 1000 corporate directors. And it was really to talk to them about generative AI and the issues and concern concerns and what they should think about, because they themselves didn't feel that they were sufficiently versed to bring those perspectives to the corporate boards they were on. We also asked, does your organization have well established policies and practices in place around data and AI responsibility and ethics? Less than half, 42.3% responded that they did. So, a lot of work to be done there. And then we asked, has the industry done enough to address data and AI ethics? And only 15.9% said yes. Roughly 85% said that the industry had a lot of work to be done.
So it sounds like there really is a lot of work to do in terms of educating executives. In terms of what does responsible AI actually mean and how do you actually make sure it happens in your organization? Does that sound like a reasonable interpretation?
Yeah. And earlier this week, I spoke to a group that I was just introduced to called the Responsible AI Institute, and they're working with it. They were an offshoot five or six years now out of University of Texas, Austin, and they're working with a number of Fortune 1000 companies to help develop the responsible AI standards and practices for those organizations. So they have a template in place. And I was introduced by one of the Fortune 1000 companies I work with.
Okay, it's good that these organizations exist just to try and help educate these leaders. We talked a bit about risks to businesses, but you also mentioned, I think was like 4% of companies just not touching generative AI at all. I'm wondering what the outright blockers are to adoption.
I would say it's the same issues that have been barriers to adoption for any data and analytics initiatives within organizations. And it's a question that we've asked over the years. And we say, basically, what is the principal challenge to becoming data driven or data and AI driven? And the short answer is the answer is consistent. We ask whether it's culture, people process an organization or technology limitations. And almost universally, 77.6% this year said it had nothing to do with technology. It was all about culture, people change, management, communication, really all of those issues that it takes to embed or launch a capability within an organization. There's a lot of natural resistance, a lot of lack of understanding. I sometimes tell the story about going into organizations and I meet with the data leaders and they say, oh, here's all the capabilities that we're creating. And I said, that's very commendable And I meet with the CIO and the technology organizations, and they say, here's the architectures and the infrastructure that we're creating. And then I go and meet with the line of business leaders and I say, so are you getting value out of your data analytics investments? And they're like, not as much as we'd like. We actually don't trust the data. And we're not getting the data that we need to make, the decisions that we need to make. We're not getting into a timely fashion. And on top of that, we don't understand what the data and technology people are talking about. They talk in a language that we don't understand. Master data management, data mesh data fabrics. We don't know what that means. We're the ones running the business and speaking a language over our heads, and we're the ones that. That have to keep the customers coming in and make a profit and derive revenue. So there's been this antipathy or antagonism where a lot of the businesses get frustrated because they're just saying, hey, you know, we're running a business, we need to sell stuff. And there's a story from a number of years ago where a chief data officer I know went into, the president of the organization said, I need $25 million to build a massive data management capability. And the president said, I have no idea what that means. So the answer is no until you can speak in business terms like please leave.
It does sound like, although there's been a lot of progress this year and lots of success stories in terms of making progress with AI, it's actually the fundamentals, the data quality, the data governance is the hard part, where executives seem to be struggling. Does that sound right? Like, where were the biggest pain points?
So we've been asking this question for six years now. We ask about the progress of data and analytics initiatives. And I'm going to note that these numbers improved significantly this year. And I had mentioned at the outset that we thought that this was due to generative AI, But I'll share last year's number, which had been basically flat for five years. And this year's number. Is your organization driving business innovation with data? Last year, 59.5% said yes, which was the exact same number as 2019. This year it jumped up to 77.6%. So significant, whatever that is, 20%, 30% increase, we said, is your organization competing on data and analytics? Last year, 40.8% said yes. So less than half, which was down from 2019 when it was 47.6%. This year it went up to 50%. So half said yes, half said no. It's a little bit like glass, half full or half empty. So a lot of progress, but a lot of work to be done and a lot of need and demand for employment in the future. We asked, are you managing data as a business asset? Last year it was only 39.5%. This year it jumped to 49.1%. So pretty significant. Again, all in this context of generative AI. We asked, have you created a data driven organization? Last year it was pitiful. 23.9% said that that's where they were. This year it more than doubled to 48.1%. So that's extraordinary. I hadn't seen that in any previous version of the survey. And lastly, we asked, have you established a data and analytics culture? Last year it was 20.6%, 1 in 5, but this year it jumped to 42.6%. So again, we attributed these massive increases to the attention that generative AI has brought to the organization, how it's really elevated data and analytics in all areas of the firms.
Okay, so that does sound incredibly positive then, that there's been a lot of progress made in core analytics just as a result of the sort of hype from generative AI.
And to your question, one other data point we did ask, have efforts to improve data quality been successful? 37% said yes. So still a long way to go in data quality. I don't know if data quality will ever be solved.
Yeah, I mean, I guess you can always have higher quality data and it's always one of those things where no one really wants to do it, but it needs to be done.
Yeah. Okay.
Do you have any more of a sense of like, what's causing some of these problems? Like, because a lot of these cases where it's like, is your organization any good at data? That sounded like it was. Generally around half think that they're doing okay and the rest are struggling. Do you have a sense of like, where the problems are here? Like, what's preventing greater success with data?
I would go back a little bit to the cultural issues. Maybe we could talk a little bit about the role of the chief data officer. But the point is that this role didn't really exist for the most part prior to 15 years ago. And for major banks who were the early adopters, it came about as a result of the 2007, 2008 financial crisis. For example, we asked, has your firm appointed a chief data and analytics Officer and in 2012, only 12% said yes. This year, 83.2% said yes. A couple things to keep in mind. The rules still are relatively nascent. A generation ago, there was the old joke that CIO stood for career is over because it was so volatile. And that's a little bit what chief data officers have been experiencing. Tom Davenport and I wrote in Harvard Business Review a year and a half ago that the average tenure was roughly 24 months. We've actually seen that go down in a number of organizations. In 2023, we saw more turnover in Fortune 1000 Chief Data Officers than in any previous year. Just a couple other things.
Sorry. That's just quite a surprising statistic. Like, chief data officers last less than two years, and it's getting a shorter tenure now. Do you have a sense of what might be driving that?
Yeah, absolutely. Two factors. One is, this year there was a push to deliver business value, and a lot of that was driven by economic factors. So organizations had concerns how much money they'd have to spend, and they had to make decisions about where they spent it and didn't spend it. And over the past five, six, seven, eight, nine years, organizations, every year, you know, 98% of them said they were spending more on data and analytics initiatives, but there was still gaps in terms of achieving quantifiable, measurable business value in terms of profit, revenue generation, customer acquisition, things of that kind. So a lot of organizations have been spending time on building infrastructure, architecture capabilities and so forth. So early on this year, a number of organizations, the leadership turned to the chief data officer and said, so where's the quantifiable results from this money that we've been investing over the past few years in data and analytics initiatives? And they're like, well, you know, it's like we're building capabilities, but we don't necessarily have those numbers. And in some cases, the executives said, then, you know, you're. You're terminated. And in other cases, they said, well, you know, you have, like, 60 days to, like, produce these numbers. And as a consequence, some people, basically, they weren't in a position where they could do that. And others just said, we're not going to deal with this. This is just, like, irrational in the sense that all of a sudden the clock has turned so suddenly, and now it's these demands are being made that were not set up organizationally to respond to. The other factor, in addition to the demand for demonstrating business value was generative AI. And with its rapid emergence, many executive leaders turned to the chief data officer and said, okay, so what is, what's the impact of generative AI? What should we be doing? And those Chief Data officers, like the rest of us were learning, you know, they didn't have, like, automatic answers. And so as a result of that, a lot of the organizations said, you know, we need to get somebody in place that can figure this out now where we can't wait. So it's. The role this year has really evolved more than any other year, and it continues to shift. And as I mentioned, I was at the event last week and there was a Chief Data Officer, Chief Data Analytics Officer day. And then the following day was Chief Artificial Intelligence Officers. This the creation of this new role. And one of the questions that we asked was the. In terms of the Chief Data Officer mandate. So we asked a few things. When the role first started, you know, it was Chief Data Officer, and it was a lot about data management, defensive activities. And then as organizations start to move more onto the offense, they said, well, we need the analytics combined in there. So it evolved from CDO to CDAO role. So we asked, is our analytics today part of the CDO CDAO function within your organization? 70.8% said yes. So analytics is very much integrated for most organizations. But then we said, Is generative AI part of the CDO CDAO mandate? And 61.7% said yes. So slightly more than half. And then we said, should generative AI be part of the CDO CDAO mandate? And 79.4% said yes. So clearly it's a work in progress. It's a turf thing too, where organizations are starting to say, you know, this generative AI, it has transformational power for our organization. Is that something that fits under the role of the Chief Data officer? Because maybe we thought of the Chief Data Officer more as a specialized type of role. Is this too big for that? Where does it fit within the organization? So it's very much a work in progress. From what I'm saying.
It does seem like someone who wants to be in charge of massive data management and making sure that metrics are harmonized across different departments. That's a very, very different role to how can I improve productivity using generative AI?
Yeah. And I'm often asked about why is this gap between in the 10 years of chief data offices and why some Chief Data officers are successful and some are not successful. And initially there was a lot of feedback or comments that the business leaders don't know what they're doing. They don't really have a firm plan in terms of how Data and analytics fit into the organization and the overall picture. On the other side of that is that a lot of people were promoted or appointed into the chief Data office and chief Data and analytics officer roles were fundamentally subject matter experts. So they weren't business people. They didn't necessarily have the vision of how data and analytics fit into the overall profit picture and business goals of the organization. They were new to the C suite in terms of both the communication and the turf fighting and the sharp elbows and building relationships and sponsorships. So they were kind of a little bit like sheep to the wolves, if that's the right analogy. It has been a learning experience. Some chief data officers now are saying, you know, I'd never do that again, and are opting out or opting for different roles. And now there's a new and next generation of chief data officers stepping in. But the expectations continue to change and evolve. But you know, this shouldn't really be shocking because as noted, it's 10 to 15 years into it. And early on in my career I was working with a Fortune 1000 CIO who was Vice chairman of their organization. He said to me, you got to understand that nothing happens in less than a decade. And I was like, wow, that's a very cynical point of view. But now, having lived through many decades, I've experienced that particularly for legacy mainstream companies. They undertake change and adopt change in a very gradual fashion. They're trying to be responsible. Their job's not to be bleeding edge. That's left with the Googles and Apples and Facebooks and Microsoft and others. They're competing with the others in this sector. If you're an insurance company, you're not, you know, going to do like cartwheels. You're, you're going to basically continue to serve your product. That's why organizations like Ally Financial that I mentioned, or Capital One in banking have been revolutionary in the context of their particular industries because they've been very data and analytics driven as opposed to the traditional product and service approaches. Yeah, for mainstream companies, change takes time and it tends to happen more over a period of decades. I mean, we're 25 years into the digital revolution, digital transformation. I know a lot of the organizations I work with said during COVID that that's where they made their exponential leaps in terms of their digital capabilities. There was no transformative urgency before that. But when people couldn't come into the branches or their offices, they had to figure out how to deliver all their services in an online digital fashion.
Maybe I've spent too Long this start, because decades seems like an impossibly long time. So I was going to ask you, like, how you think the Chief Data Officer role is going to evolve over the next five years. Maybe it needs to be a longer time period, but. Yeah. What do you think is coming next for Chief Data Officers?
You know, I think that's the crux of the issue. The most important question now in terms of that role, that function, you know, right at this moment, it's a work in progress. It's the issues of this generative AI is the responsibility of the CDO or not? I mentioned a panel that I did in October in Boston, the CDO event. I had three panelists. I asked that question to three panelists, and two of them said it should absolutely be part of the CDO role, and the third said it should absolutely not be part of the CDO role. And some of the reasons they pointed out was that there's so much risk and regulatory and privacy and ethical issues that need to be dealt with first that really needs that type of safeguards and people that are experienced with that and bringing new capabilities to bear as opposed to just saying, you know, let it rip. It's a new function. And the whole set of considerations that come into play relative to the role in general, you know, it's moved more from the defense to the offense. Integration of the analytics capabilities. Now we'll see where AI fits into that. And also there's been an elevation into where the role reports in the organization. For example, this year, roughly 35% of chief data officers are now reporting to the CEO or chief operating officer. Five years ago, most of the chief Data officers reported to the cio. So it's really moved out from under the CIO function in terms of being a peer function now and greater elevation in the C suite hierarchy.
That's interesting how the reporting structures changed. Now, you mentioned data culture before. So I'm wondering if you've seen any changes in executive attitudes to data in the last year.
Yeah, in the past year, there's been a significant transformation. And again, that's been driven by generative AI because it's produced an urgency to potentially see how this technology can be leveraged to, you know, achieve the productivity gains, elevation of job responsibilities, et cetera. So if before, for many organizations, the idea of data and analytics was a little bit abstract, in other words, yeah, of course we should be data driven, a little bit like motherhood and apple pie. But now it's experiencing that level of urgency where organizations are saying, we really need to think about this in a serious fashion. We're at a transformative juncture because, for example, you know, when I went to the Wall Street Journal Tech Live event, people were asking me, oh, you know, what's happening with AI? And. And also organizations were saying, can you come to speak to us about AI? I'm like, you know, I don't feel qualified. I'm not an expert. It's moving too quickly, et cetera. But my biggest takeaway from the Wall Street Journal Tech Live event was that it's inevitable and it's happening quickly. So whether you love it or hate it, it's going to happen. So you got to get over it and you got to get your arms around it and plan for it. In the same way that the railroad shortened time frames, the invention of the airplane and shortened time frames the engine increased productivity. You know, I was reading something the other day where you talk about the railroad or the airplane, but it was about the development of North America and Native American peoples. And it said when the horse was introduced from Spain and from Europe, all of a sudden it made, you know, there was thousands of tribes, but they never interacted. They were a couple miles apart. But now with a horse, people could get there in five minutes. Generative AI will undoubtedly have an impact of that magnitude. And the other thing is that people say AI can destroy everything, it can destroy mankind, and a lot of arguments can be made for that. But at the same time, it's just a tool. And any tool can be used for good or bad. You know, somebody's asking me the other day, and I was just thinking about the most simple thing that came to mind, and I said, a screwdriver. Screwdriver can be used to fasten things, but a screwdriver can also be used to, like, bludgeon somebody if you were someone blind. Technology, any type of technology, or any tool can be put to very good and positive uses. And there's all kinds of things in terms of curing diseases, scientific knowledge, medical, health care, advancements that can happen exponentially through generative AI. And organizations are working on that now. Instead of going to a doctor once a year, you can be monitored, like, throughout the day, hour by hour. But there's also, particularly around social media, elections, things of that kind, leadership. There's a lot of dangers of misuse, propaganda, things of that kind. Many positive opportunities, but also some really negative ones. I mean, you can create images now that see, oh, here's the proof in the photograph, or here's the proof in the filmed pictures that we have, but they can be created. You know, I think we saw something the other day where they had police arresting Trump and carting him away, but it was just generated.
Absolutely. So I suppose fake images have been around for a couple of decades now via Photoshop, but this really does scale up the ability to create these sorts of things.
Absolutely.
Okay, so I'd like to go back to data culture a little bit because you mentioned that at this point all executives are saying, yeah, we need to be more data driven. But I think the stat was less than half think their companies actually are data driven. What's going to change that culture to increase the stat?
One answer is time. Because change, change management tends to be gradual, particularly with regular mainstream legacy companies that risk averse and are going to move at the pace of their peers. And we're not going to try to be Amazon, Google, et cetera.
So one common refrain from some of the other guests on the show is that data skills are still a common or a lack of data skills is still a common problem. So do you think this is still true or has some progress been made?
I just came up with the second part of my answer to your previous question and that is generational. Younger generations are now more schooled and it's second nature to use various technologies as opposed to C suite executives that may be in their late 50s, 60s and so forth. So for them it's a transformation effort. It's this is the way we've done things. Yes, you know, we're going to be sensitive to the data and AI capabilities, but it's just not something that's in our DNA and in our nature and particularly within mid management levels. But as next generation comes along, it seems like a long time because you know, What's a generation? 35 years. But some of that is just that it's hard to teach an old dog new tricks, but a new dog where it's in the DNA, it just becomes something that's more natural. So some of that is just a function of time in that regard. In terms of data skills, I know that there's a lot of investment in initiatives like data literacy and things of that kind. All of that is good. But too often I hear those terms that are just that they use loosely, like if only we were data literate, if only, you know, we had democratization of data. You know, what do those things really mean? I mean the most successful organizations, from my experience of those organizations where the data leaders, the chief data officers are establishing strong partnerships with their business peers the business leaders, they're engendering their trust. They're using data to answer one question at a time. And being able to do that establishes the credibility and trust. And when you can do it two or three or four times, that establishes momentum and traction. But not enough data leaders are doing that. So it's my experience that the data leaders that are really hand in hand partners with business leaders are the ones that are successful. They have somebody guarding their back, they have somebody that's an advocate for them. I know at J.P. morgan, Jamie Dimon, at the most recent annual meeting, he said, there's four factors that are most important in our results and our future and our success. And I think two or three was data and analytics. So he was out championing the importance of data and analytics as opposed to a number of instances have seen some chief data offices say, hey, you know, we're doing great things. Why doesn't the business see this or championing us? So when the business is taking the lead and saying, you're doing a great job, you know, it's better to have them saying, hey, you're doing a great job, than you saying, I'm doing a great job, but nobody seems to appreciate it.
Okay, what are you most excited about in the world of data and AI right now?
The important thing to understand is that data is the foundation. Okay? So if you don't have good data, you're not going to have good AI in terms of the outcomes, in terms of the predictive analysis that you're performing. A successful AI is predicated upon data, which means that if you want to be successful with AI, that increases the urgency, the importance, the momentum, everything around basically building out your data structure and making sure that you're working off a great foundation. So in that regard, it really, the use of AI only emphasizes the importance of data. And as organizations do more and more with AI, you know, they'll see that the quality of their data, there's a direct correlation between the quality of their AI initiatives and results and the quality of their data.
I suppose there's two sides to this. So, like, the one side is that although everyone really wants to be doing the fun work with AI, actually they need to be just doing grunt work in terms of making sure the data quality is high. And it's a sort of dirty secret. It's like there's a lot of, like this sort of stuff goes on in order to get good performance. But on the second thing, it's like we talked before, that data quality has been a persistent problem. And maybe this is the motivation in order to increase data quality. All right, wonderful. Do you have any final advice for leaders who want to improve data or AI capabilities in their organization?
I would just say the more of that, there's that partnership and sponsorship with business leaders. And I think that's the best advice I can give to any new chief Data officer is go out into the business, understand the business, understand what the key issues are, understand what the key mission and drivers are for the organization. And basically wearing your data hat, think about where data can make a difference and begin to articulate that and gain buy in, explain it, and gain buy in from the business leaders. So they say, you know, that's very interesting. We haven't thought of that. Let's try that. Let's try this. Let's answer some questions that we haven't been able to answer before, and then when that's done again, that builds that trust and credibility and momentum, and that's when organizations start to really move the dial, and that's when the Chief Data officer really becomes a trusted partner and an integral member of the C suite of the organization.
Okay, so get your data leaders and your business leaders talking to each other and hopefully working together, and that's how you get to success.
Yeah, I think that's essential. Without strong business relationships, without strong business sponsorship, and without that business trust, you're really growing forever or against the tide.
All right, super. Loads of great insights there and lots of great advice. Thank you very much for coming on the show, Randy.
My pleasure. Nice to be here with Chief.
More from Richie
#359 My Best Friend is AI with Valerie Tiberius, Professor of Philosophy at University of Minnesota
#358 How AI Agents Will Work While You Sleep | Ruslan Salakhutdinov, Professor at Carnegie Mellon
#356 The Forecast for Time Series Forecasts with Rami Krispin, Senior Manager of Data Science at Apple
#355 AI's Impact on Databases with Shireesh Thota, CVP of Databases at Microsoft
#354 Beyond BI: Decision Intelligence with Graphs with Jamie Hutton, CTO at Quantexa
#352 AI Agents at Work: What Actually Breaks (and How to Fix It) with Danielle Crop, EVP Digital Strategy & Alliances at WNS
#351 Will World Models Bring us AGI? with Eric Xing, President & Professor at MBZUAI
#350 How to Make Hard Choices in AI with Atay Kozlovski, Researcher at the University of Zurich
#349 From AI Governance to AI Enablement with Stijn Christiaens, CEO at Collibra