#203 How a Chief AI Officer Works with Philipp Herzig, Chief AI Officer at SAP

Hi, I'm Maggie, VP of Curriculum at datacamp and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com.

You know there's all these things, Richie, that change all day long, right? The tech changes, the LLM changes, the tools change. But what shouldn't really change are our values. Like how do we think about how the world looks in an AI age and what is important to us when we design AI into our workflows and applications. And that is also something that we are continuously evolving.

Foreign. Welcome to Data Framed. This is Ritchie. With every company trying to get in on the act with AI, questions start to arise about who ought to be in charge of these initiatives. At the moment, it's a bit of a toss up between the Chief Technology Officer, the Chief Information Officer and the Chief Data Officer or maybe a committee. With all three taking responsibility for some projects. Some forward looking organizations have decided that AI is an important enough part of the business that it deserves a dedicated seat in the C suite. The Chief AI Officer role is naturally fairly new, so today we're going to explore what such a job actually involves, where it fits into the rest of the organization, and what success as a CAIO looks like. Our guest is Philip Hertzig, the Chief AI Officer at the enterprise software company SAP. He's held a variety of roles in that company on both the engineering and product sides. These days he's tasked with leading the company's business AI strategy. Let's find out about his job. Hi Philip, welcome to the show.

Thank you so much, Richie. Pleasure to be here with you today.

Excellent. So there are a lot of technical C suite positions. You've got Chief Data Officer, Chief Technical Officer, Chief Information Officer. How does the Chief AI Officer role differ from these?

That's a great question. So I mean, first of all, it's a new role, right, that we also at SAP, we have established at the beginning 2024 and at least, I mean there are different interpretations right, on what that role entails. The way or why we have set it up specifically in SAP is to really take a 360 perspective on AI for all the functions. I mean, as one of the largest enterprise application vendors, right, specifically out of Europe, we have to take not of course, only the adoption internally, right. For our processes and for our people into account, but obviously how we think about the products, how we bring AI all to our products. We can spend a little bit more time, I think about our product strategy in a minute and then really think this end to end through, right. How do we do legal aspects, the commercial aspects. And as we all know, right. Who are also regularly listening to this podcast, there's so many things that happen every day, every week new models come around the corner. There's something happening on the regulatory side of the house. So the question is really like what is the center of excellence or like the nucleus in the company that takes all those signals, all those concepts into account and fit them together like differently or every single day, right. And there's a, every day there are new opportunities, new risks. And in this role, the way how we interpret it is really take an end to end view. But at the same time it's also just a relatively small department because at the end of the day, and that's part of our strategy as we embed AI, specifically all in our, our, our enterprise applications is then to work with all the connected board areas, right? Product and engineering technology department, the cto, the CIO for internal adoption, our sales and marketing teams closely together, right. To then scale the entire approach across the entire portfolio that we have today.

Okay, sounds like you're working with a lot of different teams, as you mentioned, commercial teams. Okay, so can you go into any more depth? Like what do your day to day responsibilities look like?

So first of all, as part of this chief AI officer mandate, we have basically all the roles in there, right? So we have, for example, part of my team that was the team also before, right. So AI is not a new thing. I was before that I was on the engineering side of the organization, right. This is also why this cross collaboration and this work is so natural to us as a team. Because I was actually part of the product engineering board area within SAP, right. Where all the applications like our ERP or HR applications, finance, supply chain procurement applications like Conquer, Fieldglass, Ariba, SuccessFactor, some of them you may know located. And I was always the cross engineering guy, if you will, right. Looking top to bottom, looking at our user experience, looking at our process integration, our technology adoption with respect to our technology platforms and AI, we also did in the last couple of years there as well and we moved it out directly on our CEO Christian Klein. And engineering is still with us. Where we build platform services like for example Joule, our generative AI co pilot, which all of the applications are adopting. Where we build what we call the generative AI hub, which is kind of the glue, if you will, where we bring the best of large language models or foundational models together with the real time data that is stored within SAP and non SAP systems. A couple of other reuse services for document information extraction, data attribute recommendation, other recommendation services that are kind of reuse services that you can embed in multiple different applications. So we build those, that's part of our engineering layer that we provide as a service platform service to our own applications. But then of course we have a dedicated product partner management team, we have a dedicated marketing team, we have a dedicated go to market and sales team, we have a dedicated implementation adoption team that helps customers. Then all the innovations that we are building to adopt them very quickly, but also funnel the feedback from, from those customers back very quickly. And as you can imagine, what does my daily job look like? Like all of those things. So every day we look at what's new on the product side of the house, right. How can we bring that directly to the customers? How can we enable our field with that? How can we turn first customers, right, really to adopt also our latest innovations with that. What is the impact on our marketing approach? And we are reviewing all of those things every single day in order really to move really, really quickly and in alignment with our customers expectations.

Okay, so it sounds like you've got quite a lot of different AI products going on there. I'd like to get into a bit more about where your role sits within the organization. So you said that you work with a lot of different teams. Who do you report to?

Yeah, so I report directly to our CEO Christian Klein.

And then who is in your team? Like who reports to you?

So within my team there is for example Walter San, who joined us last year in September from Microsoft. They're working also in the business application space, you know, around Dynamics 365 and, and some of their platform services. And he runs all of our AI engineering functions. Both our research that we are doing with universities, you know, like Stanford and Berkeley and a few others in Germany like Hassel Plattner Institute or the Technical University in Munich, and a few more up onto all the services I described already. Right, so really on the technology platform, all those reus services that we are building and then we have, as I said, right, we have A dedicated leader, a very renowned leader within SAP, Jan, who reports to myself, looking after go to market and spreading basically across all the reaches our enablement material. Right. And how we position, how we are selling then also AI to our customers. We have dedicated marketing team, product and partner management as we also as part of our strategy, we are partnering with many of the companies, with Microsoft, with Amazon, with Google and many more Nvidia. And then on the other side, of course we enable all the other partners. Right? So obviously SAP is also happily partnering very closely with all the global service and consulting organizations around the world in order to also scale from an implementation and consulting perspective as well.

Okay, so it's not just working with internal teams, you're doing a lot of partnerships with other companies as well, since that external facing side of the role too.

Yes, exactly.

Okay, so when you're working with so many different teams, I imagine like the communication flow and like just managing the logistics of doing so many different teams is a challenge. Can you talk me through how you approach that?

We have kind of for all the various functions we have very close and regular alignment meetings on. Give me just one example already last year, specifically when the entire hype right around ChatGPT and what, what do we now have to do with respect to generative AI? What, what does this mean for us, right? And what, how do we bring this into our how into our applications? How can we help, right, to make HR more productive or shared service centers more productive sales, more productive services marketing. And we formed a little group already last year was January or something where we formed a small team and thought hard about how do we build the first scenarios. Basically because our strategy just here to back up a little is really first of all, SAP does not strive so much for a general purpose technology platform, right. That helps customers to build AI like tools for prompt engineering or retrieval, augmented generation or data pipelines yourself, right? And then you have some smart people who can build those things like a chatbot or digital assistant or something. Our strategy is that these things, things are embedded in the applications and just turn them on and just work out of the box, right? And so the question first was what were high value use cases that we would develop that where customers say like okay, that is actually something where I see generative AI embedded into an SAP application actually helps me. And we figured those cases out very quickly and just started executing was a small, nimble but very capable team that we've started with in order to move that forward. And it turned over the last year. It turned out into the standard call, we run it on a weekly basis. When we actually wanted to release our first scenarios, it was actually on a daily basis where with all the teams, we went through all the challenges that are there, right. In order to productize them and put them in front of the customer. And we are still running it ever since. So all the graph, meanwhile, released via this alignment call, released 39 generative AI scenarios in Q4 and Q1 alone. There are 40 more in Q2 till end of this year, there's a hundred more that we are releasing. And this is how we are aligning that. Right? And every, every time somebody comes up with a new challenge, right, or, or new thing that needs to be addressed or it needs to be rolled out because we come up with a new concept on the commercial side of it. And we are doing this all in this one single call, if you will, in order to really let the flow of information flow throughout SAP as a, as a obviously very large company. And so far it worked very well.

Okay, so it seems like you're just doing lots and lots of communication with lots of different teams just to make sure everything's.

And then also as part of my previous gig, just also how do we align architecturally? Because I think there's also something that I'm very, at least very happy with. SAP is a very heterogeneous company. Right. We have many lines of businesses. Right. And the various areas. But also as part of my previous role, and that helps obviously a lot, I was running what we call cross product architecture. So that was a approach we established in the last three years to really get all the various lines of businesses, no matter whether it's Conquer or whether it's SuccessFactors or Reba or SAP S4 HANA to agree on the common technology foundation, a common set of platform, services and capabilities that everybody has to adopt in order to come up with what we call sweet qualities. Right? So a set of qualities that are equal, right. Amongst the various application. Very simple things like that all the mobile apps, right. That the login works the same way, all of them are using, let's say face ID or have the same look and feel down to. Okay. The identity management is, for example, harmonized and that was a huge program we did over the last three years and that obviously helps a lot. Right? Right. Because now we could design also the AI foundation for all the applications the same way. So when I talk, for example, about Jewel, our generative AI co pilot, this is not like a jewel for HR or a jewel for finance or A jewel for supply chain. No, it's one jewel that sits at the heart right of the architecture and all the applications are connecting into it. And the more applications you connect to it, the more powerful the assistant eventually becomes.

Okay, so that seems like an interesting thing about AI is because you require all this data, having a single AI connected to lots of different things is going to be more powerful than having lots of individual. Okay, all right, so I'd like to know a bit about what constitutes success. So how do you know if you're successful in your job?

So first of all, it's all about basically three main aspects, right, that we always consider. First of all, it's do we deliver the right scenarios for our customers, right. So that it really has the relevance to be adopted at the end of the day. So the ultimate success metric for us is clearly adoption specifically in the cloud, right? I mean you can build best, most amazing thing in the world. Nobody actually adopts it. You probably build the wrong thing. So we are entirely guided by adoption and making sure it can be easily turned on out of the box. It's available as a service because we figured out over the last couple of years that specifically, you know, it's not delivered out of the box, not as a service. Adoption usually is very hard. So that's number one. The second one is clearly reliability. I mean we all know what the limitations of large language models are, right? With all those things, hallucinations only has been trained on public data up to a certain point in time. Of course they do not know the current realities of your business. Like oh, where are my sales orders, where are my general ledger accounts, where's my current goods receipt or where's the delivery right of a certain, of certain goods, et cetera, et cetera. So we need to bring together in a reliable, secure and data privacy preserving way the real time data of the SAP systems together with the large language models in order to really bring in the contextual knowledge you need in order to make actually the relevant uses cases to happen. But of course we want to do that in a very secure and trustworthy way because this is what our customers come to expect from SAP for, for decades. And the last piece I'd like just to mention is also our approach to responsible AI. So SAP was one of the first companies to establish what we call an AI ethics policy back in 2018 and it guided us well through the years. So this is really. There's all these things, Richie, that change all day long, right? The tech changes, the LLM changes, the tools Change. But what shouldn't really change are our values. Like how do we think about how the world looks in an AI age and what is important to us when we design AI into our workflows and applications. And that is also something that we are continuously evolving. And that's also part of one important success metric, that we're always sticking and staying true to our values when we design specifically generative AI into our apps.

I like that your sort of main judgment of all criteria for success is just like you are shipping AI products and your customers appreciate those. I wondering like in more depth, can you talk about like do you have any specific metrics or targets that are used to evaluate your success as a chief AI officer?

Well, we obviously want to get as much customers to adopt our things there in total. I mean overall, what I can just share is we have already. I mean AI overall is not a new topic for SAP. We have roughly 27,000 enterprise customers right around the world who are using our AI capabilities on a regular basis. And yeah, of course our ambition is to heavily continue that growth from an adoption perspective in the years to come. And specifically with generative AI, it seems.

Like you've got a lot of different products and projects on the go. Can you talk me through how you prioritize which AI projects are going to be most important?

Prioritization is always done from a value perspective. So what has the highest value? There could be multiple things, right? There could be potential top line impact for a customer in particular if you think about gentle generated scenarios in sales and services or marketing. So front office related roles could be bottom line improvements, cost of goods sold, improvements through generative AI or just simply as we target that with joule to rethink the way how people interact with the system in a way more delightful and easy fashion compared to, you know, rather classical click through all the UIs to accomplish your task fashion. And that is usually the way how we evaluate the scenarios. So when we go through the scenarios we are always looking at it from a what's the return on investment from a customer perspective. So what is really the benefit that helps the customer to do something faster or easier, right? Or helps them to achieve more with less. So those are always the questions that we ask already very early in the product life cycle before we go further down the road in order than to implement and spend our resources on those cases. And this is how we run it for the most part.

Okay, so the three things you mentioned there were improvements to top line, so increasing revenue and then improvements to bottom line, it's cost saving and then working better with your software. So I guess it's a customer experience improvement. Can you go into more depth, just explain to the audience like so what sort of projects are going to help improve revenue, what sort of projects are going to help it save costs and, and what sort of benefits you get from working differently with your software.

So just give you a couple of examples that we are having. I mean one of the more classical scenarios that we have been just to draw in a few numbers that we have been doing for decades is data entry in your system. There's still, across all the different business processes, there's still like tons of paper. And with paper I also don't need like printed paper, but email and PDF and digital paper, still paper. And still there's like a plethora, right. Of processes that require you to tediously manually enter all of those things into transactions, into into screens. Right. And just transmit basically what's on paper into that system. And I think there's a lot, there's a tremendous amount of potential to just give you a few numbers alone and conquer, for example, just with B2B in the background, we are processing like somewhere in the range of 60 to 100 million invoices. Right. In a fully automated fashion, for example. Right. And it's just in the system, fully itemized. Right. They're just in there. There's no manual error prone and tedious work just to key those, key those in. And everybody who has filed an expense report. Exactly. Knows how tedious that sometimes can be. And we can save a hell lot of effort and money by that. Right. And a high triple digit million number range. We also look for example into shared service center. Right. Think about accounts payables for example, or accounts receivables. If you have a large shared service center where all the various complaints, let's say for example accounts payables, right. You have all your suppliers and they are complaining because you may have not paid them and the contractually agreed on times because maybe of quality issues or what, whatever. Right. Blocks the payment to the supplier and, and you easily, depending on your size of the company, you can like have hundreds, thousands of these cases on a daily basis. Yeah. And then of course it starts with simple things like oh, how can I catch up with the threat and the discussion that was already happening via email maybe with that supplier. Imagine there's a new person looking at that thread and generative AI instead of reading all that emails helps you to just catch up with that thread in like virtually no time, right? Versus reading 10, 20 minutes per case through that email. And then you get suggestions what you do with it, right? You can say like, okay, maybe you do it, do a recommendation, maybe give a voucher or give some more time or maybe give certain discount or all those things which would be then recommended via jewel. And then once you have to resolve that problem of that ticket, it then also, of course, helps you, for example, to generate an email. And not just a email like the standard thank you email, or we solved that email, but one that is specifically written with the business context in mind. So having the reference to the invoice number, maybe to the purchase order, right? So that also the supplier has a good chance to reconcile that again back in their business much faster and much easier. So it's not about creating a plain vanilla email, but really one with, with context that usually if you would craft this manually, getting all those fields out and so on, the email can easily take you, I don't know, 20, 30, 30 minutes to write. And now we can just do it in, in seconds and multiply that with the volume of cases. That's a huge time saver. And it's not only a huge time saver, it will also help, right, to improve either your supplier experience, right, with your suppliers, because you're reacting faster, you're resolving the cases faster. And accounts receivable side, of course, also helps with the customer experience because obviously customers will be much happier because you resolve their cases faster too. And those are maybe two examples that I can share, I think, that are relatable, that are not too deep in the business process world.

Yeah, absolutely. It always feels amazingly clunky, like how much effort it takes just to either pay money to another business or receive money from a business. And so just being able to automate that, that just seemed like an ideal use case for AI, speeding things up. All right, so can you talk me through what are the big challenges with your role?

I think overall, one of the big challenges that we deal with every day is how can we absorb the massive change and the massive transformation that is happening at this, to be honest, unprecedented speed. I mean, we had some other huge technology disruptions before. Everybody always compares generative AI, what's currently happening with the Internet, or what's happened with mobile, or the introduction of the microchip in the PC. And the difference is all those trends were maybe as disruptive, but they were not as fast as this currently happens because all the infrastructure is there, or lots of infrastructure at least is there. And it's so accessible and taking that in and determining what are the right products to build, but then also helping our customers at scale, right? Not just in a certain market or for a certain pocket of customers. That's simple, that's usually easy, but really scale it throughout all the various departments within SAP, but then also reach all the customers with the same narrative, with the same message, right? And also keeping them up to date is really a challenge. I just had a, just to give you an example, I just had a partner meeting today and they just told us, Philip, you need to help us to keep up with this pace, right? It's, it's just too fast. That's on the one hand, side, it's a, it's a good problem to have, but it's also a clear challenge, right, because it's pretty tricky to. Everything that happens on the tech side, right, to immediately absorb, build the right products and then enable every customer and tell them, look, this is the way how we, how we design it. So doing that at scale for like hundreds of thousands of enterprise customers is.

A big challenge, I'm sure. Certainly keeping up the pace of AI development is a challenge for everyone. So I'm wondering, what's your solution to this? Do you have any advice on how to keep up with.

That's a fun question, but I also have, I think I have a little bit of an unconventional answer too for you. So first of all, I read a lot of things myself, so it helps a little bit that I'm still an engineer myself. So I go into many things. Also over the weekend I'm trying out many things. Some of those large language models, not some of them. I try them all of them myself, try to run them locally as much as possible. Think of some of the open source models like Llama, right, or the Mistral models. And then Basic and of course read all the, read and hear podcasts like Data Frame for example, and, and others to keep up to date. But also, you know, relate a little bit like different opinions, different point of views on what people think about it. And then what I basically do is not just hearing those things and trying to understand it for myself, but I sit down basically every week on the weekend and I write all of that stuff down in an email and that goes out to thousands of SAP employees. It's basically a voluntary subscribe to. There's always a link and hey, if you want to, if you don't want to receive this anymore, click here. If you want to receive it, click there. And luckily I get more click there than Click here Things and meanwhile reaches a couple of thousand SAP employees on a weekly basis. And it's not just a rundown or write down of all those things, but also actually it's like a big action item. Like, hey, have you seen this? And in this technology there's this new model. Please evaluate it deeper. It looks good to me. I've tried to following things out. I think that's helpful for this service. I think it's helpful for that service or when there is something happening in the market that may relate to some of our products, right? Either on the technology side or on the application side and say like, hey, dear peer L1 leader, please check this out. I think you should do a deeper dive on what that means and whether we can incorporate that or if it's of any use. And it's like a, it's both, like a rundown or breakdown of all the things that happen in this week. Plus it's a, hey, here's an action item. Please come back next week and please let me know if there's something that's useful to you. And you know, that kind of process helps to not only digest and by writing it down again, but also to really, you know, reflect on where does this belong to in the company and figure out who should be on point. Right. To evaluate whether that technology or that tool or whatever is coming out or that research paper. Also read a lot of research papers obviously, and then to figure out what that would mean and if we can incorporate those things within our solutions somewhere. Okay.

Actually, I have to say I really like that approach. So when you read something interesting, you then think, well, who else in my company is this going to be relevant to? So you can say, okay, this is going to be great for this service or this product and, and that way it actually gets used rather than just.

Exactly, which then is kind of this translate, right? What this whole thing is not just, oh, here's the rundown, think for yourself. But here's already. I think there's an anchor point here, right? There's a, there's an idea, a concept, and I think you really should take a look and that helps a lot that people then really take this more seriously and say like, okay, well then I take a look at maybe sometimes also turns out I was wrong, that's fine. Right? And the approach doesn't work because it's maybe not yet mature enough. We had this a couple of times, people came back and then said, oh well, yeah, we tried it, but maybe version two, let's see that Also happens, but still they, they expose themselves to, they, they got to know that. And obviously this is part for me of this whole organizational learning that needs to happen in order to stay up to date with, with what, what is happening.

So if you could go back to when you first started this role, what advice would you give to yourself?

I think it's always important to stay hungry and foolish and try out all these things, but you know, it also can get daunting sometimes and in particular in this fast paced environment that we are in. And I can only give this advice to everyone to just continue. There will be also lots of stressful and painful days. But I think overall if you can train your resilience in the current times quite well and so far I think the reward is much higher, bottom line. And yeah, if I were just going to go back and say like, yeah, know what you're subscribed to, know what you're up to, but then also have the resilience and the patience to work it through.

All right, super. So I have to say you weren't selling the GPA officer role to be able to, yeah, you need a lot of resilience and a lot of pain here. But I like that there's some good rewards with the role as well. Okay, so can you tell me what you're most excited about in the world of AI at the moment?

I think just top of my mind right now is, I think just last week is the release of how do I pronounce this is UD or something. Because I'm also just on my personal level when it comes to AI progress because I'm a wannabe musician for 20 years. I think we just reached the next leap in terms of music generation. So that is something that is like extraordinary to see, knowing what it takes also, you know, to play yourself and create and design and play music for years. But no, I think overall it's just exciting to see really that with AI in general, I think we are at the tipping point of an entirely new century and having the privilege, right, to be on the forefront and being able to influence and design also both in a relevant but also responsible way, this technology for enterprises around the world. Right. I mean the largest companies on this planet rely on SAP and I think simply the impact that we can create with that, again in a relevant but at the same time responsible way that is this just mind boggling on the impact that this can create in the next years to come. So this is, this is what, what motivates me every day because it's it's really, really impactful, the work that we can do.

Okay.

Yeah, you write.

There's so much amazing stuff going on. I would also like to second the idea of having AI generated music. It's gonna be a br. I always think, like, it would be kind of nice to be a rock star, but also I have zero musical talent. So if technology can bridge that gap, then so much the better.

But I don't think that everybody would gonna be a rock star. Right. Because while it's impressive, right at the end of the day, there's still always some music. And this is something where I honestly believe, Jenny, I will never take this away. Music, also programming, right. These are more like liberal arts than just mathematical things. And you know, there's a big difference between a solid and well constructed song and a song that touches people. And to be honest, I think this will never go away. Yeah, okay.

Maybe my rock star aspirations have been dashed once again. All right, so do you have any final advice for aspiring chief AI officers?

I think the chief AI officer for the most part needs really a heterogeneous set of skills. So I think on the one hand side, obviously it helps a lot to be knowledgeable in AI and know the technology. I don't think you have to be the topmost expert in that domain like 10 years data science, right. And then written tens of research papers like with Google, DeepMind and others. But I think it's in particular the combination of skills. Right. You need to understand the technology well enough in order to understand its implications and what that means, where the market is going and combine that with the skills on how to identify the right scenarios, do a lot of stakeholder management, come up with the right architecture, find the right messaging and narrative from a marketing perspective, determine how you can actually also from a cost, but also from a revenue or sales perspective, relate that so that you come up with a viable product. So all these various dimensions I think are critical as a chief AI officer to be really successful at the end of the day. So it really needs a, I think a well rounded profile, right. Of all these various things in order to bring those concepts together. So I can only recommend aspiring chief AI officers to really look at this topic not purely from, let's say, one angle, like let's say technology for example, but really read and educate yourself in a holistic fashion with all the various aspects in order to be successful. I think that is critically important to bring all those various concepts really together in the, in the right manner.

Okay, so really broad set of skills. It sounds like this is going to be a role for generalists, but you also need some pretty high aspirations, I guess, if you can bring all those teams together. All right, super. Thank you for your time, Philip. This is very insightful stuff.

No, thank you so much, Richie, for having me. It was a pleasure. Thank you so much for having me.

On the show, Sam.