EP 317: How Enterprise Companies Can Escape the AI Inertia

This is the Everyday AI show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business and everyday life.

It's no secret that AI implementation at the enterprise level is pretty hard, right? Especially if you work at a company with thousands or tens of thousands of employees. And maybe you're seeing these smaller companies get all of these great gains from implementing generative AI and you're wondering how can I escape this AI inertia? How can our big enterprise company with so many moving parts pick up some steam in this AI world to keep up? Well, luckily for you, we have some answers for you today as we have a leader at enterprise AI at Microsoft joining us the show joining the show today. So I'm extremely excited for today's show. So before we get started, we're going to start as we do going over the AI news for the day. So if you're brand new here joining on the podcast or the live stream, thank you Everyday AI. This is for you. We're a daily live stream, podcast and free daily newsletter helping us all learn generative AI so we can leverage it to grow our companies and career. So go to your everydayai.com and sign up for the free daily newsletter. But let's dive in to the AI news for today. A lot going on as always. But first, Meta is withholding new AI models from the European Union amid regulatory uncertainty. So Meta will not release its next multimodal AI model in the European Union, citing unclear regulatory guidelines, according to Axios. So this decision could escalate tensions between the US tech giants and EU regulators, highlighting a trend where American companies are withholding products from European markets. So Meta's multimodal Llama model, which integrates video, audio, images and text, will be available in other regions such as the US but not in the EU due to that regulatory unpredictability. So a text only version of Meta's LLAMA 3 model will be available in the EU, indicating Meta's selective approach to product releases in the region. So Meta had planned to use publicly available posts to from Facebook and Instagram users to train its models, and had informed EU regulators well in advance but received minimal feedback. All right, our next piece of AI news for today. A quarter of Japanese firms have adopted AI, while many remain hesitant. So a new study shows that nearly 25% of Japanese companies have integrated artificial intelligence into their operations. So that's according to a Reuters survey conducted by Nikki Research. Despite this, over 40% of firms have no plans to adopt AI, highlighting a divide in the technological adoption across corporate Japan. Yeah, that's right. I didn't stutter there. 40% of firms, according to this small scale study, have no plans to adopt AI. Wild, right? So the study included responses from about 250 out of the 506 companies and it was conducted earlier this July. So key motivations for AI adoption included addressing worker shortages, reducing labor costs and accelerating research and development. All right, our last piece of AI news for the day, at least for the podcast, some new healthcare and AI news out of Stanford. So a recent workshop by Stanford. Hi, sorry, Stanford Health Artificial Intelligence, or Stanford Hai as it's called, has highlighted significant gaps in healthcare AI regulation. Pointing out minor tweaks won't suffice. So some findings from this survey, it found that only 12% of thought leaders believe healthcare AI should always have a human in the loop. Wow. 12% only believe should have a human in the loop. That's kind of, that's a shocking number there. And Also a strong 58% say human oversight is unnecessary with proper safeguards, while 31% support human supervision most of the time. All right, so there's a lot more, not just AI news, but just everything that's happening in the world of AI. New fresh finds from across the Internet, tools, software, all that. As well as in the newsletter today, go find out why OpenAI researchers have developed a model where it competes against itself. All right, so enough about that. You didn't tune in to go over the AI news. You are here to learn about how enterprise companies can escape the AI inertia, right? I. I can't imagine. Right. I run a small business. I can only imagine when there's tens of thousands of employees how AI adoption actually works. So we're going to have some answers today. So I'm extremely excited to have our guest for today. So please help me welcome Rajama Krishnamurthy, the senior director of Leader of Leader Enterprise AI at Microsoft. Rajama, thank you so much for joining the Everyday AI Show.

Good morning and thank you for having me. Jordan.

All right, I'm excited for this one. And hey, shout out to Rajama. She's one of only like a dozen of people who have joined us early west coast time. So thank you for waking up early with the rest of our audience. And hey to our live stream audience. Thank you for joining us. Tara and Dr. Dr. Scott, Ernesto, Danny, everyone. Danny, sorry, if you have questions, please get them in. But Rajama, let's just start. Tell us a little bit about what you do in your role in leader enterprise AI at Microsoft.

I have been in enterprise technology for most of my career. I was in HR technology in some companies on the east coast before I moved to Microsoft 10 years ago. Over the last two years I have moved into the working in AI and I started with working in AI leading an AI center of Excellence which is just something to pull together people to move us out of the inertia. And I'll talk about it a little bit more later. And then I also now after doing some of that work, I have moved into a role where I am actually a product leader in a product that is going to be in your desktop in your desktop somewhere near you very soon. So that's what I do. And when I'm not doing that, I'm also a professor, adjunct professor at I NYU teaching technology and AI.

I, I love this. So not only someone helping enterprise companies use AI at one of the largest companies in the world in Microsoft, but also someone teaching it. So I think we're going to have a lot of insights today. But maybe Rajma, let's just start at the end. Let's just start and answer everyone's question right now. What are some of the biggest reasons that enterprise companies. Right. Because first of all, number one, AI is not new AI itself. Right. It's been around for many decades and generative AI, you know, you could say it's kind of new ish. Right. Couple years. But what is one of the biggest reasons that enterprise companies still today in 2024 haven't fully implemented generative AI into their companies?

I think cost is a big factor and the, and the understanding of what the ROI is against the cost is also a big factor. So nobody has said oh, there's a lot of studies or it's going to reduce 40% of, you know, it's going to increase your efficiencies by 40%, productivity by another 30% and so on. But nobody has the evidence in front of them. They can't touch and feel it. So people are finding it difficult to put the money where it is still studies and nobody has proved it out. Finally. The other problem is the education to be able to having people know what AI is about, where they need to implement it, how they will get what they need to get out of AI is something that's another thing that stops them from thinking about this space.

Yeah. And it's, you know, just in about a 45 second answer there, I mean what you just said right there, that's the million dollar, the billion dollar or the trillion dollar answer that people are searching for. Right. Like trying to understand the return on investment and the cost and education. I mean, those, those pieces are. So I'm curious even for you. Right. And I've had a handful of other people from Microsoft and I love asking them the question, right. Because people who are building Copilot and all these great AI features that we use. I just had an episode this week talking about the Microsoft Edge browser and how much I love it. But what have you even personally seen yourself or maybe in your department, in your experience actually implementing generative AI from one of the largest companies in the world, how have you started to see, you know, some of those things? Productivity and efficiency, even if you don't have a specific stat. Maybe. Can you talk about what your experience has been, not just building the products that we're all using, but actually using them and benefiting from them as well?

Sure. Let me talk about something that we all use daily and so that it can, you can relate to it. You know, we all use our emails daily. I know you talked about Edge browser and you're also. I use my teams daily. This is how I do my work. The other day I had to go find some document that one of my colleagues had handed over to me maybe three months ago or something like that. All I had to go in was go into a copilot and ask, hey, can you give me a list of documents that Jordan handed over to me in the last three months? It gave me all the list and it took me less than 10 seconds, 20 seconds to go find what I needed to. Instead of combing through my emails, combing through all the documents, which would have taken me at least another 10 minutes. Just imagine, just that little act of doing that has been so valuable to me. I use Generative AI or Copilot daily in my work. Whatever I do, it gives me the summary that I need to, what I need to focus on today. So whole, it's the personal assistant that I don't have to pay for. And it helps me kind of get through my day in a much more easier way. So if you talk about productivity, it gives me the time to kind of go do my real work, which is I need to think about strategies, I need to think about the products that I'm building, I need to think about where I need to take that. And instead of like worrying about which document that I need to go find in the, in the next 30 minutes.

Yeah. And you know, I'm sure that, you know, you and, and your team There spent a lot of time working with these large enterprise companies, you know, asking some of these same questions and trying to find some of these same solutions. You know, let's just say assuming companies have Microsoft 365 copilot, right? Like where do you start with enterprise companies? You know, obviously you have to have conversations around governance and data security, but let's just say after companies have that piece figured out and they're like, okay, where do we start? Where do we start seeing some of this roi? Where do we start seeing some of this increased productivity and efficiency? Where can you point enterprise leaders on, hey, this is a great place to start to see some of that return.

First of all, I don't think the AI is the destination. AI is your enabler, it's your accelerator. So we need to make sure that you're not looking at which how do I implement AI? Instead you're actually focusing on how do I, what is my business strategy? What are the business problems I need to solve? Where do I need to be in the next two years, three years, five years? And how, where does, where can AI help? For example, if you want your customers are not happy with your customer support or if you are having problems with, let us say even internal employee shared services, where do your problems lie and how do you actually kind of go after it? And I think those are some of the things I would focus on the business strategy. So if you say, if somebody explains that, oh you know what, if I can get the customer support thing solved, I can actually start thinking about focusing on my marketing problems, focusing on creating new products and moving my company forward, then go after it. Look at these are the, to me having things like customer support or employee support. And those things are low hanging fruits in your company. And these are the easiest things to fix because AI can read your content that an individual was doing, can summarize answers, can personalize solutions for whoever is asking those questions, these could be where I would start. These are not sensitive use cases, even by EU AI act reading. So this would be some things that I would absolutely encourage customers to think about.

You know, one thing that I'm always thinking about is where companies are at now, right. Presumably most enterprise companies are have either implemented generative AI across their processes or are somewhere in that process. Right. Whether that's, you know, three month, six month year long process, I'd say most enterprise companies are somewhere there. What's your kind of message or what's your thought on companies that are maybe still even on the fence? Especially you know, Larger companies when we're saying, you know, maybe, I don't know, Fortune 500 or Inc, 5,000 companies that are still on the fence. Right. And I kind of read that news story there about, you know, these 40% of Japan companies that are saying, no, we're not going to use it for those big companies that are still on the fence. Is that a smart place to be? And I know it's kind of crazy to be asking that question here in 2024, but what are your thoughts on those companies that are still like scratching their heads and like, let's, let's be slow, let's see what everyone else does.

I think that it is not just true about Japan. There's a whole lot of companies in the United States as well that are still on the fence and they're still thinking about maybe we will let the others go in front, make the mistakes that they need to make and then I will follow and be on the right path. I think first of all is here to stay. It is part of how we will do work now and in the near future and in the far future as well to get off that. And you're already a year or two late in the space of generative AI. If you haven't done used AI as machine language or anything else that AI was already in the enterprises for, then you're already too late in using those tools to improve and accelerate and enrich your experiences, your customer experiences, or just kind of getting through a day for your marketing professional or your legal professional or your HR professional in your company. So to kind of answer your question, remember those days when we were all implementing ERPs and they said it's going to take six years before you realize what the ERP will do. There is no magic with AI either. When you start implementing these, it will take a few months, a few years for you to see. In some cases you will see results very quickly. In some cases it's going to be a little bit of a slow medicine that's going to cure a lot of things in your company. There needs to be patience in the long haul ones and there needs to be excitement about the short haul ones and moving forward, which means the excitement comes with a little bit of risk taking and understanding the risks while you take them and so on. So my advice to most of these companies are get off the inertia and kind of move forward as quickly as possible because you are already, if you haven't started, you're already behind and you don't want to be behind your colleagues or your competitors in your space because they will certainly have an advantage over you in all kinds and all forms, whether it is retaining the right talent because they have better experiences or retaining the right customers because they have better support and care than you because of AI.

Yeah, and I think those are just some great pieces of advice there. You know, one analogy that I always use, it's probably a little too much as I say, like this is like, you know, companies maybe in the early 2000s that were like, we're not going to use computers or the Internet. Right. We're just going to keep doing things the way it was in the 80s. Right. Nothing was wrong with that. So let's just say though that there was someone on the fence, someone listening, maybe on the podcast right now, a decision maker at a big company. And they're like, all right, we're gonna, we're gonna do this, we're gonna go all in, right? We, we need to compete. We should have done this a year ago. But let's, let's go. What are those first steps that people need to take, especially in big companies? Because I, I personally think sometimes it's the smaller and the medium sized companies that have a huge advantage here in enterprise. It's, it's difficult. Where do those companies start when they finally get off the fence?

I would say first of all start putting a team of people together, a like minded set of people together that are passionate and I wouldn't say experience because all of us are getting experience as we speak, but I would call it people who have an innate passion for learning this space and wanting to do some work. Have a good understanding of your business strategy and have been and of your business. Pull those people together. It does not need to. It would be lovely if you can have it at the central level with your CIO and your business leaders kind of leading that, empowering that team. Or even if you want to start small, go into one of your verticals. I would call it vertical in an enterprise would be an HR or customer support or sales or whatever else I would say that takes to put a small band of people. Start looking at this in two or three pillars. The first pillar being the business strategy. So like I said, AI is not the destination, it's your accelerator. It is the rocket on which you are going to go sit and solve your zoom into the solving problem space. So put your business strategy, understand what are the business problems that could be solved by AI to solve those business problems, look at from an architecture perspective, do you have the data to solve the problems, do you know what the models that you will use? Do you have the infrastructure to go after it? The third pillar is obviously education. Education is not just about doing and executing and getting results. It's also about adoption. So ensure that you're educating your various levels of organization that needs to be educated about how AI is going to help their work, how they need to future proof their own careers, because AI is here to stay. So if there is reluctance on their side, we can sell that by making sure that they know that this is not just about the company, it is also about them. Because the next job they want to find is going to ask for AI skills similar to how today, even about five years ago people were asking for Excel skills and computer skills and so on. And for engineers it was about cloud skills. Now it is all about AI skills. Excuse me, there is that. The third pillar I would call is education or culture, the adoption and so on. Then the last is more of a horizontal pillar. If pillar could be horizontal is basically the conversation about responsibility. Everybody needs to be educated about how to use AI responsibly. How do you think about it, how do you go about it from the very inception of what projects that you need to choose to, how do you test for it, how do you put it into production, how do you make it explainable and transparent to whoever is using it? Then how do you monitor that for any ethical aberration? I would say that all of these four things get that started within the strategy. Start thinking about the low hanging fruits that you can go after you have the data ready, you can easily get a model, you can easily ground it to whatever data that you have and you can start the ball rolling on those I told you about. Anything that can read your content and answer questions is the most easy space. So think of areas where you can do it very easily. Whether it's customer support or employee support, or anywhere else where you want to summarize or create content. Go in and get that done as quickly as possible.

Are you still running in circles trying to figure out how to actually grow your business with AI? Maybe your company has been tinkering with large language models for a year or more, but can't really get traction to find ROI on gen AI. Hey, this is Jordan Wilson, host of this very podcast. Companies like Adobe, Microsoft and Nvidia have partnered with us because they trust our expertise in educating the masses around generative AI to get ahead. And some of the most innovative companies in the country hire us to help with their AI strategy. And to train hundreds of their employees on how to use Gen AI. So whether you're looking for ChatGPT training for thousands or just need help building your front end AI strategy, you, you can partner with us too. Just like some of the biggest companies in the world do. Go to your everydayai.com partner to get in contact with our team or you can just click on the partner section of our website. We'll help you stop running in those AI circles and help get your team ahead and build a straight path to ROI on Gen. Y'. All. My, my fingers hurt from typing so many notes from, from Rajma here. She's dropping so much great adv. So one thing that I picked up in there is talking about even your next job, right? AI skills are going to be a requirement. And you talked a little bit about being also a professor at NYU of AI. With that in mind, what would you say, especially for students? Because I don't know and let me know if I'm wrong here, but I think that maybe some of the skills that we've built over the course of decades, people who are in the middle of their career, they've really been rewarded for their institutional knowledge, right? Oh, they know something that very few people in the company know, so they get rewarded for that. But now when we have very capable models, right, such as, you know, GPT4O that you see in Microsoft's, you know, co pilot now, sometimes that that knowledge isn't as exclusive to a certain few, our students or our people who are working AI first, working AI native, are they at an advantage for their careers that we maybe haven't seen or understood yet?

No, I don't think so. And the reason is you have the institutional knowledge you had or you have, you have been using that to think through. I mean, you're not going to talk about institutional knowledge. You're using it as the foundation to make decisions. So now somebody else is going to provide you with all that knowledge. And if you have to make the decisions so that the thinking part of what you are, the intuition part of what you are, the instincts and the insights that you are able to generate with that, that still works. One of the things that I tell my students is that we have been across any of the professions that we are working with, we've got to start thinking about how do you bring that in, which is how do you read data and make decisions and that is what. So that kind of help, you know. Yes, I have used this analogy quite a few times, but basically think of the AI Right now, as your golf caddy, it's going to bring you all of that, you know, I don't play golf. So in any case carries your bags. It'll tell you what you know, iron to use. You've got to pick the right iron to use because you know, this is what it's going to take to kind of go move forward with the next shot. So that is what you need to know. You need to be able to look at that bag full of. Even if the caddy is suggesting to make your decision, that decision making, that thinking power will always remain. And so we need to teach our students how to read data, how to take other people's suggestions, how to, and how to bring in your own thinking part of it, how to be intuitive about your decision making from all of that information that will be laid in front of you, basically. But the best part is you don't have to have the grunt work to go find the information. It's going to be made available to you. I love that part. I'm really jealous of the next generation of workers because this is going to be, you will be doing the fun part of your work mostly and you know, the grand part of the work will be done for you. That's going to be very exciting.

Yeah, that's a good point. Is the future of knowledge work, which I know someone, Bob here is talking about. And, and you know, who knows what that future will be. But I think you kind of just answered it right there where, you know, hopefully spending more time, I like to say more time on the meaningful and less time on the mundane. Right. Those mundane knowledge tasks. And I love that example that you gave there of AI is your golf caddy. Right? I'm trying to learn golf. I'm terrible at it. But I think that's good, that's a really good example. But, but getting back to enterprise and, and getting back to implementation and this, this whole concept of, you know, getting over this inertia, so to speak, is this one of the first times ever? I think. Well, and, but I want your, your thoughts on this. I think historically it's always been the enterprise companies that, oh, only they could afford a certain, you know, software solution. Only they could implement and have the best technology. But now it seems like for the first time anyone with, you know, $20 or $30 a month can go get Co Pilot Pro as an example and, and use a similar technology that the biggest companies in the world are using. Is, is that true? Right? Is this maybe. Do you think it's one of the first time that the playing field is actually kind of level. And then if so, maybe on the flip side, what's some advice for some of those smaller companies to chase after those enterprises?

I, I think, I, I certainly think technology per se actually democratizes a lot of these things. I mean, whether it was the cloud, whether it was the Internet or now AI more so than ever. And for the smaller companies that can move fast and that have the advantage, move fast, get the advantage, you can compete with the big ones there because you have made most of the grunt work go away in your area and given your team, your people more time to think, more time to create, and more time to work in spaces that they didn't have before. And that will certainly put you in a great advantage more so than ever. And I can, I am already seeing that in smaller companies. I sit on a few advisory boards on, you know, smaller startup companies or even on a, you know, VC firm. And I see that there is just that excitement and I see how quickly they are adapting to it. And not only AI companies, but also companies that are not AI kind of bringing AI into their space. Some of those examples are really beyond me. For example, there is a company that just gives cancer care to patients and how AI can help nudge on the right times, be a body and so on. Things like that that were never thought so before can now be brought into your areas that you had never thought AI or even technology can help. You always thought just human beings will be there to help. But these buddies are helping, you know, people who are vulnerable, who are suffering through certain things to be having some companion that's just kind of, hey, did you take your medicine today? Or, you know, did you go visit your doctor today? Do you need any help? Do you need somebody to come and help you, kind of take you to the next appointment or things like that. You know, people are not, you know, don't have to just rely on somebody, some other human being, but they also have, obviously in any such cases, you want humans in the loop, you want human care, but you also need the structure that comes with some, you know, in cares like this will can be provided by AI. Basically, yeah.

And Rashoma, getting back to something that we talked at the very top of this show, right when we said, hey, let's, let's kind of fast forward to the end and talk about what are some of the reasons why there is this AI inertia and why it takes so long for enterprise companies to gain momentum. And you know, some of the things you said is cost and understanding the roi. But real quickly I wanted to talk about the educational side because I think that's ongoing. Right. And at least from my perspective, it can be hard for companies to, to become educated when the pace of development seemingly is impossible to keep up with. Right. I follow AI every day. I have a daily AI live stream podcast. It's hard for me. So I can't imagine for people that aren't spending hours a day. But then, you know, related here. Tara's question is what are some examples of training in organizations that you've seen successful? So yeah, how can companies both educate and train their employees to keep up?

I would say the, the, you don't know, but many of the rebels in your, in your organization are already using ChatGPT. They're just using it on the side. They're coming back and doing that. So that's a great thing. I would encourage it. So don't discourage that in the first place. But how many of us really prompt do we actually use prompt? No, mostly we use search. We were using search with Google and Bing. Now we are using search with ChatGPT and Gemini and publicity or whatever the choice of models that you're using. So what I suggest is that start there, start having some basic courses about prompting. How do you use prompt? How do you talk to AI? How do you, what are the options that you have in, you know, talking to AI? So that's one. Another thing is across the board, each of the discipline do not need the same kind of learning. For example, an engineer would learn different needs to learn different things about AI, but product leader wouldn't learn something else. An office administrator would need to learn something else and so on. So there's a, there are different learning paths for different people. So figure out what those learning paths need to be. There is enough online courses across the LinkedIn and other places which are just free. And then there are, you know, obviously paid courses available for more in depth learning about data structures and deep learning and other things that you want to have many of your engineers learn. I would say first of all, have everybody learn prompt engineering. Have everybody learn what ethical use of AI is. It's very important because both the users and the creators need to know ethics about AI and then create these multiple learning paths based on the discipline. One size doesn't fit all basically.

So so much good advice here. This is, this is going to be hard. You know, I, every single day I, I go back and you know, write this newsletter to recap. This is going to be a hard one. Rashma, you've given us a lot to think about. But as we wrap up the show here, you know, we've gone through a lot. We've talked about, you know, even your simple daily efficiencies. You know, what companies who are still on the AI fence should do kind of these, these three pillars to AI success at the enterprise level. But, but as we wrap up here, maybe what is your one most important takeaway? For those companies that are still, you know, kind of trying to get over this AI inertia and to pick up momentum at the enterprise level, what's your one most important takeaway?

I like I said earlier, AI is here to stay. You can't avoid it. If you avoid it, you'll be the, you'll be behind your competitors. Start thinking about spaces where you will feel the most comfortable and just kind of move on. Move out of your inertia. I would say don't worry too much about the cost and the ROI just yet. Start experimenting because with learning will come comfort, and with comfort will come the next steps in moving into bigger spaces. There are some moonshot spaces in your area that you will get to, but you can't get to it unless you know what you're doing in the more smaller and more comfortable spaces. Because you've got to learn this space and then you can go solve the big problems in your enterprise with AI.

Amazing. Amazing. I mean, thank you so much, Rajma, for joining the Everyday AI Show. Hopefully our audience just gained a lot of insights from a leader, literally a leader in the world helping us all, you know, use AI there and Microsoft and also an educator. This was a great show. So thank you so much for your time. We appreciate it.

Thank you very much, Jordan, for having me. Have a wonderful day.

All right, thank you. As a reminder, everyone, please, if you haven't already, go to your everydayai.com sign up for that free daily newsletter. This is going to be a tough one to write. It's going to be a whole lot of information we're going to be sharing as a recap from today's show as well as some other resources. If this was helpful, please please share it with a friend. Subscribe on Spotify or Apple Podcasts wherever you're listening. And no matter what, please join us tomorrow and every day for more Everyday AI. Thanks y'.

All. And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit youreverydayai.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.