67% of Enterprises Scaling Generative AI Pilots

Today on the AI Daily Brief, where enterprises are with generative AI as we kick off the fall. Before that in the Headlines what Wall Street Thinks About AI Currently, the AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, follow the Discord link in our show. Notes. Welcome back to the AI Daily Brief Headlines edition. All the daily AI news you need in around five minutes. Last week we did a show about the five questions that would shape the near term future of generative AI and one of the big ones was how Wall street would think about Genai Heading into this autumn season, Reuters did a little bit of a summary of the market as a whole, summing it up by saying tech market values fall on AI costs and recession fears. And basically the idea here was that in August there were two big reasons, or seemingly two big reasons at least, that companies like Alphabet and the other Magnificent Seven companies had a bit of a struggle. One was macro, a rising risk that the market was placing on the possibility of a recession. But the second was the narrative concerns around the ongoing and even escalating AI infrastructure buildout cost. One of the best examples of this was Nvidia, whose share price fell by 7.7% in the last week of August. That came after it projected third quarter gross margins below estimates and reported revenues that only slightly outperformed investor expectations. Interestingly, Reuters framed it as only met expectations even though it was actually a couple billion dollars over analysts average estimates. Still only being a couple billion over, not being some huge blowout did feel like it met expectations. One counterweight to this came from Meta that showed some ROI from AI investment based on strong digital ad revenue growth. Still, by and large, the narrative heading into the fall is summed up by this Bloomberg piece. Shorts are circling some of the AI boom's biggest question marks. Bloomberg writes, it's the story of so many stock market manias. A transformative technology juices a few companies, a bunch of more questionable outfits follow in their wake, and Wall street buys it all. Then time sorts out what's real from fake. Now this specific report looks at a group of companies that have been recently targeted by bearish research reports. Hindenburg Research, for example, went in on server maker Super Micro, ripping around $10 billion of the company's $36 billion market capitalization off the top last week. And other companies like Lumen and Symbiotik have also faced bearish reports, although perhaps nothing on the same level now. Interestingly, this report makes it clear that there is a big difference between the Nvidias of the world, even though Nvidia did fall last week and the hangers on that benefited from the tailwinds set by companies like Nvidia. Mahoney Asset Management CEO Ken Mahoney characterized what the market is doing as separating winners and losers, pointing out, quote Nvidia and the other big AI plays in the Magnificent Seven have been largely consistent with results and growth. That's what sets them apart from the likes of Supermicro, which have been sporadic. Hindenburg came back again on Thursday with another report with an even more dramatic accusation against iLearning Engine's holdings, accusing it of faking its financial figures. While the company denied it in a release, shares were still down 53%. One of the things that I've tried to make clear on this show is that Wall street repricing some of these companies is not the same as AI being the next great bubble. John Belton of Gabelli Funds put it this way, the stock market always has a way of moving too far too fast and then going through a period of digestion. For a lot of these companies, we're in a healthy period of digestion now. One interesting little bit of speculation is at what point some of these big private market AI companies will have to go public. Corey Weinberg from the Information wrote a piece this week called why OpenAI needs an IPO that outlines a set of reasons that the company might want to consider something like this in the future. That includes the incredible capital intensiveness of OpenAI's business model, the potential that private capital might be running dry or might be problematic with the US not necessarily loving a big dose of funding from a sovereign wealth fund in the Middle east, for example. And while Corey is clear that he's, quote, thinking about the following set of facts rather than acting on any particular news tip, I wouldn't be surprised if we see a little bit more of this chatter in the months to come. Staying on News from these public market companies touching AI people are digging in a little bit more deeply to Meta's recently announced AI numbers. We discussed last week that Zuckerberg had said that Meta's AI tools have more than 400 million monthly users and more than 185 million weekly users, but some wondered if that was driven by accidental usage, given that AI rolled out as just an extension of the search bar. Microsoft is dealing with its own challenges from the Wall Street Journal Microsoft rolled out AI PCs that can't play top games and there's no quick fix. The short of this story is that in the process of getting a new generation of chips that are good enough to actually run these AI applications, Microsoft has made some of its most cutting edge laptops incompatible with games like League of Legends and Fortnite, which are made instead to work with Intel's x86 chip. An independent research analyst tested 1300 PC games and found only about half ran smoothly on the new AI focused PCs. Now obviously it seems like this will likely not be all that much of a problem in the long run, but in the short run, it's another sign of the potential bumpiness of adoption that we're likely to experience. Lastly, today in the headlines, a follow up from our long read about the homework apocalypse. Axios pointed out that even beyond AI, schools are having a rethink when it comes to homework. In 2012, for example, 21% of 13 year old students said that they had no homework assigned. In 2020 that figure had gone up to 29%, and by 2023 it was up to 37%. Another study they referenced found that 67% of high school students cited homework load as a major source of stress, which went up to 80% among those doing three or more hours of homework a day. Then there's recently a bill that was passed by California's legislature that would recommend that school districts evaluate the mental and physical health impacts of homework assignments as well. One assembly member, Pilar Schiavo, said, quote, as a single mom, I only have a couple of hours with my kid at night before they have to go to bed. Spending most of that struggling to get homework done creates a lot of stress on a family. The point here is that it's not just AI forcing this conversation, although certainly AI is forcing it a lot faster. For now though, that is going to do it for today's AI Daily Brief Headlines edition. Now next up, the main episode. Today's episode is brought to you by Fractional. When we wanted to build an AI powered feature of Superintelligent, our AI tool Finder, I went straight to Fractional. The Fractional team is a group of senior engineers in San Francisco working on some of the most exciting projects in applied AI. Working with them is basically like hiring an absolute top flight AI engineering team, but in a way that you can customize exactly for your particular needs. Like I said, that AI Tool finder feature that we built with them is already a key part of the super intelligent platform and we are working on something new as well. Fractional works with everyone from startups to the Fortune 500 and to request A free consultation. You can go to Fractional AI if you want help identifying and building AI projects for your business, then I highly recommend you hit pause, open a web browser and go to Fractional AI to request a free consultation. Today's episode is brought to you by Venice. The leading AI companies store your entire conversation history and attach it to your identity for forever. That's every question you ask, every answer you receive, every image you generate, every thought you share with the machine, it's all being spied on. If you trust all the company's hackers and NSA board members that will ever have access to your AI conversations, then rejoice for you are well served. For the rest of us, Venice is an alternative. Venice is a powerful AI app for text, image and code generation that respects you as a sovereign individual and believes privacy and free speech are not only human rights, but necessary for civilizational advancement. Private, permissionless and uncensored. You can try it for free without an account. A Daily Brief listeners receive a 20% discount on Venice Pro. Visit Venice AI NLW and enter the discount code NLW Daily Brief. That's NLW Daily Brief. All one word. Today's episode is brought to you by Superintelligent, which is of course our platform that helps you learn how to use AI tools and perhaps even more importantly, gives you ideas on the best use cases that are actually going to help you achieve whatever it is you want to achieve. To recognize the end of summer and back to school, back to work. We are running our best promotion ever. When you sign up for Super Intelligent between now and the end of August using code so back your first month will be 100% free. The platform features over 600 fun, highly practical AI tutorials that get you using AI fast and with an eye to actually transforming how you get things done. We've just launched super for Teams. So if you have a group of people at your company that want to figure out how to use AI together, I highly suggest you check it out. But for those of you who are using superintelligent as an individual, once again, if you sign up for Super Intelligent between now and the end of the month using code so back you will get your first month 100% free. Go to be super AI and check it out today. Welcome back to the AI Daily Brief. In the headlines today, we discussed how Wall street is kicking off this fall when it comes to attitudes around generative AI. And so it only sense that for our first main episode of September, we're looking into AI in the context of the enterprise and specifically where generative AI sits within companies. Now, after nearly two years since the launch of ChatGPT, every quarter Deloitte releases an update to its state of generative AI in the Enterprise Report. This latest report that just came out covers the period between May and June of this year. They call it moving from potential to performance. And sum up, investment is increasing, but the clock is ticking to scale and create value. So let's discuss what their big findings were. Broadly speaking, one of the things that you're starting to see everywhere is that we're moving out of the technology novelty phase of AI adoption. Individuals and organizations are both getting more nuanced and sophisticated in terms of how they're thinking about using generative AI. And along with that is coming more expectation and less doe eyed interest in anything anyone has to sell them. Unsurprisingly, a big theme for this report is organizations wanting to move beyond pilots and into a phase where generative AI is actually driving real business value inside their companies. Deloitte sets up a bit of a path diverging in the WOOD analysis. On the one hand, as you'll see, there is a ton of movement shifting from pilots to deeper integrations. But as they also point out, quote, the clock is ticking for organizations to create significant and sustained value through their generative AI initiatives. Promising pilots have led to more investments, escalating expectations and new challenges. During this pivotal phase, C suites and boards are beginning to look for returns on investment. There is a chance that their interest in generative AI could wane if initiatives don't pay off as much or as soon as expected. It would take a whole episode to discuss why. I think that enterprises are largely in a situation where it would be much, much riskier to turn away from experimentation with AI than to accidentally overspend on it. But for now, let's just stick closely with what Deloitte is presenting here. So what is actually going on inside these companies that are testing AI? Couple big blinking banner headlines for me. First, 2/3 of companies are increasing their investments because they've seen strong early value to date. That is an extremely telling statistic that suggests there is much more than just hype going on inside these organizations. What's more, the benefits that people are finding are getting more diverse. On the one hand, improved productivity and efficiency as well as cost reduction are still both the top benefits sought by organizations as well as those most cited as the most important benefits achieved to date. But importantly, 58% reported they realized a more diverse range of most important benefits, such as increased innovation Improved products and services or enhanced customer relationships, meaning there's a certain growing sophistication that's not just about ROI measurement. Something that we discuss a lot at Superintelligent is the way in which the AI transformation is proceeding through a sequence of steps. The first step, the low hanging fruit, if you will, is personal productivity among employees. This is where a ton of the experimentation is happening now, when some individual is just figuring out if their work happens faster, if they integrate ChatGPT with their email writing process. That's sort of this first step. The second phase is all about unlocking new opportunities that weren't possible before. Totally reconsidered ambition, for example, around a marketing campaign, because things that were simply outside the capability set of the participants are now firmly within their grasp. This, to me is what this 58% reflects when they see increased innovation, improved products and services. This is more than just personal productivity, it's wider benefit. Then of course, there's an even higher level stage, which is organization level transformation. Some organizations have gotten here, but by and large this remains something for the future. This is all about how we might see organizations redesigned and reimagined from the ground up based on the new capabilities that AI enables. I think where you're likely to see this happen first is actually with entrepreneurs who create totally different types of organizations and model how you can do more with less or much more with the same. And slowly those types of transformations will find their way into the enterprise as well. In any case, again, two big banner headlines here. Two thirds of organizations are increasing their investments in Gen AI because they're seeing strong value, and nearly 60% are seeing benefits that are not just about productivity and efficiency or cost reduction, but are about innovation and improved offerings. There is lots of challenge here though as well. 70% of the organizations surveyed said that their organization has moved 30% or fewer of their Gen AI experiments into production. In other words, there does seem to be a big scale barrier. Those barriers come in a few varieties. One is around data. 55% of organizations told Deloitte that they're avoiding certain genai use cases because of data related issues. Some are worried about regulatory uncertainty. Three of the four things holding organizations back from developing and deploying genai tools are risk regulation and governance issues. And then of course, there's the old chestnut of ROI measurement. More than 40% of respondents said that their companies are struggling to define and measure the exact impact of their genai initiatives. And less than half said they're using specific KPIs to measure performance with many standard measures of success not currently being applied. There is also definitely a broad sense here among these organizations that they do not feel prepared. Only 45% think that they have the appropriate technology infrastructure, only 37% said that they feel they have the appropriate strategy, only 23% said they have the right risk and governance Systems and only 20% said they have the right talent. For the sake of this report, Deloitte decided to hone in on two particular areas around data and governance and risk and compliance. The TLDR on the data side is that data has come even more to the fore thanks to Generative AI. 75% of organizations surveyed said that they've increased their tech investments around data lifecycle management. Data related concerns are frequently cited as holding back Genai implementations and when it comes to risks and regulation, as we said, these are meaningful hindrances as well. And this is me editorializing a little bit, but likely to get more so, especially as the US starts to deploy state by state type legislation like SB 1047 if it gets signed by Governor Gavin Newsom in California as opposed to federal regulations, which might be a little bit clearer now. Of course, even beyond these specific risks, there was never going to be a period of attempting to move from pilots to scale where the challenges of measurement were going to start to become a big issue as well as the story that this report tells is a lot of experimentation around AI ROI tracking. 48% of those surveyed have used specific KPIs for evaluating genai performance, 38% have attempted to track changes in employee productivity, 34% have tried to track non financial benefits. Only 6% are doing none of these types of things. So clearly there are experiments here even if there aren't really best practices yet. My best guess is that we are going to see a continued period of experimentation around ROI measurement. I think that Deloitte is right to call out that there is a world in which the difficulty in measurement leads to a hindrance in organizations abilities to actually scale these AI pilots. Clearly we're already seeing some of that. However, I believe that the continued pressure on basically everyone from a director level position up to have some sort of AI adoption strategy will actually produce an incredibly fertile set of experiments around AI ROI measurement that might get us to some of those best practices a little bit faster than we might anticipate. If you are interested in going deeper and checking out past reports, I will include a link to the overall webpage for this on Deloitte.com for now. Thanks to the team over there for preparing another interesting report. And thanks, as always, to you guys for listening or watching. Until next time, peace.