Why 95% of Enterprise AI Pilots Are Failing (and Why That’s a Good Sign)

And right now we're in the let a thousand flowers boom, spend a lot of money, throw a lot of spaghetti against the wall and see what sticks. And that's normal when you get a new technology, but eventually people have to find value.

I don't think money equals speed equals success. Obviously that equation breaks all the time, but in more obvious areas, specifically within AI, it feels like large amounts of capital allow you to try a bunch of things at once and maybe you can make more mistakes. But it's still unclear what works, what doesn't, because we don't even know who the winners are.

I mean, I think about some of these aqua license companies that seen that, you know, the the founding team members and early employees walked out of the door and those companies I think are searching for a second act for some amount of relevance. And are we going to have a repeat of this sort of unicorn hangover where there's a bunch of well funded companies that are casting about for something to do, but with a lot of cash on the balance? She.

Welcome to the Learning Corner by Precursor. I'm Mia Farnam and I'm joined by our GP and Managing partner Charles Hudson, where we will be talking through the articles and Goodreads that caught our team's attention over the past week. Our first topic is a piece written by Rebecca Caden from USV making the case for founders and team leaders to hire the experimenters. This strategy varies greatly from the tried and true model of startup hiring, when taking a bet on someone with a proven track record of success in a similar environment would nearly always win out as an option. Rebecc says that locating these experts lately has become incredibly difficult with the reality that expertise in something in today's AI world might not have existed as a concept six months earlier, she said, quote, underlying this change is a deeper shift in where knowledge lives. Increasingly it's moving from our heads into large models and the tools they power access is no longer the constraint. Execution speed and creativity are the new moats. I think Rebecca did a great job summarizing the hiring conundrum facing founders today. Last week we discussed how difficult it is to hire the obviously good founding engineer. This week it feels like Rebecca is encouraging founders to broaden their definition of what quote unquote good looks like in today's market. Was there anything else in particular that stood out to you from this piece?

I also thought it was good advice for VCs too, not just for founders. If I think about the evolution of what I've seen happen in AI investing, I Think in the beginning, everyone's like, oh, I know the answer, let's go find people who have expertise in a vertical and then have them apply, like those people who are going to win the, you know, the insurance AI company started by the person with tons of insurance experience. And then I think what people realized was like, well, maybe in this AI world the context and experience in a category is maybe less relevant than it was in the SaaS world. And I think pre AI, that expertise was helpful because building things took a long time. Someone with previous experience would know what to build and what not to build and would have those relationships. And, you know, getting in the door at a lot of those enterprise places took relationships. And now with AI, lots of companies have big budgets. And so getting in the door, even as a relative newcomer is easier than ever. I wouldn't say easy, but easier than ever. And I think the fact that most of the incumbent products didn't have AI capabilities as quickly as their customers wanted them also created an opportunity for newcomers to show up and just say, hey, I'm new to dental, I'm new to pet care, but I've got this AI powered thing that can meet your needs. And so I think there's something comforting in VCs believing that like, oh, the answer is just fund the people who already have the relationships and context in this industry and they'll be the winners. And I think that hasn't worked. And then we've gone through various iterations and I think we've landed in this place where I think the answer is like, nobody knows. Nobody knows what actually matters in terms of founder pedigree and attributes. And I think we've ended up in this world where I think most people believe the only investable moat, whether it's at an individual level for keeping your job and getting promoted, or as an investor investing companies. It seems to be that it's like execution and speed. And I'm just like, but how do you measure those things up front?

Right.

Obviously we have a point of view about how you can try to get your arms on it, but it's really, really tough. And that's a very different moat than we're used to. I think if you abstracted away AI and asked most investors, would you want to invest in a category where there's very little technical differentiation, there's a lot of competition, and the only potential moat is speed of execution and ability to figure things out? I think most people would say that sounds like a hard way to make money.

Well, that was like my Next question. Being able to take those quick turns and move with speed is just obviously critical for founders. How defensible can your company actually be if your current success is truly driven just by speed, which I know we've talked about this internally, I don't think money equals speed equals success. Obviously that equation breaks all the time, but in more obvious areas, specifically within AI, it feels like large amounts of capital allow you to try a bunch of things at once and maybe you can make more mistakes. But it's still unclear what works, what doesn't, because we don't even know who the winners are.

For the first time, I checked one of our portfolio companies who told me that they were making a pivot because they're in a regulated industry and some of their customers were like, hey, your competitor has raised a lot of money. At a minimum, we know they're going to be around for the next two to three years. So if we work with them on this product, we don't know if it's going to be any good, but we know that they'll be around. And I was like, that's like an actual tangible benefit of having a bigger. But the truth is it doesn't mean that that company's necessarily going to build a great product. The money gives them a longer lease on life. It means they'll be. And it's interesting, in SaaS in 2021, we ended up with a bunch of companies that had raised too much money relative to their ability to scale. And I wonder if the same thing's going to happen in AI. We're going to have a bunch of really well funded companies that don't have any idea what to build or the premise that they started building against turns out to not be that interesting. I mean, I think about some of these Aqua license companies that we've seen that, you know, the, the founding team members and early employees walked outta the door and those companies, I think are searching for a second act or some amount of relevance. And are we going to have a repeat of this sort of unicorn hangover where there's a bunch of well funded companies that are casting about for something to do, but with a lot of cash on the balance sheet and I.

Assume because they have capital, they'll continue to make changes, keep pivoting, etc. Because they have the room to do so, and so we'll continue to burn. I don't know if these companies that have tons of money will just reach profitability one day and be like, this is good enough. I think that they don't fall into that bucket that might find that route attractive money.

Mia I talked to one of our portfolio company founders yesterday and he said, hey, I've raised this money, I'm trying to figure out what the next milestone is. And we went over this thing. I said, look, nobody really knows in this environment you need to grow really quickly. And I said, we're in this world where there are two binary paths for startups. One is try to hit the high growth venture playbook milestones, hope it's enough to catch the eye of somebody and keep going, or be a rule of 40 company that's profitable and grows at a good rate and hope that you have something that feels more like a PE exit, that in between zone feels like a horrible place to be right now. And I told the founder, I said, based on the money that you've already raised, you've already chosen a path, whether you realize it or not, based on the way you've capitalized the company. And so I don't think most of the investors in these really well funded companies want the money back either. So I think we're going to end up in situations where there'll be companies that have raised a lot of money, don't have a clear idea of what to do. And I think the VCs are going to say, well, find something to do. That's your job. You raise the money, you have it. I would take it back in a pinch, but I'd rather not. I'd rather you find something worthy of the capital that you've raised.

Right. Because those companies are typically the ones that have raised this larger valuation bubble, probably have the big announcements about it. It's a loss for a loss in quotation marks for everyone involved. From just a media standpoint, if they decide, oh, we're actually going to cool down and not go after this like massive extreme exit that we promised everyone. I think people are also anchoring themselves in those roadmaps or those outcomes so much earlier. We're going to talk about more in one of our topics, but if you come in and your seed valuation is 70 million, then you've already decided and you probably haven't even built traction yet, which is crazy and I don't know whose fault, like I could point fingers at, like so many different players in that dynamic. One thing I liked about Rebecca's piece is that I think she is calling founders to just make sure like your head is on a swivel always. If you find yourself reaching a dead end, be able to change quickly. And actually I find that that's been something that we've seen in our best founders throughout times. Being able to kiss a bunch of frogs and move on quickly, that's been what separates people from success versus reaching these massive plateaus.

It's also been nice. I feel like, especially in the SaaS era, we had gotten into startup creation by playbook. These were the metrics, this is the spreadsheet. This is the only way to do it. This is the only technique, these are the only marketing channels. And it was just getting really boring.

Yeah.

And I think it's terrifying, but interesting that now there aren't actually that many playbooks for this era. And starters are always asking me, what's the playbook? I'm like, I don't know, you gotta go create it. It doesn't really exist. People are like figuring this stuff out as they go.

Pricing models for these companies that have raised tons of money haven't been figured out and they continue to change every 10 months. Which is crazy and would have never happened in the SaaS world because the entire basis of that successful industry was based on the one pricing model that had worked well. For our next topic. This past week, MIT's Nanda Initiative published a report analyzing around 300 generative AI deployments across enterprise customers. The study found that 95% of pilots were failing to deliver on promises of ROI and revenue growth. Interestingly, the high failure rate wasn't primarily due to the quality of the AI models themselves. Instead, interviews with company leaders pointed to flawed enterprise integration, especially in bridging the learning gap for both tools and organizations. The data also highlighted a resource allocation problem. Over half of generative AI budgets were being directed towards sales and marketing rather than toward lower hanging fruit like back office automation. One standout finding companies that purchased AI tools from vendors had a success rate of around 67%, while those that tried to build internally succeeded only about a third as often. So while the report highlights some successful deployments that make up the remaining 5%, it also notes that many surveyed companies were hesitant to share their failure rates. My question for you is how long do you think it will take for companies to realize that their investments aren't yielding returns, especially when so many are focused on building in house solutions?

I read this and I think I had a different take on this article than a lot of people. A lot of people, at least on social media that I saw were like, aha. I knew this was all a scam. I'm like, I don't think that's what I take away from this. What I take away from this is those failures are the seeds of future success. I build a lot of things internally using Claude desktop. Most of them end up not being that useful. But every not useful thing I've built has informed the next thing I've built. And so I think this is the creation process. And if you think about other tech waves, you've seen similar things. The technology comes out and people say, well, we've got to own this capability, we're going to do it ourselves. And frankly, in the beginning there also weren't a ton of great vendors for some of these products. So I think the combination of some companies wanting to build and own the tech themselves, kind of relative immaturity from vendors and probably overblown promises for what the tech could do. And look, I'm sure some of these generative AI pilots were just like, hey, let's just have ChatGPT write all of our markets marketing copy. Unsupervised, untrained. Yeah, it's gonna, it's gonna stink. This isn't surprising to me that people who were not skilled in using these tools, who are discovering them and applying them for the first time, were largely unsuccessful. That is like not very surprising. The thing that I think is interesting is I'm sure all of those people who have failed projects, whether they admit to them as failures or not, learn something in the process. Either, hey, we shouldn't build this internally, we should go find someone who can provide this tool. Or hey, we. To your point, Mia, we went after sales and marketing, but there's a ton of customer support and other low hanging fruit on the cost side that we could tackle and maybe that's the right place to start before we go here. Or gee, maybe we didn't succeed because we don't have the internal capabilities, but we want to have them and we need to go find people to work at our company who have more familiarity with these tools. And I think the average employee in these companies has more familiarity today with these tools than they did whenever the survey was taken. So to me this is a good sign. I think it's going to make people take these AI deployments more seriously. It means they're going to be more, I would say, circumspect around expectations. I think it means they will throw out things that aren't working and they will invest in things that will. And to the extent that AI delivers on its promise, it's only, it's only going to work when companies are getting real value from deploying the technology. And right now we're in the let a thousand flowers bloom, spend a lot of money, throw a lot of spaghetti against the wall and see what sticks. And that's normal when you get a new technology. But eventually people have to find value.

How many CEOs of large enterprise companies understand that? And so at the end of the year they might see these quote unquote failures, but then have the same perspective that you will and be like, continue to keep trying, continue to keep spending.

I think the pressure from boards is going to continue because you know, I mean, this is silly. That means 5% of projects were successful. So if a hundred percent of projects had failed, the pressure to invest and squeeze value out of AI would maybe get turned down. I don't think that's going to happen. I think the pressure is going to stay on for CEOs and management teams to figure out where in their org they can be successful. And it might be that like hey, we swung for the fences and tried the revenue side. Let's build back some credibility by maybe working on the cost side. That ought to be lower hanging fruit, lower probability of failure. Let's get some wins on the board with AI before we try the more esoteric experimental stuff.

And it seems like on the vendor side that's been more of a successful route so far. If you were a startup founder now taking a look at this data and knowing where people's pain points are, what could possibly be the pockets of opportunity that you might try to pitch to an enterprise customer that might be attractive to them?

If I were a software vendor, I would really take this survey to heart because it means you're going to walk into a lot of buildings where either an internal or external team over promised and under delivered an AI related solution.

Yeah.

And so whereas maybe 12 to 18 months you would have walked into a very optimistic, wide eyed audience, now you're walking into an audience that might have some scar tissue either from a vendor who over promised and under delivered or from an internal initiative that was a failure. And so I think understanding the context of the company that you're walking into as a vendor is going to become.

Extremely important if buying AI tools from vendors leads to higher success rates. With the market flooded with wrappers and surface level integrations, what signals should the buyers so the other side of this look to truly separate valuable products from ones that don't move the needle at all.

I actually think it's okay if you're a vendor to say I'm buying this product because I think for the next year, it's the best solution to my problem. And a year from now, I'm open to the idea that this problem will be solved in a different way by a different vendor. And that's how I even at Precursor, that's how I approach all of the AI tooling I look at. I'm like, for the next six to 12 months, this feels like a reasonable solution, but I'm prepared to throw things away in 12 months if there's a better way to do it or a different way to do it and take.

Bits and pieces of things that we like from one and take that to the next vendor and say, like, I didn't like this, but I love this. Can you incorporate that?

I remember once upon a time I paid 10 bucks a month for a tool that helped me with text generation using chatgpt, probably chatgpt3.5 or something. And now I'm like, well, I don't need the wrapper in between me and chatgpt or Claude. I actually can just prompt it myself and do it. But there was a period of time where that was a helpful thing, but that opportunity was kind of ephemeral. And I think we're going to continue to see these ephemeral things, which doesn't mean you shouldn't take advantage of them during the moment of time when they're valuable. Just don't get too attached to them.

Well, our last topic is another insightful set of data from Peter Walker at Carta titled Startup Valuations on Steroids, where valuations across stages are categorized in buckets from low end. So like the 25th percentile to wild, which is the 95th percentile, and then wow, which is the 99th percentile, the wild column. So the 95th percentile looks quite similar to 2021, where the median valuation for seed stage companies hung around $72 million. However, the new WoW column increased 123%, with median prices landing around 161 million for seed stage companies at this 99th percentile. Looking at the chart, the question that remains here is where will the strongest company service? Any additional thoughts on this data? Maybe more specifically also on the seed specific numbers too?

I think Peter might be. If we had a leaderboard, Peter's definitely number one. Or number two.

None.

It's got to be the Peter or Hunter. And you know, Peter knows I'm a big fan of all of his work. I just published our quarterly LP letter two days ago and the opening paragraph was, this is the busiest summer I've had since 2020 or 2021. Take from that what you will. And I was like, I could replace not going on vacation because you're locked inside for Covid with not leaving your phone at home because you're worried you'll miss out on some super hot expensive AI deal that's closing in six hours. Or some triple layered SPV into anthropic that just came through on a WhatsApp group. This is a frenzy. It's funny, Mia. I feel like everybody I know at the GP level is like, this is insane. I'm like, it's insane. But we're all participating in the insanity. And we just went through this three or four years ago. I think we kind of know how this tends to end, but it all feels like a great idea until it ends. And I think there's a lot of people who are like, I'm going to be able to get out before the music stops. And I really hope they are right.

Yeah.

And those valuations, to me, they just don't leave a lot of room for companies. You know, there was an old saying when I got into venture was like, oh, that round is priced for perfection. If everything goes perfectly right, that price probably makes sense. These new rounds are priced beyond perfection in my mind. Like, it's not like if everything goes right, these are reasonable valuations.

My last question to close out is, from your perspective, does this kind of aggressive pricing reflect genuine progress and innovation at the company level or does it speak more to investors obsession with winning across the most competitive deals through increasingly aggressive term sheets?

I think the biggest change I've seen in my entire career and venture over 20 years is I think founders understand how to manipulate VCs and VC attention far better than they did 20 years ago. And I think VCs have the capacity to pay higher prices for rounds which are recent. One, the scale of outcomes has grown over time. Second, the funds are bigger. So at the margin, if you want your ownership paying up to get, it works. But I think founders are just more skilled at extracting higher prices from VCs because they've had more reps and they better understand the model. And I think if you can convince VCs that your company is really hot and it's a deal that they will deeply regret missing, you can have a lot of influence on the entry price, which is very different than the price. And it's funny, I talked to one of our LPs this week and he said, you know, like, I don't know that I feel that differently about you losing a million dollars on a $10 million valuation versus losing a million dollars on $100 million valuation, you still lost a million dollars. And I was like, well, yeah, well, when it doesn't work out, the price you paid doesn't matter. But when it does work out, the price you paid, just go look at the Figma waterfall. Look at the difference between what index got by being in at the seed and what everybody else got by coming in later. It's, it is not trivial. So I think entry prices matter, but I also think being in really important companies matters. And I think not every important company starts off with a high valuation and not every company with a high valuation is ends up being an important company. But we only usually talk about the things that are at the intersection of turned out to be in it. Like when Square raised its seed round and OpenAI. The companies that turned out to be important and raised at a high valuation end up being the thing that it's like the buoy that everybody hangs onto in the ocean.

Of course. Yeah, very few articles are going to be written about the ones that raised at high valuations that kind of like went off into nowhere that we don't hear about.

We're pretty good at burying those. That company. I don't remember.

No, yeah, no, everyone just has a wonderful memory issue then. Well, thank you so much for joining. We are all out of time today. Thank you to our listeners for hopping in. Please like and subscribe and we'll talk with you all next week. Thank you. Bye bye.