SaaS Metrics Maturity Model - Part 1
16 Apr 2024 · 23 min
Dave Kellogg and Ray Reich present a five-layer maturity model for SaaS metrics, built from 15 recurring pitfalls. The root causes range from lacking a shared metrics foundation to not using metrics for long-term planning. Kellogg shares examples like CAC bludgeoning at Host Analytics and how Salesforce built numeracy into its culture.
Chapters
Live from Schenectady, New York, it's SaaS Talk with the Metrics Brothers, Growth and CAC.
And I'm Growth, better known as Ray Reich, founder and CEO of Benchmarkit.
And I'm CAC, better known as Dave Kellogg, independent consultant, EIR at Balderson Capital, and the author of Kelblog.
And together we are the Metrix Brothers.
And we go together like wine and roses.
Dave, you're still sticking with the alcohol analogies even though I couldn't close Kelly's as a sponsor?
I, I do. Technically, roses are a flower, and that was inspired by an old Jack Lemmon movie called The Days of Wine and Roses.
Dave, I know what a big Grateful Dead fan you are. When you talk about roses I'm not usually pairing those with wine.
Oh, you want to talk about roses? Well, normally in the, in the Grateful Dead iconography, roses are paired with skeletons. And did you know that one of the central Dead images comes from a book called the Rubaiyat of Omar Khayyam, a collection of 18th century poems from 1913 featuring illustrations by British artist Edmund Joseph? Did you know that, Ray?
Well, no, I didn't, and I don't think I ever wanted to. And I think we need to get back to what we're really good at, and that's talking about business and today's topic, the SaaS Metrics Maturity Model. But before we do that, maybe we should hear a word from our B2B SaaS sponsor.
The Metrics Brothers is proudly sponsored by Benchmarkit, the B2B AI and software industry's largest, broadest, and most contextualized benchmarking index. Dave Kellogg and Ray Groth Wright have put the Metrics Brothers podcast on hiatus for remainder of 2026. All episodes we have ever recorded are still available on your favorite podcasting app. You can reach Dave at Kellogg and Ray on LinkedIn at Ray Reich. Thank you so much for listening to the Metrics Brothers podcast.
Okay, CAC. So let's get into business here. Let's dive in. Where did this SaaS Metrics Maturity Model come from?
But Ray, does this mean you're not coming with me to Las Vegas to see Dead and Company at the Sphere?
I'm not, and I'm sorry to tell you that, but let's stay on task. Get back to trucking on the SaaS Metrics Maturity Model.
Oh, there you go. There you go. Keep on trucking. So look, the background of the SaaS Metrics Maturity Model. This is a model I made for my SaaStr 2023 talk. which is called The Strategic Use and Abuse of SaaS Metrics. I started out by talking about 15 problems with SaaS metrics, and it's a lot of fun. It's always fun to talk about problems with SaaS metrics, but I realized as much fun as that is, you're not really talking about how to fix those problems quite so much. So I felt an obligation after having said here that, you know, 15 silly things we do with SaaS metrics that could be warning signs. Well, I felt some obligation to say, okay, how do we solve these problems? How do we kind of not have these 15 problems? So that's why I made the model.
And I must admit, I'm a Dave Kellogg fanboy, even though I do a podcast with you. I was there in the third row and it was a packed house. So it really resonated. And I think one of the things you really did was kind of highlighted why you made a model. So before we go into the 5 root causes, what was the primary reason you even thought you had to create this model?
Yeah, like I was saying, mostly to me, I wanted to do what I call a you are here analysis. Like maturity models are not new thing. Gartner have made them for years. There's CMMI for software engineering, the Capability Maturity Model. There's all kinds of maturity models. There's a BI maturity model. And I think they're a nice way of kind of breaking a problem that can otherwise be overwhelming into layers and then saying, look, you are here, so kind of place yourself in this model, and then it tells you what to work on. Like, I'll give you a really simple example. I work with some European companies who don't have a P&L that is a standard SaaS P&L. So they look at like what they call nature-based P&L. They're looking at like personnel costs and consulting costs. So you can't even calculate a CAC ratio because you say, what's sales expense and what's marketing expense? And they say, we don't know. Right. So that's a foundational problem, right? We need to say, okay, one of the foundations is we need to have a standard SaaS P&L or we can't even begin to talk about SaaS metrics. Right. So that's just an example of why it really helps to say, look, here are 15 crazy problems. And let's try and get some idea of if we wanted to fix these, how they line up by layer. Let's try to help you place yourself on a layer and then from that know what to work on next.
Yeah, but you mentioned 15 problems. And when I grew up in corporate America, I was taught 3 to 5. You need to identify 3 to 5 things that you can actually work on and make progress. And I think that's kind of what you did, Dave. Didn't you kind of identify 5 root causes that were the reason for a lot of these problems?
Absolutely, yeah, yeah, you definitely were there and paying attention, Ray. I'm very happy. So, yeah, that was the whole point, that, look, we had these 15 things that everybody could get a chuckle about and go, wow, that feels like me. But, well, what's the underlying causes? And the answer is, I identified 5 of them. And in order, one, there's no shared metrics foundation, meaning, what metrics mean, how they're calculated, kind of why they matter. So if you don't have some foundation for metrics, for example, if you don't have a SaaS P&L, you really can't start on the journey 'cause everything's gonna be a manual reconciliation. I mean, you can produce metrics for fundraising, but you can't operationalize 'em. So the first layer was there was no shared metrics foundation. The next was there was no trust. There was either no trust in the data, where it was coming from, in the calculation. Sometimes the people who make the calculations, the process for producing them. Do the metrics tell the real story, or are they being kind of cherry-picked and spun? Smoothing is a classic way, leaving out customer success from a CAC. There's all these kind of standard tricks, amortizing commissions and forgetting to reaccelerate them. If you find all these sort of surprises, you go, ah, I'm not sure I trust these metrics. So that was the next level. And a lot of that comes from misuse of them. If you're always bludgeoning people with metrics, then maybe they're going to start spinning them, right? For every action, there's an equal and opposite reaction. So, you can kind of beget some of these problems by the way you use metrics.
Hey, Dave, before we go to the next root cause, I was just thinking, you know, we're going to talk about each of these 5, but one of the big values we can give is what to do about each of these. And I know we're going to talk about a maturity model. We may not get all that done in this episode. So, would it be okay with you if we don't, that we do a second episode where we really talk about the maturity model? And what to do for each of these root causes?
Yeah. I mean, I think my guess is that's what's gonna happen as we do the episode. So we should feel free to, you know, if you wanna elaborate on some of these as we go through, feel free, or tell a war story. But the first one was basically, look, if there's no metrics foundation, it's very hard to get going, and we need to fix those issues first. The second is, if people don't trust the numbers, then we need to go build trust in those numbers. So I'm both identifying the problems, and this is gonna map directly to the maturity model in a minute.
Yeah, let me go back to there's no shared metrics foundation. You know, you mentioned why they matter. One of the things I find in so many SaaS companies is they're saying, okay, this is such an important metric, net revenue retention. I'm just making one up, right? But the people in customer success know that it's a priority, but they're not even sure how it's exactly calculated and why it's important to the business. And I think One of the things we do in the SaaS industry so often is we talk about metrics, but 95% of our employees don't even understand why they're important. So, I think that's a huge root cause issue.
Yeah, I mean, look, I think there's a difference between operationalizing metrics. And when I operationalize them, everybody knows what it means. They know how they can drive it. They understand that as a VP of customer success, you know, you can't prevent sales overselling. So, nothing unless you're CEO, and even when you're CEO, nothing is really totally within your control. But you can still feel ownership of the metric and go, yeah, I own this metric. And by the way, the reason our target is 110 instead of 125 is we know we're doing some overselling, and last year it was only 108, and we're trying to make it better. So when you've operationalized metrics, people understand what they mean, they feel as accountable as they reasonably can feel towards producing the number, and they understand why they matter. Conversely, when metrics— the thing I've seen, Ray, is when metrics are just a fundraising thing. It's like, oh, we need to raise a round. Let's go calculate all our metrics, right? That itself is a huge problem. It's actually layer 4 in my model. It's not having a data-driven or metrics-driven culture. But you can see that. I mean, the people I've worked with that don't have standard SaaS P&Ls, for example, they do produce all the SaaS metrics, but only when they're building a fundraising deck, right? It's done effectively manually. It's not operationalized.
Yeah. And I love your second root cause. There's no trust. Probably the most common thing I see when I do our SaaS metrics and benchmark assessments is, oh, we're looking really good compared to the benchmarks. I really think this is a great metric, and there was almost no questions. But then there's a time where a metric is not performing well, or the board pointed out, and they're like, well, how do you calculate the metric in your benchmark? Here's how we calculate it. And people actually try to defend their metric based upon how they calculate it when there's no common alignment on how it's calculated between the function, the CEO, and the board of directors or investors. It's a common issue I see.
Yeah, and the bigger problem is, let's just say you view— because a lot of times people will view that as quote-unquote correcting the metric. Oh well, at our company we do things XYZ differently, or we're focused on bookings, so we're going to calculate a bookings CAC, not an ARR CAC. And that's, you know, look, that might even be correct. But first, I don't think it's a good idea to run a SaaS company based on bookings. But, but let's just pretend you and your board do and you've decided you're a bookings-focused company. You, you can't— then the issue to me is what I call one-sided corrections. You say, okay, we're going to correct the CAC ratio to be done on bookings, correct, quote unquote, to do it on bookings. Well, that's fine, but then you can't compare it to an ARR-based CAC. And that's where I get off the bus, right? It's very hard to get me on the bus for the bookings CAC, but you might. Right? If you convince me your business is somehow very different— and I can think of ones, by the way. If you're doing $10 million, 10-year deals that take a really long time to cook, okay, maybe we are better off on doing them on total bookings because you really did buy that customer for 10 years. I recently saw a company that did that. Massive deals, long sales cycles, long commitments. Okay, maybe you sold me on a bookings CAC, but then you can't just turn around and compare that to an ARR CAC just because, you know, it looks favorable.
Now, I remember Sitting in that presentation, because one more thing on trust and we'll get to the next one. You talked about using metrics to, I think, bludgeon either another organization or the recipient. Can you talk a little bit about that using of metrics to mislead or bludgeon?
Yeah. I mean, like I mentioned earlier, when you bludgeon people with metrics, they respond accordingly. So, what do I mean by bludgeon? I mean that with a Without a whole lot of understanding or thoughtfulness, people just say, your CAC is 1.8, 1.8 is too high, we're inefficient. Like, for example, this happened to me. I was running Host Analytics. Our CAC would run 1.8 to 2.0, which is a high CAC ratio, and, and we kind of got bludgeoned with it. There wasn't a lot— they were like, oh, you're inefficient. And I'm like, I don't think we're any more inefficient than our competitors. I think we're in a competitive market, and I've got a deal. And I've got it at $50K and I'm doing pretty well, but my competitor Adaptive shows up and they offer the same system at $25K. So I can win the deal at $30K, I can get a price premium, but I can't get 100% price premium on it. And what happens to my CAC? My CAC just nearly doubled, right? When I had to cut my— on the last day of the sales cycle, my CAC nearly doubled, right? And by the way, when Adaptive finally filed to go public, they never went public because they were acquired first, their CAC was around 2.
Oh, wow.
And I was like, see, I told you. For 6 years, I was telling you that There's no magic here. They have the same sales model. They're hiring the same people. That was my derivation. Their SDRs look like our SDRs. Their quotas look like our quotas. Their staff look like our staff. I think they're running at the same CAC. I think this space is a high CAC in part because we're bludgeoning each other, right? That's a thoughtful way to look at the metric and not just go, oh, you're 2.0, it's too high, it should be 1.5. It's like, whoa, hang on. So that's what I would call bludgeoning.
Got you. Well, let's move to the 3rd root cause. I don't know if you remember that special guest who kind of snuck in a few episodes ago, Nick Mehta from Gainsight.
Of course I remember Nick.
He actually told me this one time when we were talking about or debating the merits of metrics. And he's like, Ray, if metrics aren't tightly aligned or integral to the strategy, they're not nearly as valuable. That's actually your number 3, right?
It is, yeah. Metrics as an afterthought are not a good thing. We touched on this one earlier as well, but some companies view metrics as a financing tool only, right? calculate them when they're doing a round. We talked about that. Not a great habit. I mean, yes, you need to calculate them when you're doing a round. But if that's the only time you're calculating them, you should look in the mirror. They're not linked to OKRs. I mean, I like OKRs. The notion of saying, here's our objective, and having 3 to 5 objectives, and then here's a key result associated that kind of defines or proves the objective, I think is useful. So, if your SaaS metrics are important to you, let's just say, I'll give you a concrete example. You're doing an upmarket push. Because you think you could take your ASP from $25K to $50K and just say, I'm the person leading the sales team upmarket, I should have an OKR to get a $50K ASP. And by the way, we should manage that intelligently and not bludgeon me if it comes in at $44K in our first quarter of trying, right? So, but yeah, if you care about these metrics, like for example, if you think your CAC ratio is too high because your deals are too small and you think the answer to that is to push upmarket for bigger deals, then somebody should have an OKR first on developing pipeline in that richer segment, and second in actually winning deals with that higher ASP, average sales price. So the other thing you'll find when metrics aren't linked to strategy is they're not being used as kind of leading indicators, or there are no leading indicators. We're, we're not using them day to day to run the business. We're kind of metrics on the side, right? And if you do this correctly, you're looking at the metrics every day. I couldn't see a forecast for the CAC when I ran Host. I saw a forecast for my CAC every week, right? Because it's not that hard. I know what new I know what sales and marketing expenses forecast. I can forecast the CAC and keep an eye on it, which I did.
Yeah, and that's like the last episode of SaaS Talk. We actually talked about customer lifetime value to CAC ratio, a multivariable compound metric, right? And if you don't understand the leading indicators that's driving that and can make operational decisions to improve those leading indicator metrics, there's no way that that top-level metric is going to be relevant to you on a day-to-day basis.
Yeah, totally agree. By the way, let me give you my example of the indicator. It would be pipeline ASP as a close-to-close won ASP. In my example of pushing up market, that, that the real proof of the pudding is am I closing $50K deals, right? But, but we could look at pipeline value as a leading indicator, right? So, so it's just another example of like what things tend to happen before other things. And even if they're not end results, which kind of definitionally leading indicators aren't end results, but But closed won deals, they're a tough metric because they're almost immediately history. We closed it, we won it, it's over. And by the way, I won't watch it change for 6 to 9 months if I have a long sales cycle. So we need to look at both types of these indicators.
Can you dig a little bit more into the 4th root cause, the culture is not metrics-driven?
Sure. So the 4th underlying problem or root cause of SaaS metrics problem is we don't have a metrics-driven culture. Discussions don't require numbers. Jim Barksdale, the founder of Netscape, so this is an old story, but he had a great quote worth repeating, which was, if we have data, let's look at the data. If all we have is opinions, then let's go with mine, which is an absolutely fantastic quote. I think all CEOs should say this all the time because it's a great way to incent people to bring data to conversations. And in my opinion, Ray, I've noticed that there's almost 2 types of companies. I tend to be pretty binary in these regards, but there seem to be companies where numeracy is a prerequisite for conversations about the business and companies where they're not. In my belief, and I don't have any data to support this, my belief is the type 1 companies do way better than the type 2 companies. For example, when I was at Salesforce, I didn't work for Benioff. I was one-off Benioff, but I would sit in for my boss from time to time in Benioff's staff meeting. And if you don't know your numbers in Benioff's staff meeting, you're a dead man. You are done. So, so numeracy is required as kind of an entry ticket. And that's why I think people who don't work in kind of numerate cultures, they don't understand that. They think all people do is talk about the numbers. And it's like, no, we're not talking about the numbers. We're talking about the business, using numbers. And that, that's really the goal.
Boy, that brings back so many memories because, as you know, I grew up my first 8 years after undergrad at GE, and Jack Welch was the same way. You could debate him on anything. And one time I had that opportunity and I knew you had to have 2 things, numbers to back it up and critical thought to justify why you're recommending to do something different. So I love that.
So now is a good time for me to reveal the secret of why we're live from Schenectady, New York. I don't think we've ever talked about this on the show before, and I don't think anyone's ever asked. But the reason is I knew Ray had worked at GE and I pictured Ray as kind of a GE guy. And you'll have to have worked, unfortunately, 20 years ago to know what that means. But I was like, ah, Ray, he's a GE guy. He's got the GE guy feel. He must be from Schenectady. So I literally thought you lived in Schenectady for the longest time before I found out you lived in New York.
Well, so I respected that. And, you know, I hate to do this because, you know, I like to talk about business, but I got to give a 30-second story about Jim Barksdale. 30 seconds. So I did have the chance to work closely with Jim and executive team at Netscape back when I was helping launch e-commerce there. And he had another saying. He's like, you know, there's 2 types of people in an organization. He goes, it's a little bit like when you look at your breakfast plate, you'll see an egg and you'll see bacon. He goes, in that environment, the chicken was involved in the business, but the pig was committed. Are you a chicken or a pig here at Netscape? I love that saying.
Yeah, it was a super common saying at the time. I love that saying as well. I didn't actually understand it. Like most of those sayings, I didn't actually understand it the first couple of times I heard it. But it rubbed in. Okay, so in any case, old memories are fun, and fun stories. Let's wrap up here with the last of the 5 underlying causes, which is our metrics are not being used to define trajectory and long-term goals. This to me is kind of a plus one on the whole model, because the rest of the model is, in my mind, kind of ordered, right? It's like, you gotta have a good foundation, you have to trust the foundation you've built, you need to kind of operationalize metrics and link them to strategy, you need to create create a culture that is data/metrics driven. Those 4 things to me are a natural layer cake. This one on top is a bit artificial, but I think it's super important, so that's why I put it there. But I think when it comes down to the problems that get caused, it's when companies don't have a defined long-term trajectory. And you could use metrics and numbers to say, this is the company we're trying to build over the next 5 years. So whenever I submitted a budget to the board, I would submit the 4 quarters of next year with targets that we would be held accountable to, right? That's the budget. Approve this and we'll be accountable for it. But I would present a model for years 2, 3, and 4 beyond that and say, this is what we're trying to build for 2 reasons, 3 reasons maybe. One, that the next year model became the default. Like when we started discussions about the plan for year N+1, we'd start with what I showed as the model a year ago. So it was a nice kind of default. Second, it agreed on trajectory of what we were trying to do. And third, I actually think it built trust because the number one way to game a budget in software is to cut all incremental spending in the second half designed to drive growth in the following year. And then you can show kind of great EBITDA or great profitability or great cash flow in year end, but you have teed up nothing for year end plus one. And you can only do that trick once or twice before your board gets very angry with you. But if you're showing a multi-year model that dovetails into your financial plan, you can't do that. And so in some ways, you know, when I taught my kids to drive, I taught them, you know, like, don't look at the hood ornament, right? Look out way in front of the car, right? Like, you won't be surprised if you're looking way out in front of the car. If you're driving looking at the hood ornament, you're going to be surprised constantly. So it's the same thing with this long-term trajectory. So, so this one is basically saying another great way to use metrics is in conjunction with one of these 3 to 5 year long-term models, often called an LRP or an LRM— long-range model, long-range plan. If you have one of those models, You have it also full of the SaaS metrics you're trying to hit. So everyone understands, for example, that, hey, we're shooting for a CAC of 1.8. We're not shooting for 1.4. And 1.4 might be best of breed for you, but in our space with our competitors, we think we can only hit 1.8. So if you look 3 years out and see a CAC of 1.8, let's have that conversation now because I think that's the best we can do.
Totally agreed. So I want to do real quick, I want to summarize the 5 root causes. One, no shared metrics foundation. Second, no trust. Third, the metrics are not integral to strategy. Fourth, a culture that's not metrics-driven. And then fifth, that metrics aren't being used to define a trajectory and long-term goals. And those are the 5 root causes that made you create this SaaS Metrics Maturity Model. So, I'd like you to introduce it at a high level, David, and then we'll go into the next episode and talk about it in detail.
Sure. When we talk about the model, we're going to have 5 layers, very unsurprisingly. Layer 1 will be foundation. Layer 2 will be trust. Layer 3 will be strategic linkage. Layer 4 will be culture. And layer 5 will be strategy.
Okay, I can't wait to dive into the details. So that's going to wrap up this episode talking about the root causes and needs to have a structured SaaS metrics maturity model. David, thank you for teeing this up, and I can't wait to get into the maturity model itself.
Awesome.
The Metrics Brothers AI, SaaS, and More is a production of The Metrics Brothers Growth and CAC. And a member of the Benchmarket Podcast Network. By accessing this podcast, you acknowledge that the Metrics Brothers make no warranty, guarantee, or representation as to the accuracy or sufficiency of the information provided or the humor content of the jokes told.
Ray.
Information, opinions, and recommendations presented are, according to our spouses, probably wrong and provided for information purposes only. This podcast should not be considered professional or, for that matter, unprofessional We disclaim any and all liability for any direct, indirect, indirect, misdirect, incidental, special, ordinary, consequential, inconsequential, or other damages arising out of any use of, or God help you, reliance upon the information presented here. Ray Grothreich is based in New York City and available on X @RayReich.
Dave Kellogg is based in Silicon Valley and available on X @Kelblog.
Schedectady, which is French for unspellable, is not our actual production location.
Thanks for listening.
