Benchmarks - How to Evaluate and Use

Live from Schenectady, New York, it's SaaS Talk with the Metrics Brothers, Growth and CAC.

And I'm Growth, better known as Ray Wright, founder and CEO of Benchmarkit.

And I'm CAC, better known as Dave Kellogg, EIR at Balderton Capital, independent consultant, and the author of Kellogg.

And together we are the Metrics Brothers.

And we go together like fine wine and cheese, right?

Okay, yeah. Now this is a pairing that I think is highly relevant for the two of us. Though I think that fine wine, which maybe like somebody on this podcast gets better with age, is my favorite option on this pairing.

Well, so benchmarks do not get better with age, but benchmarking gurus get better with age. So it is in fact a fine pairing. And Ray, did you know that chèvre, fromage de chèvre, also known as goat cheese. And speaking of goats, I thought maybe, I'm all over the place here, right? But speaking of goats, I think we could talk about some of the GOAT benchmark reports, but a different meaning.

Yeah, as you know, Dave, I love SaaS benchmarks, so I can't wait to bite, or should I say sip, a little bit about this topic.

Got it. And just because we're bungling the script, GOAT, the other goat meaning, well, one goat is fromage de chèvre, which is goat cheese. The goat we're talking about his greatest of all time. But before we hear about our greatest of all time benchmarks, yes indeed, let's hear a word from our sponsor.

The Metrics Brothers is proudly sponsored by Benchmarkit, the B2B AI and software industry's largest, broadest, and most contextualized benchmarking index. Dave Kellogg and Ray Grothright have put the Metrics Brothers podcast on hiatus for the remainder of 2026. All episodes we have ever recorded are still available on your favorite podcasting app. You can reach Dave @kellogg and Ray on LinkedIn @rayreich. Thank you so much for listening to the Metrics Brothers podcast. Okay, Dave, so, you know, I chose to do this episode not just because I live in benchmarks every day, but You know, benchmark reports are popping up like mushrooms. We've covered several of the GOAT benchmark reports previously. So I wanted to talk a little bit about, at a macro level, what are some of the things to consider when you're analyzing benchmark reports? What are some of the attributes of a good benchmark report? What are some of the attributes of a not-so-good benchmark report that maybe can be a little misleading? And then I want to talk a little bit about, as SaaS operators, What are some of the best practices of utilizing benchmark reports to help set some of your goals, objectives? And then at the end, talk about some of those GOAT-like industry benchmark reports and sources. Sound like a good episode?

Yeah, it sounds like a good topic, Ray. Look, in our work, we both read a lot of benchmark reports, some of which we cover on this podcast— tends to be only the GOATs— many of which we don't, but we read both good ones and bad ones. So I think it'll be a good episode to talk about kind of what separates the two. What do we look for and what do we try to avoid?

And I think, you know, since I live in this every day, I'm going to kind of provide some of my insights and I really look forward to your feedback.

So yeah, I forgot to mention Ray also makes benchmark reports. Sorry, Ray, I forgot to mention that. Ray is actually in the business. So we'll hear the insider view as well.

And I'm looking forward to your feedback of how we can make them even better, Dave. Okay. Topic number 1, what are some of the top things to consider when analyzing benchmark reports and the benchmarks that are included? So first of all, a simple one is the timeframe of the metrics. Our industry is moving so darn fast that benchmarks that are more than 9 to 12 months old, it can be outdated information. So that's the first thing I look at is when were the benchmarks calculated in? The next thing to me that's really important is You know, are the benchmarks based upon averages versus medians, or even the top quartile? Because some of these benchmarks report out there, they present all the 3 on different slides, or they just use one. So I think that's really important. Understanding the total population size and the profile of those participants, very key as you're looking at a benchmark report. The next thing is Understand if the segments that are being used, maybe that's $1 to $10 million, $10 to $20 million, $20 to $50 million, is that relevant to your company profile? And then 2 last things. Benchmarks can be biased based upon the selection criteria of the population. And I'll give an example there. When I first saw the Andreessen Horowitz benchmarks about 18, 24 months ago, God, they were off the chart good, like 145% median NRR. And then I compare that to something like ChartMogul, which has a lot of very small companies under $5 million, very different benchmarks for the same metric. So understand that selection criteria. And then the last thing I would say is a lot of private company benchmarks are based upon ARR, but a lot of public company benchmarks will be based upon Either implied ARR or even GAAP revenue. So understand the differences between private and public. So I have a lot of things I said there, Dave. What do you think?

So, boy, there was a lot there. I think first the point that resonates with me is number one, make sure you know what you're looking at, right, is the kind of meta rule because benchmark sponsors pull different audiences. Like you mentioned, Andreessen Horowitz, I think Iconiq does this a lot. They, they pull their companies, which are all very high-performance companies because they're kind of a growth-stage, high-momentum investor. So they tend to have very high benchmarks. I think benchmark sponsors sometimes have agendas. Like, VCs in general want people to go faster, try harder. So they're only too happy to put out a benchmark where the averages are shown as perhaps above the actual average because they're pruning out, you know, some types of companies. Uh, PE firms have different profiles than VC. So the benchmarks for, say, growth and, you know, growth is higher and profits lower in VC-backed firms than there is in PE firms. And obviously benchmark firms, uh, tend to be optimizing for sustained existence and cash flow. So I think it's important to understand who's in the survey and what are you looking at for sure, Ray.

And I'm just going to double-click on something you said. Like, you know, at the end of today's episode, we're going to talk about some of the GOAT reports. Iconiq does some amazing benchmarking research. But one of the things I've noticed about their latest reports is sometimes they'll be showing the top quartile, right? So at the 90th percentile, other times they'll be sharing median. And you got to be really careful to know for a particular page in their report whether they're showing top quartile or median.

So— Yeah. So I, by the way, on Iconiq, Ray, I want to agree that they're doing fine work first. Second, that they are among the more rigorous, like they have glossaries and footnotes and they do a really nice job in general. of telling you what you're looking at. Um, but, and the last part of know what you're looking at, I think, Ray, is what you're saying is, is you got to know whether you're looking at a mean or a median, which is why my favorite reports are almost always box and whiskers charts, as they call them, where you're seeing the distribution, you're seeing the median, you're seeing the 25th and 75th percentiles. That's the box. And few people add the whiskers, though I wish they would, uh, because the whiskers would be the min and the max. And that way, with that visual representation, you automatically know what you're looking at.

Yeah, I totally agree. In fact, we have a debate all the time. Do we show the min and max or is that noise? But you mentioned something about the iconic, and that's kind of a first attribute of a really good benchmark report, that the metrics that are used to create the benchmarks, there's a glossary of what the metric is, how it was calculated, or if it's not in a glossary, at least it's in a footnote. And that's a really good best practice. The other thing is, if you see a benchmark report that takes the same metric, let's say it's net revenue retention, and they show it by company size segments and average annual contract value segments, that's a more valuable report because sometimes the average annual contract value is more correlated to the metrics value than company size. You mentioned the box and whisker. Totally agree. If they give you At least 25th median and 75th. It's a really good report in my mind that they always highlight the timing of when the data was collected. So, you know, you're looking at information from Q4 '24 or calendar year '24, and it's not from just Q1 of '24, which might be too old. And then the last thing on that, showing the population— not to get too much into any statistical detail because I'll embarrass myself. But, you know, if the population of a particular benchmark is at least 30 per segment, then according to this thing called the central limit theorem, you can probably use that rule of thumb to say it's probably a statistically valuable benchmark. So that's what I think. What do you think of some of the attributes?

So, so I agree with all of that. 2 clarifications. One, I think for statisticians, when Ray says population, he means sample. So when the sample is above 30, it works. Uh, not the population. And in general it does. And you can look up the central limit theorem if you want to have some fun. Um, and then on the other one where you talked about timing. Yeah. One of the most interesting timing things Rhea and I ever saw was a benchmark that was published in like September and it had that year's financials. And we're like, this thing was published before the year was over and they were actually doing it based on a forecast, which is weird. So I like my benchmarks based on actuals. Once in a while, you'll do a 2024 benchmark report and ask 1 or 2 questions about next year's plan. Hey, is the plan for next year up or down? At what growth rate? But, but in general, I don't want to do benchmarks, which are by and large about historical actuals based on somebody's forecast for what they think is going to happen because the publisher decided to publish 2 months before the end of the year or 3 months before the end of the year.

Yeah, no, I can't agree more. I couldn't believe this. That report actually showed the forecast for the year. Yeah.

And then clear, right? They were just showing it as an annual benchmark, right? And I had to dig it out.

Yeah.

Yeah.

Now on the flip side, this is not rocket science, you know. So what are some of the attributes of a not so good benchmark report? Small populations always concern me. If I see a total population that's got like maybe—

just hear sample every time Ray says sample. I'm giving him a small electric shock every time he says population. Go ahead, Ray.

So if you see a sample size that's small for that particular report, maybe it's 80, 90, but anything less than around 100, I at least— it makes me think about how valuable this report is. If I see a lot of averages, Dave, so they say the average pipeline generated by marketing is 40% versus, hey, the median is 40%, the 25th is 30%, and the 75th is 48%. That makes me feel a lot better because those outliers or a larger standard deviation can really impact averages. That there's no sample size on each chart so that, you know, there's 150 people, but they show a chart that has nothing, and maybe it only has 10 people versus 80 people. So that kind of makes me think about the Benchmark integrity. And then there's no definitions of how the benchmark was calculated. And/or, as we said before, the data is old. I don't want to look at a 2022 benchmark when I'm making decisions for 2025. So those are some of the attributes of a not-so-good benchmark report. Anything to add to that, Dave?

Yeah, I'm still lagging, but I want to talk about 2 more things on segmentation. So Ray talked about kind of classical segmentation and also just, you know, beware when you start cutting these things. If you only have 10 data points in the sample, it's not good. So beware of small sample sizes for that reason, because when you start slicing and dicing, you may not be able to basically. And Ray talked about classical cuts like by company size. One of the more interesting cuts I like is when you do like the CAC ratio segmented by growth rate, right? Because sometimes the growth rate of the company drives the CAC ratio more than anything else. It's not size, it's growth rate, right? So those, I'm just going to call them creative segmentations. Another one I like, which I beg Ray to do, I don't think he's done it yet with his work, is because Ray's benchmarks include a fairly open number of people participating to get them to identify by type of funding, bootstrap, PE, VC, and then segment by that. I don't know if you've done that. It's been on my list for a couple of years, but that's another interesting segmentation that I think matters more than a lot of other things. Because as I always say, if you want to go to Harvard, you should benchmark off the people going to Harvard.

Yeah.

If you want to go to Chico, benchmark off people going to Chico, that you should benchmark off people with similar aspirations to yourself. And PE and VC and bootstrapped aspirations are all actually pretty different.

Yeah, we've actually started doing that on about 60% of our benchmarking research programs. We look at private equity, we look at VC, and if they say VC, what stage? Stage, is it a Series A, Series B, Series C, and bootstrapped? We even for the international crowd did government credits or government funding because it actually changes things. So we're doing it now, Dave. Thank you.

Good, Ray. That's awesome. So, so look for, if you want a good benchmark report, it's got what I'm just going to call creative segmentations. It's not all just split by size, but CAC by growth rate, you know, Rule of 40 by investor type. These are interesting segmentations.

We're actually even finding pricing model is a really interesting way to segment things because if you have a usage-based pricing, we're seeing a correlation to higher net revenue retention rates.

Yeah, NRR by pricing model would be a beautiful one. Yeah.

Okay, so let's move to the operators out there. So how do I use these? What are the best practices? Well, some of this is going to be repetitive, that you want to use benchmarks that are for companies similar to yours. If you're a $10 million ARR company, you don't wanna be using public benchmarks to say, hey, I should be doing this for Rule of 40. It just doesn't make sense. You need to understand how benchmarks were calculated. So hopefully the benchmark report will show you that. I'll give you an example. I love Insight Partners. They do some really good work, but when they calculate CAC payback period, they don't gross margin adjust it. And we at Benchmarkit and with the SAS Metric Standards Board, we do gross margin adjust CAC payback period. And honestly, you can see a 20 to 30% difference in CAC payback periods whether you gross margin adjust or not. So the calculation formulas are important. And then the thing I tell everyone is benchmarks are not absolutes. They provide guidelines with some guardrails. to provide a sanity check against your internal metrics performance, but they're not absolute. You don't need to say, hey, because the median was 105% NRR that we have to hit it. Now, your investors may give you those goals, but it's not necessarily has to be the goal. And then the last thing I would say is there are trade-offs just like they are between metrics. Benchmarks are simply the result of a collection of metrics. Example, if growth rate is your number one goal and you're saying, I can hit 30%, it's a $50 million company, you may sacrifice your CAC ratio. You may be willing to have a CAC ratio that's 20% higher than the median because your growth rate is 20% better than the median for companies like yours. So I think it's really important to understand those trade-offs. So those are some of the feedbacks I have for best practices. What do you think, Dave?

Couple things, Ray. I'll reinforce a couple. So one, you know, my wife and I never shot to have 2.5 kids 'cause that's the average, right? So you always have to be careful when you're shooting for something that just doesn't make sense given the benchmarks. Number 2, benchmarks are really a function of strategy. Ray was touching on that, but I'll give a similar example to Ray's. But just say I have a product that expands like crazy once in a customer, say like a 200% NRR rate. I should be willing to spend a lot more than just 20% more than the median. I might want to spend 2x the median CAC, right? Like if I've got a customer who expands at, you know, 200% a year for 3 years, I might be willing to spend $2 or $3 to acquire them where the norm might be like $1 or $1.50, right? So it's not just plus or minus 20%. These could be big swings. So if you have a teeny land, big expand strategy, you may end up with a very high CAC. But it's more than paid for by the NRR. And there's 2 ways to do that. One is you could actually just be honest and say, no, it did cost us this much to get the person in the door, and it was $4, and it seems like a lot, but watch how they grow. That's the way I prefer. The other is to actually, you know, build in the year 1 expansion into the ARR in the land. Some— I've seen people do that as well, where they try to say, well, the land isn't really the first contract we got, it's kind of the accumulated spend we got in the first year. So there's ways to handle it, but the main point Is that the benchmarks, you can't, if you shoot for average on every measure, you have no strategy in my mind. So in some ways, choosing where you want to deviate from the medians is in fact choosing your strategy. I think the last would just be the other thing I try to do from benchmarks, because they can be overwhelming, is I always try to derive rule of, rules of thumb, like sales should be, you know, twice as much as marketing or expansion ARR should be 30% of new ARR. Uh, et cetera. So I'm always looking to kind of pull rules of thumb out so I can remember them. And finally, the last thing I'd say in presenting financial plans, I think people agonize about putting kind of one benchmark column, right? They put next year's plan and next year's growth rates. I actually would like to put 2 or 3 benchmarks, agree on, like, say you're at a size boundary. So you do benchmark 1, 10 to 50, and then benchmark 2, 50 to 100, because say you're crossing from 30 to 60 in that year, right? So you lay out both of them and say, here's what the smaller companies look like. Here's what the bigger ones look like. And then maybe put another benchmark where it's cut by industry or it's cut by something else, right? This is a triangulation process. So don't be afraid to show more than one shot. And the shots don't all have to align perfectly. The point, in some ways, the point is the difference. Where are you different from the means and why? And then what are we looking roughly to become and who looks like that?

It's really good insight. Now, we're coming up on the end of our episode already, but I was, you know, spent a lot of time up in Schenectady, New York, right? When I was first getting trained by GE and they said, hey, benchmarks are more than numbers. The process that underlies the operational execution of those numbers is really important to benchmark against best in class. So if at all possible, let's say it's the customer acquisition process or The retention process. If you can find some companies to benchmark their process against yours, that is a best practice that so many companies aren't doing today, Dave.

Cool. Okay, Ray, since we're coming up on time, why don't you talk about some of your faves and I'll even let you talk about some of the ones you make.

Okay. So when it comes to public company, I think you and I are aligned on this. Maritech and their website has some great public size company benchmarks. Iconiq does some great work. A couple that I've really liked recently was their marketing budget productivity report and their state of the GTM report, but that was last year. I actually liked what Emergence Capital did last year with their first Beyond Benchmarks report. And then of course there's the granddaddy of them all, the people who started this in SaaS from my perspective, the KeyBanc Capital Markets and Sapphire Ventures Benchmark Report. I would say look at it, maybe apply some of the ideas we provided today to their most recent versions. And then I always love some of the work that Kyle Poyer does, and he did one with High Alpha last year. And then lastly, yeah, Benchmarkit, which is the company I'm involved with. You know, we have our annual SaaS financial performance metrics. You know, we typically have a sample of anywhere from 1,000 to 2,000 companies. So it's pretty good. And then we just launched this one, which is the B2B Marketing Benchmarks for 2025 that I did in partnership with people like Carrie-Lou Dietrich, John Miller, and Bill Masaitis, with help from my brother, Kak, here. Those are some pretty good reports, if I do say so myself, Dave.

So I'll just share my comments on those reports. So for publics, I use Maritech and their stuff is great. Iconiq definitely wins the rigor award and comprehensiveness award. Um, so I use their stuff a lot, but you need to be thinking of kind of growth stage, high growth companies. So if you're working with, you know, a company that's $50 million growing at 100% that wants to go public, um, be kind of high flyers, I use the ICONIQ stuff. I think it's less applicable to say a PE-backed $25 million business, maybe inapplicable. Um, so I tend to use Ray's stuff, the Benchmarkit stuff, for smaller and/or private companies. Um, I tend to use Iconiq for the high flyers, Meritech for the public. Uh, I like the Emergence report. I can't remember why. I think they win the kind of brevity award. They had a couple of really good tight slides. Um, I don't yet give KeyBanc credit for being back. They were the bellwether for a long time, but they put out a few crappy reports, and I think they're recovering, but they're still in the penalty box for me, so I don't rely on them that much. And then Cobb Wire stuff, I agree with you, is always quite good. He used to be at Insight. He's now at a new firm. I can't remember the name, but definitely follow him and his work.

Okay. Well, that brings us to the conclusion of today's episode. And I appreciate my metrics brother partnering with me here on talking about benchmarks, which of course, the underlying foundation of all benchmarks are metrics, Dave.

Awesome. Well, thanks a lot, Ray. And we'll see you all next time.

Okay. Have a good week, everyone.

The Metrics Brothers AI, SaaS, and More is a production of The Metrics Brothers Growth and CAC and a member of the Benchmarkit Podcast Network. By accessing this podcast, you acknowledge that The Metrics Brothers make no warranty, guarantee, or representation as to the accuracy or sufficiency of the information provided or the humor content of the jokes told.

Ahem, Ray.

Information, opinions, and recommendations presented are, according to our spouses, probably wrong and provided for information purposes Only. This podcast should not be considered professional, or for that matter, unprofessional advice. We disclaim any and all liability for any direct, indirect, indirect, misdirect, incidental, special, ordinary, consequential, inconsequential, or other damages arising out of any use of, or God help you, reliance upon the information presented here. Ray Grothreich is based in New York City and available on X @RayReich. Dave Kellogg is based in Silicon Valley and available on X @DaveKellogg. Kellogg. Schedectity, which is French for unspellable, is not our actual production location. Thanks for listening.