Segments vs. Cohorts: What’s the Difference?

Live from Schenectady, New York, it's the Metrics Brothers, Growth and CAC.

And I'm Growth, better known as Ray Reich, founder and CEO of Benchmarkit.

And I'm CAC, better known as Dave Kellogg, EIR at Balderton Capital, independent consultant, and the author of Kellogg.

And together we are the Metrics Brothers.

And we go together like correlation and causation.

Well, Dave, those two are often confused, and we all know that correlation does not imply causation.

Yes, we do, Ray. And my favorite examples always involve underlying confounds, like drownings go up with ice cream sales.

Wait, ice cream sales causes drowning? Now I know why my mother always told me not to eat before I go swimming.

Well, well, Ray, mom was right, but you can't conclude that. You shouldn't eat before swimming, but it's not because it increases your odds of drowning. The confound is temperature there. The hot days means we sell more ice cream, and it also means more swimming, and it's more swimming that leads to more drowning, not more ice cream sales.

Ah, so you can't infer causation from correlation.

Nope, there's a whole field called causal inference, which I think, you know, my son Alex works in, And if you want to read one book on that, it's called The Book of Why by Judea Pearl. There are certain instances where you can, but for the most part, no, you cannot infer causation from correlation. Which leads us, Ray, to another thing, another pair of things that people routinely confuse.

So Dave, was that a causal inference there?

Causal inference? Now you're killing me with the dad jokes, Ray. That's one level too deep.

Okay. Well, let's talk about something a little bit more meaningful to a lot of our audience, and that's segments and cohorts. I know a lot of finance and RevOps people use these terms interchangeably, and that can actually create a problem.

It can. You had me thinking Sound of Music. I was going like raindrops on roses, segments and cohorts. But no, yeah, we did see, in fact, Ray, we saw an episode of this just last week where Norwest, the venerable Norwest, in a benchmark report that they published, wrote cohort when they meant to say segment. And that was actually, for me, the final straw. This has been on my pile for a long time for episodes, but that was the final straw for me.

Okay. So today, we'll explain the difference, why it matters, when to use each kind of these analysis. But before we do, let's hear a word from our sponsor. The Metrics Brothers is proudly sponsored by Benchmarkit, the B2B AI and software industry's largest, broadest, and most contextualized benchmarking index. Dave Kellogg and Ray Groth Wright have put the Metrics Brothers podcast on hiatus for remainder of 2026. All episodes we have ever recorded are still available on your favorite podcasting app. You can reach Dave at Kellogg and Ray on LinkedIn at Ray Reich. Thank you so much for listening to the Metrics Brothers podcast. If you'd like to sponsor the Metrics Brothers, send us an email at metricsbros@benchmarkit.ai to learn more.

So, Ray, you know, I keep hearing the term cohort where it clearly doesn't belong. You'll hear stuff like, well, our enterprise cohort is outperforming our mid-market cohort. And for the most part, I'd say 90%, maybe plus of the time, that's actually an error. That statement's wrong. What you meant to say was segment. And why do people do this? I think some people just like to pick the fancier word, like utilize versus use, and they think they sound smarter if they say utilize. But in this case, unfortunately, unlike utilize and use, they're not exactly synonyms. In fact, they're not synonyms at all. A cohort is different than a segment. So I think people who misuse the word cohort, A, can make themselves look bad. So if nothing else, this episode could help you look better. And by the way, it might reveal a deeper lack of understanding, like, hey, there is a thing called cohort analysis. Do you know what it is? And maybe you should be doing it because a cohort and a segment are not the same things.

Yeah, in fact, I often use this as one point of diagnostics when I start working with a company on metrics and benchmarking, because what I find is when you have A RevOps person or a sales leader saying, well, we just, you know, did a cohort-based analysis, and our enterprise market has a very different CAC payback period. So, I'm like, ah, maybe there's not really a culture of numeracy here. And I kind of use that as how do I approach this particular customer.

Okay, I'm not sure how many levels of irony this is, but I'm gonna assert that you mispronounced numeracy.

Numeracy.

But maybe numeracy works, but I was— I always called it numeracy. Um, and, uh, and I think you're right, it is a sign. By the way, jokes aside, it is a sign of people who lack in numeracy. And I do think numeracy, like literacy, right, uh, is something people should be aware of and focused on.

Yeah, in fact, and we're going to get into this, and maybe I don't want to give away what we're going to talk about, but then you also have the entire focus on time series analysis. And that also can get fused with cohort versus a snapshot looking at time-based trends. So maybe we should get into the definitions of segments versus cohorts first.

You know, Ray, they always say that the best way to detect someone's a spy is if they can't put the emphasis on the words correctly in English. Have you ever seen those spy movies where they ask like, who is the baseball pitcher on the Yankees, or they go to cultural stuff? But the One of the hardest parts about learning English, which I constantly get wrong myself, is emphasis. As my mom always said, you put the emphasis on the wrong syllable, Dave. So in any case, yes, with that brief aside, let's get back to basics here. Segments and cohorts, what are they? So a segment is a group defined by a shared characteristic or attribute. In some ways, if you think of a data table, segments You can make segments by looking at a column, right? Because there's going to be a column called vertical industry. There's going to be a column called size, account size by revenue or account size by headcount. There might be a column called deal size, right? And then you could segment by deal size. So to me, for the most part, when we're talking about segments, we're talking about slicing. In OLAP analysis, we would call this slicing data. according to some attribute. And since I said the OLAP word, I know we have a lot of finance readers out there. Let me just expand it. Online analytical processing, also known as multidimensional analysis. And I think it's an excellent mental model for what you're doing when you analyze. You have a hypercube of data, i.e., a cube of data like product, geography, time, or industry, right? You've got these multiple dimensions. Those dimensions can be hierarchical, by the way. So, you know, dentistry is part of healthcare, which is part of something, right? Or New York City is part of the Eastern District, which is part of the Northeast region, which is part of the US country, right? So the dimensions on which you're slicing could be hierarchical, but what lives in the intersections of the cube are what they call measures. And those are the numbers that we're actually measuring. So brief aside there on OLAP/multidimensional analysis, but I think those concepts are useful here. So one, so what's a segment? A segment is a slice. So of all the customers in this industry, or all the customers who did deals in this deal size bucket, um, or all the customers in this country. So those are all segments. A cohort is a group defined by a shared event and tracked over time. So for example, the cohort of opportunities that became stage 2 opportunities in Q2 of last year, that would be a cohort, or a cohort of customers who signed up in 2023. So I'm picking a year instead of a quarter, right? And by the way, these 2 things can use together. They're actually orthogonal concepts, right? I could do a cohort analysis to say what happens to the ARR over time of customers acquired in 1Q 2023. So the cohort is defined by the timestamp. And then I could say, show me the telecom slice, show me the finance slice, or I could say, show me the big company slice versus show me the little company slice. So these are in fact orthogonal concepts. But the basic idea to me is segment means shared attribute, i.e., slice. Cohort means group anchored to some moment in time and tracked longitudinally.

In fact, Dave, I live in a world of segments because when I do benchmarking, I know that there's going to be more correlation of a particular metric to a particular segment. Give you an example. ARR per employee can look very different in a $10 million company than a $100 million company. CAC payback period looks very different for companies that sell a $5K product versus companies that sell a $50K product or a $200K product. So, I've always said best practices, don't be looking at benchmarks to help inform your own decisions. And you're less, unless you're looking at them on a segment-by-segment basis or even a combination of segments, size and ACV. Now, on the flip side, In benchmarking, I almost never use cohorts because I don't have access to going back and saying, okay, what was your net revenue retention for customers acquired in 2021 versus your net revenue retention for customers acquired in 2024? Now, that's going to be very important to get insight into what's driving different NRR performance for different cohorts. And it's especially important as a former head of sales I use cohorts a lot with win rates, but we'll talk about that more in a minute.

So, so let's go back. So first, what you say makes sense, Ray, which is in the benchmarking world you want to be doing a lot of segmenting, right? Because it's less useful to me to know the ARR per head of all companies in the benchmark survey than of the companies in my size range. If I'm an $8 million company, I'd like to know what ARR per head for the $5 to $10 million bucket is. That makes a ton of sense. My favorite example of the segmentation you were talking about is like— my favorite example is CAC versus growth rate, because to me CAC goes up with growth. It should from first principles. So I want to compare my CAC ratio to that of other companies in my growth bucket. So if a company is only growing 10% and they have a high CAC, I have a huge problem with it, um, and the benchmark will reveal that. Whereas a company could actually have a very high CAC But if they have a very high growth rate, a lot of that CAC ratio might be for basically future capacity that's not producing right now. So it drives the ratio through the moon. But when you segment the benchmark by growth rate, you can see that behavior.

Yeah. Dave, I wish you hadn't reminded me because I was at a board meeting and our CAC ratio went up, but our growth rate was about 15% higher than our competitors at median. And the board member challenged me on the CAC ratio. And I said, yeah, but our growth rate's higher. So you need to make a sacrifice. And they kind of looked at me like, why did you just challenge me in a board meeting on that? But I totally agree with you.

Are you still on that board, Ray?

Nor did I keep the job very long, Dave.

I don't know. I have this board turnover issue. No, I think the one I see, Ray, is the opposite where we used to tolerate a CAC of 2.0 when we were growing really quickly, but growth slowed down and we still want a CAC of 2.0, in which case it's like, hey, wait a minute, that, that 2.0 CAC was justified at an 80% growth rate, but at a 25% growth rate is really not. Um, so anyways, those— and that whole conversation was about segments. Now let's switch gears. Now we're not gonna talk about cohorts just yet. We're gonna talk first about dashboards, um, because dashboards do something— I'm not even sure we're mentally aware of it, but if I show you a dashboard with our win rate In 3Q of this year. It's important to remember, like, like a board member may not even know this, Ray, that what is that measuring, right? And this is what I would call a snapshot analysis with a long aperture opening of a quarter, but it's still a snapshot, right? But I just told the camera to stay— keep the lens open for a whole quarter. But it's a snapshot analysis. Well, the statistician— oh, here we go with emphasis again— would call A cross-sectional analysis. So what does it mean? I always think of it this way. I sometimes call this milestone analysis too, because my visual metaphor is I'm sitting on a milestone on the road with a clicker, and every time somebody goes by, I click. Like, how many stage opportunity— how many stage 4 opportunities got created this quarter? I sit, I sit at stage 4. Every time somebody crosses, I go click. I have no idea when they stage zeroed, when they were created. I have no idea how long they were in stage 1 or stage 2. I know one thing, I'm sitting on my stone, and every time Every time a guy goes by, I click, right? That's in effect what we're doing for a dashboard win rate analysis. We're sitting on a milestone for a whole quarter and we're counting every deal that's won, every deal that's lost, uh, or every deal that's no decision, right? But we know nothing about where those deals came from. They could have been created this quarter and killed this quarter. They could have been created a year ago and won this quarter or lost. I would say a terminal state. So they're going to come to some terminal state in the quarter. But we, we don't know anything about their history. And I think this can cause misinterpretation of what people are looking at. Thoughts?

Yeah, let me give you a real-life example of that. You might be looking at a snapshot of your win rate, and for the last 3 quarters it was 22%. You're like, hey, I'm not improving, but I'm not getting worse, so we're okay. But if you look at that on a cohort basis and say, okay, for those opportunities that were hit some minimum level of qualification. Let's say they were qualified stage 1 or stage 2 opportunities. I found that the ones that we qualified in Q1 are closing at 28%, but the ones that were closing in Q3 are at 16%. Immediately in your mind, I'm like, well, something's wrong. Wait a minute. We changed the discovery process in Q3 and we weren't quite as rigid. And what we passed over to the AEs because they need more sales calls. So that's an example where a cohort analysis would have really provided more insight into something's not working as well as it used to.

Yeah, so, so we haven't quite defined a cohort analysis yet, but Ray's teeing us up for that. The issue with this sort of snapshot analysis is you don't know anything about where stuff came from. And one way you can view the snapshot is as a series of cohorts passing through. Right? Like, there's a bunch of opties that pass through that were created in Q1 of last year. There are a bunch that pass through that were created in Q2 of last year. There are a bunch that pass through that were created in Q3 of last year, right? But when I'm hitting my clicker, I'm not logging that, right? I'm just going click, click, click. Another one was lost, another one won, another one no decision, right? And when you do cohort analysis, you kind of take the other view. Rather than sitting on the milestone, you hook yourself to the cohort and say, hey guys, we're all the opties that were created in one queue of last year. I'm gonna ride along with you and see what happens to you. And some of you are gonna die early, and some of you are gonna win, and some of you are gonna, you know, derail and go to nurture and get won 2 years from now. But I'm gonna ride along with the cohort to see what happens. And to interpret data, Ray's example was one where interpretation was made easier by the cohort analysis, because say your process is changing, as per Ray's example. Well, the people in the Q1 cohort went through a discovery process than people in the 2Q cohort, right? So maybe they will have a higher or a lower win rate depending on the effectiveness of that process. Thoughts before I define cohorts, Ray?

No, the only thing I would say, and maybe this is putting the cart before the horse, but those snapshot things, you know, I can do it right after the quarter ends, so at least I have some insight.

No, no, snapshot analysis, you've got it all, Ray, right? There's no some insight, right?

Oh yeah, but I, what I, if I didn't do it by cohort, I wouldn't have known that some of my opportunities that we qualified in Q3 weren't performing as well. That's all.

Got it. That's true. 'Cause the cohort has to kind of come to resolution is the way I think of it. Right. That's what you're talking about, I believe. But technically speaking, on July 1st, I know everything I'm going to know from my dashboard for Q2 win rate. Right. 'Cause all I had to do was sit on that milestone and watch things come to a terminal state and say which bucket they went to. So, and by the way, that's one of the things we like about snapshot analysis, because when it's done, it's done and it doesn't change, right? The cohort analysis is the opposite. If I strap myself to the Q1 lead cohort, well, first I'm not going to know anything for a quarter or 2, right? I have this built-in lag because people have to go through the sales cycle. I have to get qualified out. They have to get through to a demo. They have to get through negotiation. They have to get won or lost, right? So the cohort measures are inherently more lagging because I'm riding along with a group of people. The other issue is they change, right? Because as the cohort continues to resolve itself until the last guy in the cohort has hit the terminal state, if you say, what was the win rate for 1Q21 created optis? I can't say. Or I can say, well, provisionally it's 21%, but until that last one hits a terminal state, I don't know. So a snapshot analysis, we like it because by default it's what most dashboards produce. You just say, I'm going to sit in Q2, I'm going to watch everything go by and count it, right? So it's— you can get it, you can get it fast, it won't change once you got it. Cohort analysis is more meaningful, right? Because you're kind of riding along with leads that were maybe created by a different marketing VP with a different message targeted at different people going through a different process. And you can kind of map the— oh wait, we switched the CMO in Q1. Oh wait, we put in new stage definitions in Q3, right? Like that, you can watch the cohort pass through those changes. So cohorts, really, if you want to think about them, they're more like yields. And the easiest way I think of cohorts is I always do the stage 2 opportunities. I want— or even stage 1. I'm going to know the cohort of either MQLs or stage 1 opportunities created in a time period, and I'm going to ride along with them. And it's inherently over time, right? Because they're going to close over time. Some will close this quarter. Let's just say I'm doing a close rate, which is a cohort-based win rate in Kelloggian nomenclature. So I want to know what percent of the opportunities eventually close. And I say close rate instead of win rate personally to disambiguate it from win rate, right? Because everyone else calls the other thing win rate. So I'm going to call this close rate to signal that it's not a win rate, it's somehow different. And, and the issue is if you ask me the win rate for the, you know, the, the Q1 26 cohort, right? I'm not gonna be able to tell you much right now, right? Because it's only 2Q26, right? And we're gonna have to ride along with them for a long time, and those numbers will be changing over time as some are won, some are lost, some are derailed. But the, the cohort analysis to me is more meaningful. It produces a yield, which is in the fullness of time we're going to win 20% of these, right? It also tells me when we're going to win them. Right? Normally, it's a bell curve. Win very few early, then there's a fat middle, then we win some at the end. So, it's much more useful for kind of forecasting and driving the business. Thoughts?

Well, I also think it can be more effective for forecasting what pipeline you need to close deals. And this kind of gets into the pipeline coverage ratio. And we did an episode, gee whiz, over a year ago that Pipeline coverage ratio is not just the inverse of win rate. And I don't know, do you want to talk a little bit about how you can use cohort analysis to have better pipeline generation insights, Dave?

So, there's 2 things there. One, the post you just mentioned is something along the lines of win rate, 1 over your win rate is not your pipeline coverage goal. And the reason for that is win rate is a snapshot-based metric. And you— some of, some of the opportunities were in there very well aged and going to be won or lost soon. Some are fairly new and may have actually been put in with the, the wrong close date in the quarter and they end up slipping or whatever. But, but the main idea is if you want to know what your coverage should be, you should look at your week 3 conversion rate, which is also a snapshot-based metric. But it says, hey, um, in any given quarter If we sold this much in ARR, how much in week 3 pipeline do we have? Let's divide one by the other to get the win 3 conversion rate. So that's one post. Those are both snapshot-based metrics. The, the close rate one is more interesting, Ray, because remember, it's not just telling you what percent of them are won, but when. So you can imagine the pipeline in the same way, kind of like in any given quarter, it's almost like a series of waves rolling through the quarter, right? There's a wave of the stuff created 4 quarters ago, 3 quarters ago, 2 quarters ago. And they'll all be converting at different rates. And more importantly, looking for the other direction, if I create 27 opportunities now, I kind of know how much pipeline I'll have and when. I know how much of it's going to create this quarter pipeline versus next quarter pipeline versus quarter after that pipeline versus quarter after that. So your actual pipeline in any moment in time is an overlay of all those cohorts, right? It's this notion of waves passing through or just an overlay. And yeah, I can fork— if I want to forecast pipeline 2 or 3 quarters out, That's exactly how I'm going to do it.

Got you. We spent a lot of time on pipeline. Should we also talk about how you can use cohort analysis for customer expansion and NRR, Dave? Is, is it a good transition point?

Yeah, I think it is. So first, let me just do the basics. People don't realize this. NRR is a year-over-year metric. Year over year, one year. It's, it's this year to last year, right? And, and you're like, yeah, but there's people in the la— in the last year that have been with us for 7 years, and there are. Right. But for the purposes of NRR calculation, we just care about, you know, on January 1st, '25, what were they compared to what they are on January 1st, '26 and how long they've been a customer. You know, on Jan '25, we don't care about what their history before Jan '25. We don't care about when you calculate NRR. It's a 1-year lookback. And once again, it's blending cohorts because you've got a customer cohort that's 6 years old, 5 years old, 4 years old, 3 years old, 2 years old. Right. In that. So you have this kind of weighted or blended average of different cohorts. and how they're behaving. And I think NRR is kind of short-sighted basically because it only looks back a year. And if you do cohort analysis, which is a table we've all seen, it's kind of an overhanging table where you put when they became a customer and what their ARR is, right? In the kind of cell 1, 1, you put like, you know, Q1 many years ago and you put how much ARR they had. And that's the cohort of customers, the people who signed up in Q1 of year 1. And then you watch them over time and every quarter you put in their current ARR value. And then you can look at customers behave in the fullness of time. It's an overhanging chart because, right, you have less time, right? For everybody who starts a quarter later, we have one less quarter of history. So that chart in my mind is far, far, far more useful for predicting long-term customer behavior. And that is a cohort analysis than NRR, which is kind of inherently a snapshot analysis. Yeah.

Yeah, another way that I've actually used it in practice in NRR and GRR, and it really was valuable because of cohort, was when I implemented new processes. Things like, I have a new onboarding process and it started in Q3 of '24, or I started doing more in-app recommendations to get our users to more high-value aha moments. By actually doing cohort-based analysis for both GRR and NRR, I was able to look at what the numerical impact was of those new processes. And it also showed me a couple of times, Dave, that, oh, I find that this sweet spot for getting expansion deals is between months 18 and 30. So I might overallocate maybe additional sales resources or programs to go after those existing customers months 18 through 30. So 2 other examples were cohort.

Okay.

Can really help.

So because we're the Metrics Brothers and we're obligated to bicker even though I know we're near time, I— you said one thing that I got to call you on. You said you do that cohort analysis for NRR and GRR, and that has to be incorrect because NRR and GRR are inherently defined as year-over-year metrics, right? So let me just correct what I think you meant to say, which is I'll do the cohort analysis using ARR both including and excluding expansion. But with the number in the cell is ARR, right? That is correct. And you're looking at how the ARR progresses over time, sometimes looking only at shrinkage, sometimes looking at the net of shrinkage and expansion. And that's what you meant by NRR versus GRR. But just technically speaking, we both know NRR and GRR mean this year over last year, right? And what you're— so it's almost antithetical to say I did that for the last 3 years. So anyway, I just wanted to clarify that for the customers. I knew what you meant. And I like that. Ray's very good here picking applications of how you would use this stuff. And one of the beauties of cohort analysis, as we talked about earlier, is you know when things changed. So when you interpret the data, you can say, ah, that's when we change this, or that's when we change that, or that's when we launch product number 2. Look at all the NRR rates go up, or look at the ARR go up. I did the same fucking thing. Look at the ARR, the inclusive expansion ARR. or the net ARR go up in that table. So I think, Ray, um, the other thing I'd say is you could either use time and your knowledge of time to do that, or I'll give you another example. Just say you think SMB is a bad segment for you and you get out of it. You could do a cohort-based analysis and cut it to say, here's, here's the cohort-based analysis for SMB customers in the 1Q21 cohort and for mid-market customers in the 1Q21 cohort.

Oh, Dave, and you know, we got to wrap up here, but I don't know if we've emphasized this enough. It's not either/or. Often the best practice is an and, to your point. You want to have it done on a segment-by-segment basis. That might be what products they have, what vertical they're in, how much they actually bought upfront, 1 product, 2 products, 3 products, and then overlay that with the cohort analysis, the time-based. Those together are the best.

Yeah. So you're saying what I would personally describe as a segmented cohort analysis, where we're going to look at the telecom segment versus the finance segment and we're going to take the cohort from 1Q21 and watch how they behave over time. So basically cohort analysis to net this out, folks, is over time. Segment analysis is cut. These 2 concepts are orthogonal. You could do one or the other, or as Ray just pointed out, both. I'm a little less bullish than Ray on doing everything both ways because you can drown yourself in data. So I think you, you do both with care, but, but certainly you can do both.

Okay, Dave. Hey, thank you for this. And by the way, I have to highlight, you've written so much about this on Kellogg. I'd encourage everyone to do a search for whether it's win rates or pipeline or cohorts. I think there's 4 or 5 different articles that people get a lot of benefit from, Dave.

Well, thanks, Ray. Thanks for mentioning that. Uh, we'll see you next time, folks.

Bye-bye, everyone.

The Metrics Brothers: AI, SaaS, and More is a production of the Metrics Brothers Growth and CAC and a member of the Benchmarket Podcast Network. By accessing this podcast, you acknowledge that the Metrics Brothers make no warranty, guarantee, or representation as to the accuracy or sufficiency of the information provided or the humor content of the jokes told. Ray, information, opinions, and recommendations presented are, according to our spouses, probably wrong and provided for information purposes only. This podcast should not be considered professional or for that matter, unprofessional advice. We disclaim any and all liability for any direct, indirect, indirect, misdirect, incidental, special, ordinary, consequential, inconsequential, or other damages arising out of any use of, or God help you, reliance upon the information presented here. Ray Grothreich is based in New York City and available on X @rayreich. Dave Kellogg is based in Silicon Valley and available on X @davekellogg. Kellogg. Schedectity, which is French for unspellable, is not our actual production location. Thanks for listening.