Klaviyo S-1 Metrics Breakdown
14 Sept 2023 · 26 min
Dave Kellogg and Ray Grothreich break down Klaviyo's S-1 filing and find several non-standard metrics. Klaviyo's MRR definition ignores churn and expansion, its CAC payback uses a 12-month revenue change applied to all ARR, and its NRR calculation smooths monthly point-in-time figures in ways the hosts say mask volatility and churn.
Chapters
Live from Schenectady, New York, it's SaaS Talk with the Metrics Brothers, Growth and CAC.
And I'm Growth, better known as Ray Reich, founder and CEO of Benchmarkit.
And I'm CAC, better known as Dave Kellogg, EIR at Balderson Capital, independent consultant, and author of Kelblog.
And together we are the Metrics Brothers.
Yes, we are, Brother Ray. So last week we talked about GRR, found a few interesting rabbit holes to dive into, including the whole ARR versus ETR-based churn rates. Encourage everyone to take a listen to that episode if they haven't heard it yet. But more importantly, what do we have on tap for today, Ray?
Well, we are SaaS Talk. So what's everyone talking about in this SaaS and cloud industry today? It's the Klaviyo S-1. And I'm hoping I'm saying the name right, Dave. Is it Klaviyo? Is that right?
You say Klaviyo, I say Klaviyo. You say Tomato, I say Tomato. I think it's Klaviyo in this particular case. I think the A is long.
Okay, let's go with Klaviyo. But I wanted to take a look at their S-1, but not like everyone else has been doing. Let's do it from what we know best, SaaS metrics, including ARR, CAC payback period, gross revenue retention, GRR, and net revenue retention, NRR, because I really am intrigued by how they define some of these metrics.
How's that sound, Dave? Yeah, it sounds great to me, Ray. There's a lot of good S-1 breakdowns on Klaviyo. Then you got me doing it on Klaviyo. There's a lot of good S-1 breakdowns on Klaviyo already. My favorite's probably from Maritech. So let's not try to replicate one of those, but let's drill in on these 4 metrics that we have found and already talked about, because I think there's a lot there. But hey, before we do that, Ray, Let's hear a quick word from our presenting sponsor, Maxio.
The Metrics Brothers is proudly sponsored by Benchmarkit, the B2B AI and software industry's largest, broadest, and most contextualized benchmarking index. Dave Kellogg and Ray Groth Wright have put the Metrics Brothers podcast on hiatus for the remainder of 2026. All episodes we have ever recorded are still available on your favorite podcasting app. You can reach Dave @Kellogg and Ray on LinkedIn @RayReich. Thank you so much for listening to the Metrics Brothers podcast. Okay, Dave, let's go into the easy one first, and that is how they were defining ARR in the S-1. So instead of me reading it to you, how about I just throw it over to you and get your initial perspective on how they're defining ARR?
Sure. This will be the one simple and clear thing I say on the podcast episode. ARR is defined by Klaviyo as 12 times MRR. Mic drop. Okay.
Okay. Well, that's the easy one again. I can't believe I gave that easy one to you, but softball. I'm gonna have to, I'm gonna throw you a harder one. And since it's 12 times MRR, what do you think about how they define MRR in their S-1? Because it had something a little bit unique, right?
It did. I knew you were going to ask me this, Ray. You wouldn't let me off the hook so easily. So yeah, they pick a very standard definition of ARR, MRR times 12. The fly in the ointment here is the definition of MRR. And they define that in an interesting way as the amount of revenue that we expect to receive in the next monthly period for our existing paid subscriptions. So let's just break that down. So the next monthly period for existing paid subscriptions, so there's no forecasting of new customers, right? It's kind of a forward-looking MRR metric. It's about Next month, but next month not including new customers, which is good. Assuming no changes to subscriptions in the next month. Okay, so we're going to assume everybody is going to keep acting next month like they did this month. We measure MRR at point-in-time calculation measured on a date. Good. Here's interesting. MRR is a legal and contractual determination made by assessing the contractual terms of each paid subscription. MRR is not determined by reference to historical revenue, deferred revenue, or other GAAP measures. So in my mind, I'm going to net this out. They could ask the kind of data science or forecasting team what next month's spend is going to be. And I'm pretty sure they have a team that knows that answer. That's not who they ask. Their definition of MRR is they call legal, they call finance, and say, given the contract, how much is Ray going to spend next month?
That makes sense, especially they're primarily month-to-month agreements in their legacy customer base, those SMBs and lower mid-market. So, but I would think, Dave, that they might, they have month-to-month agreements, primarily 30-day cancellation periods, most likely that they would actually have some knowledge of customers that were going to churn in the next month. But it appears they're not including that known churn for next month. Would you agree with that?
Yeah, I don't think they're including any forecasted expansion either. Right. So say I was right at the cusp of one of their messaging tiers, because look, their pricing model has a number of drivers. In it. One is subscription tiers determined by the number of contacts in your database and/or the number of email or SMS sent. So it's very much kind of a usage-based model. You have more contacts, you pay more. You send more messages, you pay more. They also have communication channels. So if you do SMS and then add email, or you start with email then add SMS, that's going to cost you extra. They also have new products. They're like— they have a CDP now and they have a reviews product. So Basically, there's a lot of different elements in the mix that generate, you know, a monthly price ultimately. But I don't think they're forecasting at all. I think they're literally calling up legal and saying, what does the contract say they're going to spend next month? If the contract changes, if they signed up for a new plan, right, then that would be reflected. But they're not forecasting anything either on the good side, I would say, or on the bad side.
So I got that. So I think we can move on to the next one, which I got very exercised about, and that's CAC payback period. Anything else before we move there, Dave?
No. And since you got exercised, let me— I'm going to punt this one back at you. So we love CAC payback period, right? We define it typically as months of subscription gross margin to pay back the customer acquisition cost. And a good one runs from 12 to 18. That would be kind of my intro to the subject. Let me flip this one to you, Ray. How do they define CAC payback period at Klaviyo? And what caught your attention? Okay.
So I'm going to read it and it says we calculate our CAC payback period as of any date of determination by first calculating the change in revenue. And by the way, it says revenue. So I don't know if that's GAAP or recurring revenue, but it says revenue from the date that was 12 months prior to the date of determination to the revenue on the date of determination. Then they multiply that change in revenue by our gross margin as calculated over the 12 months preceding the date of determination. So since I just read it and you know I'm a little exercised, for the audience, can you tell us what you think they're doing, Dave, or is there other things that jumped out at you?
We're going to have to perform an exorcism at the end of the episode here, Ray. So yeah, happy to answer your question. So Look, in some levels this looks like a pretty standard approach. We're trying to say, hey, there's some notion of revenue, it's kind of ill-defined, but we're going to multiply that by gross margin to get gross profit. And then we're going to look at how much gross profit, how long does it take the sales and marketing expense to pay back the gross profit? So that's a fairly standard way of looking at CAC payback period. The thing that troubles me on this is it's ambiguous when it says revenue. I don't really know what change in revenue from the date 12 months ago prior to determination of the revenue on the date of determination. I think what they mean, and looking at how they do some of their other metrics, is they mean annualized change in monthly revenue, i.e., let's take December '23 revenue, multiply it by 12, and compare it to December '22 revenue multiplied by 12. The issue here, Ray, like any monthly varying SaaS company, is they don't have ARR, right? All they have in this case, it looks to be very much a month-to-month business. They have MRR, so they multiply by 12 to get ARR, right? Which is their definition. But they don't actually have a thing in ARR, so they need some proxy for ARR. And I think they're doing— there's a lot of different ways people do that, right? Snowflake does it with the 12 months, you know, the prior 12 months compared to the 12 months before that, right? When they— the trailing spend method, my favorite method. These guys are doing the annualized method, and I think they're doing it over the last month. So 12 times December '23 minus 12 times December '22 is change in revenue. In some ways, this could also be called implied ARR. But back to you. Okay.
That's a pretty logical way, right? So I get that, but maybe it's because, you know, the work that I'm doing with the SaaS Metrics Standards Board, when we think about customer acquisition cost, we think about how much did it cost you to acquire the number of customers times their ARR? Then you divide that by the sales and marketing expenses over the same period. And that's how you get CAC. And then I use the same thing for CAC payback period. And what this is saying, Dave, and I think we differ on this, is it's not just new customer, Ray. It's really the growth in MRR times 12 for all customers that were customers 12 months ago, correct?
Yeah. So, so we're irked by different things, which is good because, because we can learn from each other and the base can learn from what we're irked by. I'm irked by the ambiguity in the definition, frankly, uh, and you're irked by the fact that they're doing it for All, basically all ARR, not just ARR from new customers, right? So the active listen to what you said, hey, I calculate CAC by taking sales and marketing expense divided by number of new customers, and that gets me $1 per new customer. And when you do it that way, which I know both you and the SaaS Metrics Standard Board do it that way, then yeah, you could look at this and go, wow, this is kind of cheating because they're looking at changing all ARR. Right. What I would say in response is I actually do it the other way by default. So my shortcut calculation for CAC payback period is CAC ratio divided by subscription gross margin. And by default for me, CAC ratio is for $1 of new ARR. It's not for $1 of new ARR from a new customer. So I must confess, I at the first order at least calculate CAC payback period on all new ARR, not just on new customer ARR. But I can certainly see how you do it differently and why it would be bothersome to you. Moreover, I would say when you read the S-1, They pretty clearly say, you know, hey, there's proof of our highly efficient customer acquisition strategy. Those exact words is that we have an only 14-month CAC payback period. And it's a little misleading given how they do the calculation because the calculation isn't purely for new customers.
It is because customer acquisition to me says I'm acquiring a new customer. But believe it or not, Dave, and you know how excitable I can be about metrics. I know it's kind of sad, but it's true. The thing that really jumped out to me about CAC payback period was that it was 14 months. Because if I just looked at it as 14 months and I didn't understand the segments they served, it's like, okay, that's not too bad. But then when they say they do $585 million of revenue and they have 130,000+ customers, I just divide $585 million by 130,000 and I get the average revenue per account is about $4,500. And I'm like, whoa, wait a minute. From all the benchmarking I've done, if you've only got an average revenue per account of $4,500, your CAC payback period better be under 12. And I know that their other financials are so good, but that jumped out at me as like, I don't get it.
Yeah. So I think, I mean, with due respect, I think you're making one mistake or maybe overlooking one thing. I mean, first, so in general, let me agree with you. If your average customer is $5K a year, you're probably in the SMB SaaS land and you live in a land of very high churn. And the rule of thumb is don't spend too much to get a customer because they won't be with you that long. Okay. That's kind of the SMB SaaS mantra. And if you're spending 14 months of gross profit to get a customer, maybe that's a bit high for somebody who might not be with you very long. So on the flip side, I would say when you look at Klaviyo's business, these people provide basically the engine upon which e-commerce retailers run their business. So if you look at, you know, changing software as, you know, is it kind of a knee transplant versus a heart transplant? This is a heart transplant. If you're an e-commerce vendor and you want to get off Klaviyo, it's a big, big deal because they have all your customer data, they have all your contacts, they have all your history. They're making predictions. They may have a CDP of your customers, right? This is not an easy system to get off. So I personally think it's going to be sticky. And I think we'll see that when we look at the churn rate later on when we talk about GRR. It is sticky for SMB SaaS. It's actually quite sticky. So therefore I think it's worth it because ultimately that CAC payback period should be related to what did I get for my money? And I think they're getting a customer who's going to be with them for a while.
Yeah. The other thing I walked away, and this is assumption, but as I read the entire S-1, they've really made a significant focus on enterprise customers. They define that as $50,000 a year and over. So my assumption was That over the last few months, maybe it's the last 12 to 18 months, they're really going after enterprise-class customers, and that has increased their CAC payback period. And it talks about, or to me it suggests what you and I always talk about is the importance of segment-based metrics calculations. I would love to see the CAC payback period for their SMB accounts versus those new enterprise accounts that they're bringing on board.
Yeah, I think it's a reasonable ask. I do agree they are definitely focusing upmarket on the bigger customers, and that's perhaps driving up the CAC payback period because the bigger customers are presumably not only bigger, so you get more short-term reward with that, but more importantly, you get a longer lifetime customer because they're, they're going to be bigger and less likely to churn because they're not in a high-churn segment. So I agree with the analysis. I, from my reading of the S-1, I don't know if there's data there to support that or not, but I think it's It's a, it's a great question.
Now, this is one of the differences between us metrics brothers, right? You are a little more precise, and I really love that about you. So is there anything else that we should dissect here on CAC payback period before we move to the next metric in the S-1, and that's their net revenue retention?
I think it's okay. I think we hit everything that I wanted to hit. So let's go on now to NRR. If you want, I'll start I'll just jump right in. So I'm going to read from the S-1. We calculate our dollar-based net revenue retention rate by first identifying the cohort of customers as of 12 months prior to the date of determination. That's good. So we now know they're not doing a survivor-biased rate. They're starting with the year-ago cohort. Good. We then calculate the ARR for this customer cohort as of 12 months prior to the date of determination. Okay. Prior period ARR. What's ARR again? MRR times 12. So we're basically saying we're going to go back a year. We're going to look at, if it's January, we're going to look at what finance and legal, sorry, what legal tells us people are going to spend in February, not what they actually spent, which is kind of weird, but what we think they're going to spend next month. And we're going to multiply that by 12. Okay, so that's where we are. And then the ARR from this customer cohort as of the date of determination. So now we're going to come up to this December and do the same thing. Say, okay, what is finance or legal telling us they're going to spend next month? other contracts. Okay. And that becomes the current period ARR. Current period ARR includes any expansion, price increases, and subscriptions that are activated or deactivated and reactivated. Okay, that's fair. You die and come back to life. That's okay. It's unusual, but in monthly SaaS, it can happen. And also reflects any contraction or attrition over the last 12 months. So it's an all-in metric, as it should be, right? Every contraction, every expansion, every lost customer, every reincarnated customer in their case, all counts, which they should, because that's the essence of a cohort-based metric. Tell me about these, you know, 10 people a year ago. Tell me about these 10 people now. And that will include definitionally everything that happened along the way. It excludes ARR from new customers in the current period, as it should. Again, the essence of a cohort-based metric is it should exclude them. We then divide the total current period ARR by the total prior period ARR to arrive at a point-in-time NRR. Hallelujah, they are done. The problem is— No, Dave, they're not done.
They're not done.
Why don't you go through the next piece, Rayce? Because this is where it gets mind-blowing. In my mind, they're done, but over to you.
Well, and then they say, and it's that last sentence that really threw me off, we then calculate the weighted average point-in-time NRR as of the last day of each month in the current trailing 12-month period. To arrive at the NRR with the weightings determined by the total ARR at the end of each period. What the hell does that mean, Dave?
Yes. So first I'll tell you what it means. I had to spend like an hour with a spreadsheet trying to figure out what they were doing. In short, what they're doing is just like the thing we did for December. They're going to do that for every month and they're going to say, okay, there's a point-in-time NRR for every month in that trailing 12-month period. And then we're going to take a weighted average of those point-in-time NRRs. Why not just a flat average? Don't know, but we're going to take a weighted average. What are we going to weight by? We're going to weight by the cohort NRR at the end of the period, which strikes me as a little bit bizarre. I mean, it's a very, to me, counterintuitive thing to weight by. But what they're trying to do, I think the normal effect of that since cohorts grow is it backloads it, right? So it's going to be a backloaded average of these trailing weights. So why are they— so that's what it is. And the question is, why the hell are they doing it? And I think the answer is they really want to smooth For some reason. I think that reason is in their business, which is largely a monthly business, things are volatile, so it's okay to smooth. Why they didn't smooth in a different way is beyond me, because this is kind of a bizarre smoothing algorithm, but, but that's what they've chosen to do.
Yeah, but I think it's a really good point because since they have that monthly subscription but variable usage above that, you could see seasonality. Let's say someone who's e-commerce vendors have a lot at Christmas time, you could have an amazing growth in October, November, December, it could go down dramatically in January, February. So they really do try to smooth it out over that 12-month period. So it now makes sense to me why they do it. I'm with you though. I'm not sure how they do it makes sense to me.
Yeah, it's pretty strange. I've yet to hear a good explanation of it as I've been on Twitter tweeting with the SaaS metrics geeks about it. And we all know they're smoothing it. We all know why, because any given customer can be volatile or any given cohort can be volatile. But why this algorithm? No idea.
And by the way, they have a great NRR and I think they listed the last 6 to 8 quarters and it ranges, you know, 117, 119, 121. So it's a really healthy NRR. So good on 'em.
Yeah, I agree. And, and, and all I think this smoothing does is kind of take the year and backweight it a bit. So, so I don't think, I think it's a pretty accurate number. I think it's smooth in a bizarre way. But, but do I think Klaviyo has a strong NRR? Yeah, I do. do, particularly for kind of SMB SaaS. So that's great. This, this leads us now, Ray, to the last metric in this episode, which is GRR. And this is where things get funky in my opinion. Do you want to do the intro on this one or you want me to?
Well, let me do the intro because I'm passionate about this one. Maybe a little emotional. I know it doesn't seem like a guy who loves data and math would be emotional about something, but I am. So it says, and I'm going to say how they calculate it. We calculate our gross revenue retention, GRR, in the same manner that we calculate NRR, net revenue retention, except that GRR only includes the impact of customer losses, churn, and does not include the impact of customer expansions. That's great because that's what NRR is for. But it also then says, or contractions from the relevant customers. So Dave, They specifically don't include contraction, whether that's downsell or less variable usage in their GRR. That was of concern to me.
Yeah, and it should be. It's unusual. And at first blush, it looks like, you know, kind of BS to say, wait a minute, how could they possibly exclude contraction? Because as defined, they're only counting people who basically contract to zero, right? So it's kind of almost a logo churn metric. It ends up this kind of bizarre weighted logo churn, which is weird. If you want logo churn, just get logo churn. So I don't love this as a metric. I can't say I figured out why they're doing it. And by the way, this also suffers from the same crazy weighting as NRR. So, so in addition to having this kind of more fundamental flaw, which is we exclude contraction, we're also then gonna get an answer and then weight it in this bizarre way. But I think I know what they're trying to do. And I think what they're trying to do is damp out volatility. And I think, well, sorry, damping out volatility is accomplished by the weighted average thing. I think on top of that, they're doing something else, which is they're trying to avoid aggregating negative fluctuation. And what do I mean by that? I think in monthly varying SaaS, the reality is people go up and down and there's kind of signal and noise. And the issue is when you go and an NRR is somehow inherently fair in monthly varying SaaS, because you're letting those fluctuations offset each other. When you tie one hand behind your back and say, wait a minute, you can't count expansion, it seems unfair to make you count shrinkage. I'll give you an example. It's a kind of a trivial degenerate example, but it makes the point very clear. Imagine a customer who just alternates between 10 and 8 every month. So in January, I spend 10, February 8, March 10, right? 10, 8, 10, 8, 10, 8, 10, 8. I do that forever. If you and I look at that customer, we would both say his or her there NRR is 100% and their GRR is 100%. They never change, right? They fluctuate, but they never change. Now, if however I did a GRR definition where I excluded all those plus 2 expansions, right? Because while I'm doing NRR, not GRR, I need to exclude expansion. I'm going to aggregate only the negative side of those fluctuations. And I'm going to say that that customer generated negative 12 units of churn on the year, right? Because they shrunk 12, a total of, they shrunk twice. 6 times, right? So this is what they're trying to avoid. So, so I, I kind of applaud what they're trying to do. I think there's much better ways to do it. Like for example, using trailing span, but I understand what they're trying to do. I don't, I don't like it though, frankly. I, I don't like it either. And I think there's a better way to accomplish the goal.
Yeah. And if they're trying to accomplish that, like you said, you could look at 12-month trailing, right? Because it's GRR in my perspective is an annualized metric. So you can look at even the last 12 months versus the previous 12 months, right? Kind of that 2-year look back, like you said. There's a couple ways to do it, but I actually just think that this is trying to smooth something that they're looking at as a monthly metric times 12 versus an annualized metric. That's my perspective.
Yeah. And I think there's both things going on. I mean, I think there's definitely the smoothing, which is happening by the weighted average, but I think specifically the exclusion of contraction. Is a result of monthly varying SaaS and being forced to exclude expansion. And they're saying, if I have to exclude one, I'll exclude the other. And I understand why, because it is my opinion, a mistake. It is not an accurate representation of my fluctuating customer to say that they generated 12 units of churn because they didn't, right? They generate exactly zero. If you use trailing spend, you'll see that every year they'll generate whatever, 6 times 108 units of ARR.
Yeah.
Right. So I get what they're trying to do. I don't think it's unfair. I think it's unusual. I think they should use trailing spend. I think we agree on the solution is if you use trailing spend and either, either 12 months or I can even in my little example, well, annualizing won't work, but if I, if I annualize half, so it'll work in my example. So, so I think there is a way to use trailing spend, either trailing 12 months or typically trailing 6 annualized or trailing quarters annualized.
Yeah, totally agree. But you know, already over 20 minutes. And, you know, we only dealt with 4 metrics that we kind of looked at from the Klaviyo S-1. And just quick summary, this is why benchmarking is so darn hard sometimes because it's different definitions from company A to company B. But here we are, MRR, they looked at expected next month versus actual last month, CAC payback period. We talked a little bit about why segmentation is so important. The NRR, we actually really kind of liked it except for that weighted average thing, which we explained. And then GRR, we couldn't really explain why they excluded contractions. Anything else we want to summarize and end today with, Dave?
Yeah, well, I think we did explain. For me, I feel like I understand why they're doing what they're doing, excluding contractions. I just don't think it's the right approach to solve the problem. But I think in that one, I do understand why. The only thing I'd add, Ray, to your bigger point is, yeah, there are 259 pages in this S-1 and we talked really about 2 of them.
Right. So, well, that's on my mind. You know, we are doing this. Just so the audience knows, because we wanted to make sure we got another episode out there once a week, Dave and I were spending our Labor Day talking about 4 metrics from the Klaviyo S-1. So we got to be excused that we only did a couple of pages. Dave, it is a holiday.
Well, it's okay. My point is we could talk for, we could talk for another half hour about these same 4 pages. So there's a lot in these documents. I hope you get a chance to read them if you're, if you're into this stuff.
Hey Dave, as always, thank you so much. And I can't wait for our next episode.
Thank you, Ray. Have a happy Labor Day, and we'll, uh, see you next time.
Okay, bye-bye everyone. Thanks for listening.
The Metrics Brothers AI, SaaS, and More is a production of The Metrics Brothers Growth and CAC and a member of the Benchmarkit Podcast Network. By accessing this podcast, you acknowledge that The Metrics Brothers make no warranty, guarantee, or representation as to the accuracy or sufficiency of the information provided or the humor content of the jokes told. Ray? Information, opinions, and recommendations presented are, according to our spouses, probably wrong and provided for information purposes only. This podcast should not be considered professional or, for that matter, unprofessional advice. We disclaim any and all liability for any direct, indirect, indirect, misdirect, incidental, special, ordinary, consequential, inconsequential, or other damages arising out of any use of or God help you reliance upon the information presented here. Ray Grothreich is based in New York City and available on X at Ray Reich. Dave Kellogg is based in Silicon Valley and available on X at Kelblog. Skedectity, which is French for unspellable, is not our actual production location. Thanks for listening.
