“The Strategic Use and Abuse of SaaS Metrics” with Dave Kellogg

Main thing I'm going to do probably is cheat a little bit on going up, just so that way there's no awkward "hit dog" and then I'm like hobbling up the stairs. So I'm going to try and get right on the edge. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. As a courtesy to our speakers and fellow attendees, we ask that you please silence your cell phones. Please welcome to the stage Dave Kellogg. EIR at Balderton Capital. Hey everybody. Thanks for joining us on this lovely afternoon to talk about one of my favorite topics, the strategic use and abuse of SaaS metrics. And before I jump into it, I'd like to talk a little bit about why I wanted to give this presentation in the first place. So, and the answer is, I've been a C-level exec for over 25 years. I've sat in on, or sat on 10 boards of directors. I've literally been, I started counting them, like 1,000 executive staff meetings, QBRs, board meetings. It's terrifying when you start to count how many of these things you've done. And I've seen a lot of good, but I've also seen a lot of problems. And some of those problems are tactical. We'll talk about them another day. Like, you know, how we present SaaS metrics. You know, charting, information overload, formatting. I care a lot about those topics, but that's not today's subject. I'll do a talk on that in a few months. Today's subject is strategic and kind of the strategic use and abuse of SaaS metrics. And to me, it all comes back to this one kind of core concept. Which is the numbers don't lie, right? Good expression. But sometimes the stories we tell around them do. And we're going to talk a lot about those stories and how they get twisted from the numbers today. Before we do that, let me just introduce myself. On the left-hand side, you can see my kind of past employment, my operating background. In short, I've been CMO of 3 companies and CMO of 2. Most of my work is in the $0 to $100 million range, but I've also worked from $100 million to a billion as the CMO. And I was a GM at Salesforce of a $500 million division. So I have a lot of operating background, marketing, CEO, but very much a numbers guy. Kind of a closet CFO, if you will. Closet FP&A person, if you will. I love doing the models and the numbers. So on today's work, I'm an EIR at Balderton Capital, which is a London-based VC firm that invests in Europe. I've been a director of a total of 9 companies. Depending on how you count. Advised over 20 companies, including those— the greatest hits. Angel investor, again, greatest hits. And author of a blog called Kellblog, which I hope you get a chance to read, 'cause it talks a lot about SaaS, strategy, marketing, and of course metrics. So that's me. The 3 topics we're gonna hit today are as follows. First, and don't be scared by that number, The 15 Signs of a Strategic SaaS Metrics Problem. And yes, we're going to spend a lot of time on the problem today. We'll spend some time on the solution. But we're going to spend time on the problem first because it's fun. We get to talk about the silly things we do and the silly things we see. And second, because by understanding the problem, hopefully that will lead us to a solution. So we're going to go through the 15 signs of a SaaS metrics problem. Then I'm going to introduce a SAS Metrics maturity model I built. It's a little 5-level, 5-layer model designed to help you go, "Oh, I think my company's roughly here. We're at level 2. And if we want to get better, you know, we should work towards level 3." And then we'll look at the things you do in each level to try and move the ball forward. So that's our agenda today. Let's jump right in and start talking about the 15 signs of a SAS Metrics problem. The first sign, one of my personal favorites, is what I call bludgeoning, where SaaS metrics are used not as a tool of understanding, right? We're not trying to make the business better. We're not trying to understand our business. We're trying to club people with numbers, right? And boards do this to CEOs. They club, "You're behind plan. You're over budget. COGS is too high," right? We can club the CEO. CEOs in turn can pass along the favor, They can club the CRO. Quota attainment isn't where it's supposed to be. Productivity's too low. And you know it's really a case of bludgeoning when somebody goes, you know, we missed the lead number. And the CMO's like, we haven't counted leads in 2 years. We don't care about leads. We're all about pipeline, right? So you're literally clubbing me just for the sake of clubbing me with a number that isn't actually relevant to my business. So the first sign that there's a problem is kind of this abusive use of metrics to kind of bludgeon people, to use them as a tool of almost oppression, not as a tool of enlightenment or understanding. Now this can lead to our second problem, which is misleading, where people use SaaS metrics to mislead people in numbers of ways. One might be because they're afraid of being bludgeoned, right? The other might be because they don't have what I would call a functional relationship with the board, and they see the board meeting as something to survive. Right? We don't actually want to have an open, honest conversation with our board and look at the numbers together and try to understand what's happening. We just want to survive that meeting, feed them some BS, and go back to work. Live to fight another day. This is a reason that some people do misleading. Right? There's lots of different motivations behind it, but those are two popular ones I've seen. I'll give you the world's simplest example of misleading. Which is this table. It's a perfectly good table. Nothing wrong with that table. Actual and plan for last quarter. It's a little ARR bridge. Starting plus new minus churn equals ending. But somebody stands up and says, we had a good quarter last quarter. We ended at 96% of plan. How do you feel about that? I feel angry when I see that, right? Because we didn't end at 96%. We used a $12.5 million stabilizer right, called starting ARR to damp out the fluctuation in plan performance. That's what we actually did by saying we're 96% of plan. The CFO says it, I'm maybe a little angry. The VP of sales says it, I'm super angry, right? Because the CFO gets to talk about results sometimes more than drivers. So, you know, what we should be talking about on this slide is this, that we're 60% of new logo ARR, dramatic problem. We're 93% in expansion, so not bad. 73% on new ARR. We're actually 100% of churn ARR. Thank you, customer success. But overall, we're 60% on net new ARR, meaning our burn multiple, for people who like that SaaS metric, and it's increasingly popular in this day and age, our burn multiple will be almost double plan, right? Because our net new ARR is almost half plan, right? So this is a kind of semi-catastrophic quarter. And you can simply— and look, the most common form of misleading is what I call cherry-picking. You just don't put up the bad numbers, right? You put up different numbers every quarter. That's the most common form. And this is not cherry-picking. This is putting them all up there, but simple through this kind of sleight of hand of just talking about the 96%, that story that we tell about the numbers is kind of a lie. So that's our second sign of a problem, where people use metrics to mislead. The third, déjà vu. So you go to the same board meeting every quarter. You're like, does anything ever change at this company? I feel like we had this conversation last meeting, and I feel like we had this conversation the meeting before. I think we had this conversation the meeting before that. Right? And while this is not a direct symptom of a SaaS metrics problem, I would argue it's an indirect symptom. That the companies I work with that kind of have their metrics act together don't have the déjà vu problem, because the numbers are telling us What's not working? And look, it may be very hard to fix it. Sometimes these problems take quarters to fix, but at least we can say, hey, we're having trouble in mid-market. We tried this new messaging. We thought it would increase conversion rates. It didn't. So we have something to talk about, right? It's not the same board meeting. It may have the same problem, but it's not the same board meeting. So if you have kind of board déjà vu or QBR déjà vu, right, if you're on the E staff, Same thing can happen in a QBR. Maybe you have a SaaS metrics problem. Rat-holing. I was pretty famous for this one, personally. I'm vulnerable to it. That you, you're trying to have a conversation about the business using metrics. Right? That's the goal of this stuff. Right? We want to talk about the business, and we want to use metrics to do that, but sometimes we end up talking about the metrics instead of the business. So the CMO is up, and they say, "Hey, we generated 1,200 MQLs last quarter." Sales feels like they're starving. They're like, "What do you define an MQL as?" Right? You ever been to this meeting? Half— You have. Half hour later, people are still arguing about it. The CMO is probably gone in 2 weeks, and they were counting everybody who came to the last user conference as an MQL. Right? But the point of the matter is we— First, we had a bad definition of MQL. But second, because we didn't have it defined in advance, because we didn't agree to it, somebody did something stupid and counted the wrong thing, and we spent an hour talking about the metrics instead of the business. And that, that's a problem. Rat-holing. Next, semantics. Semantics is almost the opposite case, where you think you're having a really good conversation, and then you realize some of the words don't mean anything. Right? Like, Like for example, here's one of my favorite conversations with the VP of Sales. Hey, what's the forecast for new ARR this quarter? 4.3 million. Oh, okay. What's the most likely outcome? 4.5. What do we have here? Noob alert. Right, like any professional CRO will say 4.3 again, even if it's not mathematically correct. And then I say, oh, okay, so I got him to move once. Well, what are we really going to do? 4.7. I think if I keep asking this guy questions, I'll have him up to 5. With like 2 more questions, I can get him up to 5. So there's 2 things going on here. One is the kind of noob alert, but the actual problem is in what's worst case, 3.5. If I say what does most likely mean, what does best case mean, I get gobbledygook as answers. Well, my favorite definition of best case, "If the stars align." How often do the stars align? Is that like a frequent occurrence, or is this something, you know, happens once a month, once a year? Right? I don't understand that answer. Right? So what I encourage companies to do, and most CROs hate this, I think more velocity, more quantitative CROs are okay with it, but we're really talking about the bottom edge of a confidence interval. And when I give the forecast to the board, I want to give a 90% confidence interval, so I'm giving them the lower bound of the 90% confidence interval. Now people hate that, and if people get scared when you use those words, I have advice. Just say you get to miss it once every 2.5 years. That's what it means, right? That's exactly what a 90% confidence means. You get to miss without getting in trouble once every 2.5 years. And by the way, since you missed it last quarter and the quarter before that, You're not giving me a 90% number, right? So try and adopt this probabilistic view on forecasting, and more importantly, just try to define what these things mean, because we have these conversations and some of the most important words are undefined. So lack of semantics. Recalculation. So here's a joke. How do you— what's the difference between a venture capitalist and a private equity investor? There's a lot of them. We could have a lot of fun with this question. Here's my answer. We'll see if we're really metrics geeks. A VC asks you your CAC. A PE tells you your CAC. Very good. Glad we got a laugh on that. But it's true, right? Because the VCs tend to be trusting and say, what's your CAC? And tell me how you calculate it. And isn't that interesting? Whereas the PE people just tend to say, give me all your data. I'll tell you what your CAC is. Fundamentally, especially in the context of an acquisition or fundraising, I don't really trust your numbers. I'll do it myself. So if you find yourself in a situation— like I sit on a couple of boards where one of the board members will say, you know, this number's that, and I'll be like, where did you get that? I don't see that in the deck. He'll say, oh, I recalculated it all. I have one of my MBAs recalculate everything. I'm like, this is a symptom of a very serious problem, right? We don't trust the management team to present the correct numbers. We don't trust them to do the right formulas. We don't trust them to update the templates. If I just say, please add this row and calculate it that way, I can't even trust them for that. So I'm going to use one of my MBAs to go recalculate everything. So this is a problem—recalculation. Inexplicability, one of my personal favorites. The company's not performing. We're missing ending cash. We're behind on ARR. But when I look at all the other metrics, everything looks fine. Right? Top of funnel looks pretty good. The conversion rates look good. The cost per opportunity is okay. Stage 1 to Stage 2 conversion is okay. Stage 2 demo is okay. Demo to close is okay. Jesus, the ASPs are constant. Average sales cycle isn't increasing. Like, I don't know what's going on. And this does happen. And all I can say is you've got a problem somewhere. If the top 2 important numbers are not being achieved and you can't see why in all the other numbers, I can't diagnose what the problem is, but I can tell you you have a problem. And it does happen from time to time. Slide 2, okay, so we're about halfway through our list of metrics— signs of metrics problems. Here we go. Dissonance. The words don't match the music, right? You ever know like a song that kind of sounds sad, but the lyrics are happy? Or it sounds happy and the lyrics are sad? Like, you know, "Some Charmed Life"? Upbeat number. Very catchy chorus. Doot doot doot, doot doot doot doot. That song's about meth addiction, right? It's all about meth addiction. One of the lines is, "Doing crystal meth will lift you up until you break," right? And it has all these incredible words in it that if you're not paying attention, it's like, holy shit, we need to look at the lyrics of this song. And that's what some board decks feel like, right? I was flipping through a board deck with a friend of mine the other day, and it's like, oh, we're struggling a little bit on ARR, and we're not quite there on PMF. And, well, but, you know, employee satisfaction is really high, and customer sat's high, most of the customers are still here, and the bug backlog is down, and story points are up, burn rate was 7x, and I'm like, what? Do we crystal method, lift you up until you break? The burn rate is— the burn multiple is 7x? We're spending $7 for every dollar of net new ARR we're adding? Right, that's like a catastrophically bad burn rate. Burn multiple, right? And it'll just be mixed into the song, and you're kind of not supposed to hear it. And the answer, what's going on here, is I think it's actually a form of avoidance, that people don't want to actually admit there's a kind of a dead moose problem on the table, and they're talking about all the other things, and we're just letting this huge problem sit there in the middle and not address it. So if you have dissonance, I think it's probably caused by avoidance. The next is the numeracy, and this ultimately is a cultural issue. The symptom is very simple. You go to a board meeting or QBR or management meeting, and the only two people who ever answer numbers questions are the CFO and the head of RevOps. Anytime you ask it, and when you ask it, when they're out of the room, oh, we've got to wait for Phil. Phil's not here right now. When Phil comes back, I can tell you that. I can tell you average sales cycle length in mid-market. Right? So part of me wonders, well, let's give Phil your job then, because Phil could answer my questions and you can't, right? What value add are you doing here? And the other is, why aren't you numerate? Why don't you understand your business? And ultimately, I think this is a cultural issue. I have a friend who calls CEOs the Chief Reminding Officers. I don't love that. I think of them as Chief Norms Officers. You're setting the norms for this company, and it's either OK or not OK. Is it okay for on a Monday morning marketing not to know this quarter's to-go pipeline coverage? There's only one answer. It's either okay or not okay to know that. And if it's not okay, you need to set that as a norm. And in my experience, the better companies do. Déjà vu. So you go to the same board meeting and it never changes. Sorry. You guys are really switched on. You caught that. So I'll just— A, I put it in there mostly for a joke, but I will say I had a friend of mine quit his job and that was the reason. I said, "Ed, why are you quitting this job?" He goes, "'Cause I've been to the same meeting too many times." And I actually think it's a fantastically good reason to quit a job, right? If you're always going to the same meeting, it means you're not making progress on your problems. And again, there's a difference between not making progress on hard problems that take a long time to fix, which is fine, and literally having the same meeting every time. So we talked about déjà vu all over again. Next, disconnect, where metrics are not tied to strategy. And, you know, this is an example of how a company might present the second quarter metrics. New AR, 85%. Expansion, 101%. Churn, 18%. New logo, 25%. And I've got a lot of problems with how this is presented. Let's be clear. In that other talk I'll write for a month or two from now, We'll talk about everything that's wrong with how this is presented. But the biggest problem I have is imagine this company is in the middle of a mid-market push, that we're doing a lot of SMB business, the churn rates are too high, the lifetimes are too low, the ASPs are too low, it's throwing off our CAC ratio, it's throwing off our LTV to CAC, and we think, because we have some customers in mid-market, that if we just retargeted our energy more to mid-market, that we do better. And that's one of our top 5 strategic goals for the year. Imagine this company presents that chart. What do you think? You go, "I can see the strategy everywhere but in the metrics." Right? As you say, "I can see computers everywhere but in the productivity statistics." Right? I can see strategy everywhere but in the metrics. Where are they? And the answer is here. Right? This is what that company should actually present, where we say, You know that mid-market push that is the number one thing we're trying to accomplish? The first metric I'm going to show you is new ARR for mid-market. We're actually 111% of plan. Then I'm going to show you the ASP for new ARR in mid-market, which is 140% of plan. Sorry for the typo. Mid-market sales cycle, we were worried it was going to be long, but it's turning out to be short. It's only 45 days. Right? Mid-market pipeline coverage, are we generating enough opportunities in mid-market? Yes, we are. We've got 3.4x. Our win rate, did we win mid-market deals at the same rate? Because maybe we have new competitors. Yes, we do. Right, it's actually a better rate if it— well, I don't have it there. But it's actually, you know, very strong win rate, 35%. So to me, and yes, I put the overall numbers down because they matter. In the end, we are running that business on the bottom. But our strategy, one of our top 5 strategic goals is to improve that business on the top. So Disconnect. Next, piecemealing. This is super common, where numbers are evaluated in isolation, not holistically. You know, a VC will look across their portfolio and think of the best CAC, the best CAC payback period, the best LTV to CAC, the best rule of 40, and ask you to do, you know, beat them on all four. It's inherently unfair. It's a form of bludgeoning, right? Like you want me to be the most musical kid in my high school and the most athletic and the best at math and the champion of the chess club, right? That just doesn't happen, right? So we need to look at these things holistically. So piecemealing can be dangerous because, for example, given you have a high customer acquisition cost, what do we think about your lifetimes? So if you've got really low churn and a really long lifetime, it might be worth it. Hey, we're getting people who are going to be sticky for 10, 12 years. We can afford a CAC of 1.9. CAC ratio of 1.9, right? I need to look at them together. Given your high growth rate, right? Say you're growing at 120%. That's going to drive your CAC up because CAC is a function of growth rate in enterprise software because you're carrying unproductive/ramping resources, right? So we need to think of these things holistically. Think of them in little groups. And piecemealing in the end, when you just take them one by one, you're losing insight and you're probably in all likelihood just bludgeoning. It's a different budgeting technique. Next, misbenchmarking. I love benchmarks. There's lots of great benchmarks out there. I think it's really important to compare ourselves to others. I don't think we should just automatically try to be, you know, 75th percentile in every metric, you know, across the board. That's not wise. These things should be a function of strategy, but I think we should know them. I think we should know what those people do, and we should understand how they work together. And misbenchmarking happens for a couple of reasons. One is some benchmarks are just highly selection biased, right? Whoever answered the survey, OK? And you may have bootstrap companies in there. You may have PE-backed companies that want to sell in 3 years for 18 times EBITDA. And you may have venture capital moonshots that want to grow at 250% all the way up to $300 million ARR, right? And you're putting them together in a benchmark and saying the average CAC ratio is 1.4. I was like, "Hey, wait a minute. This is kind of crazy." And normally you can segment these by size or growth rate. That's good. But I actually think the best way to segment them is by aspiration. That if we could— and nobody offers these yet, but I want you to just consider this concept— that we should segment benchmarks by aspiration, which is if I want to sell for 18 times EBITDA in 3 years, boy, wouldn't it be nice if I could compare myself to other companies shooting for that goal? Right? Analogy: if I want to go to Harvard, I should compare my SAT scores not to the average, but to the group of people who apply to Harvard. Right? In this case, Harvard is an IPO. Right? If I want to go public and be an IPO-track company, I can't— for a concrete example— take 104% ARR out of a blended benchmark when that Harvard cohort has 125% ARR. Right? I need to benchmark against people who are trying to do the same things I'm doing. Correlation and causation, I'll just give you a simple example. I think we all know what this is. But for example, the VP of customer success says, "All of the customers who churned had multiple escalations. They're having escalations because we don't have enough agents. The agents are too busy. We need to reduce churn and hire agents." That might be correct. But you cannot by any means say that that simple correlation is causation. Right? More likely, in this particular example, maybe the product isn't working, and that's why the agents are getting all the calls, resulting in all the escalations. And if you want to fix this, right, like ice cream deaths, temperature, and drowning, right, the underlying driver is temperature that drives more swimming and more ice cream consumption. That's why they correlate, right? In this case, the underlying driver is the product's not working. So we should fix that. So we have to be very careful, because I watch companies do this. I gave you a super obvious example. But I watch companies do this all the time. They do some deep analysis of churn, so we're going to focus on these 3, and they're acting as if it's causal, and there's no reason whatsoever to believe that. It's correlated, yes. Is it causal? We need to look more. Torture is the last one, so we made it through the list, where people, you know, basically this is particularly bad when a board member does it, but they arrive with a hypothesis about what the company, what's wrong, and then they do what's called confirmation bias, right? They want to prove themselves right, So they will kind of selectively filter and torture the data in order to do so. And we all know the old expression, but just in case you don't, right, if you torture the data long enough, it will confess to anything, right? So we could prove almost anything with the data if we're determined enough, right? That's not the point. So I'll just give you one crazy example. This company walks in and says, "Hey, we've got a 5% churn rate." You go, "Wow, that's amazing. So you have a 20-year lifetime." Which means even if your CAC ratio is like 2, you're going to have an LTV to CAC of like 10+. This is amazing. God, fantastic company. And I say, can I see the data? And they put this up. And I go, hey, wait a minute. Wait a minute. Your churn rate is 25%. You lost 500 out of 2 million in ARR. So your net churn rate is 25%. If you exclude expansion, you can check the math on me, the gross churn is 29%. Your logo churn is 5%. You've only lost one logo? But the reason you only lost one logo is what? You renewed 4 customers for a dollar. Right? You renewed 4 customers for a dollar to kind of juice your logo churn rate, to juice your lifetimes, to juice your LTV to CAC. Right? And the question is, so those 4 $1 renewals are hardly the problem. And, you know, do people actually do this, you might wonder. And the answer is, yeah, people do. This is an old Chevron ad. But people do. I competed against a company that did $1 renewals. And they would juice their churn rates so they could offset their SaaS metrics. So that's what I would call an extreme case of kind of twisting SaaS metrics. That's why you need to look at both logo churn and dollar churn. So let's now look at a— what's going on here. Let's talk about the SaaS metrics maturity model, but there's kind of 5 underlying problems here. One, there's no shared metrics foundation. No definitions, what they mean, how they're calculated. Two, there's no trust in the data calculations and formulas, that the story being told matches the data, and that we aren't using metrics to either bludgeon or mislead. There's no trust. Second root cause. Third, metrics are not integral to strategy. That— and I think this happens because people view metrics as a financing thing. Oh, we need to raise a round. "Somebody go calculate all the SaaS metrics, because they're going to ask about them," right? And you haven't burned them into your day-to-day operations, right? Because they're not linked to your strategy. They may not even be linked to day-to-day ops. It's just something you do for financing. All the VCs are going to ask. The culture is not metrics-driven. We talked about this. Is, you know, is numeracy a prerequisite to have conversations about the business? Is it tolerated if you're not? And then metrics are not being used— this is more of an opportunity cost we haven't talked about— but metrics are not being used to define the trajectory and long-term goals of the business. Because when you get the other 4 taken care of, this is what you do next. You say, "Hey, wait a minute. What are we trying to accomplish over the next 3 to 5 years, and how's that going to impact our business? Because if we open Europe, that's going to hit our CAC ratio in the short term. Sorry, we're going to be opening a bunch of offices, hiring a bunch of salespeople, they won't be selling anything, that's going to be bad for CAC, in next year and the year after. It'll be good for growth in the years after that, right? Ditto for an R&D investment. Hey, we do something inefficient in the software. If we, you know, did a big R&D project, we could make it more efficient, reduce our COGS. It will hit R&D in years 1 and 2 and show up in COGS in year 3 and 4. And to have these conversations, you need kind of a long-term driver-based model, and you need to think about your strategic priorities. So towards this end, I built this SAS Metrics Maturity Model where we have those 5 layers. Level— and we're going to talk about each in a minute in more detail. Level 1 is foundation, which is definition, calcs, and semantics. Level 2 is trust. And I'll say it a couple of times here, but templates build trust. Just if you can only get one notion from this entire talk, templates build trust. So we're going to talk about templates, cadence, how you use them, right? And then continuous improvement, how we always make them better. Level 3 is linking metrics to strategy. Level 4 is culture. And Level 5 is trajectory. And I could just say that improving SaaS metrics maturity is a journey. There is, even Layer 1, Layer 1 has taken me with certain companies I work with, quarters. And I walk in every time, I have this kind of entrepreneurial blind spot, going, "This is gonna be easy. Me, head of FP&A, CFO, we'll get this nailed in 30 days, 60 days." Every time I do that, and most of the time, 3/4 later, we're still working on it. And it's good, right? I mean, if you ask founders, if you had any idea how hard it would be, would you start your company again? And most say no, right? So I actually view this as the endearing and valuable trait of someone who's entrepreneurial, is we radically misunder— misunderestimate. That's a George Bushism. We radically misestimate the amount of work it is to do something. And it's actually— In some ways, it's very helpful, because we get fresh on it every time. So, but it is hard. You can't plan on having this done in a few weeks. The change management is hard. It may require staff upgrades, particularly key analytics roles like RevOps, FP&A. Sometimes those people are not that strategic. They're not that fast, right? They're good at making a budget, and they're good at getting comp plans done, but they're not really good at thinking about the business. Will require meeting changes to cadence and agenda. And requires, you know, tru— I don't want to get all touchy-feely here, but trust and the willingness to be vulnerable. If you're going to put these metrics out there and somebody clubs you to death with them, that's not going to feel real good, right? So we have to be careful that we make sure the board doesn't club us, and that we don't club the exec staff by looking at them. So, let's now look at each of these 5 layers. 1, the foundation. So we've talked about what this means before. 2, well, 1, 2 new things on this slide. 1, sometimes this requires the rework of underlying systems. Most of the companies I've worked with that have this problem in Europe for some reason, but they don't produce a standard SaaS P&L. They produce a nature-based P&L, not a function-based P&L, and you can't do SaaS metrics off a nature-based P&L, right? They would literally go GL account by GL account, map it to sales and marketing, and you know, once a year they'd calculate their CAC ratio because their P&L was not set up to do it. And boom, that's a 9-month delay. You want to know how it takes 9 months to fix SaaS metrics? Step 1, change your accounting system and your chart of accounts, right? But you have to do it, and there's no time like the present to start. The other thing I'd say on this, and this is again from hard-learned experience, offboard this to metrics committees. There are people like me who care, and rather than trying to do this with the whole e-staff, or God help you, the whole board, say, I think we should get a committee of interested people who want to meet, you know, once or twice a month, to work with FP&A, CFO, ops, to make sure we have the templates you guys want to see in the meeting. And we put people on the committee, and we let them work. This is the answer to how to improve your staff metrics. Delegate the problem to people who care, let them work, give them time, have them check back in. Second, build trust. Templates build trust. Templates build trust. Why? Because there's no cherry-picking, and the most common form of misleading is cherry-picking. We just leave stuff out. We don't talk about expansion ARR being so far off plan last quarter. Somehow it just didn't make the slide. And when you have good templates, you can't leave stuff out. So we decide everything that we're going to show, we show it every time, it never changes. And we provide trailing 5 or trailing 9 quarters of history. And we provide context like the plan or the most recent forecast or last year. Or last quarter. So it's rich in context, and there's nowhere to hide. And I honestly believe that templates build trust. It's the only way I know to build trust is to keep working on the templates and then keep making them better. The other thing is fix data at the source. As long as people are adjusting data out of systems in spreadsheets and putting it back in, no. You must become kind of a fascist about, we will fix the underlying systems. We will fix the transforms. We will get this stuff done in a systematized way. 3, link to strategy. So I love strategy. I love this guy, Richard Rumelt. He's written 2 good books. One's called "Good Strategy, Bad Strategy." The other one's called "The Crux." Pick whatever one you want, but the point of all strategy is to identify your biggest challenges and make plans to overcome them. It's that simple. And the thing that operators get wrong is they don't underst— they don't have empathy for the board. So for example, let's say we're back to that company that's trying to do an upmarket push. And we want ASP to be one of our key results. We're going to do this upmarket push because we think the ASP is going to come up. There's two ways to approach this. One is just make ASP an OKR, and we'll wait till the end of the year to see if it changed. And you're effectively telling the board, trust me. I got this. Trust me. It's going to change. And maybe it'll change on a quarterly basis, but that's not fast enough. And by the way, the last 3 guys, sorry, said trust me. So as the board member, I'm not really in a very trusting mood right now, because the last 3 COOs have said trust me. It's not good enough. If you want me to trust you, give me some leading indicators, right? And I'm going to use a sailing metaphor here. I'm not a big sailor, but I did take sailing lessons. And when a boat tacks, the bow goes through the wind at some point, right? So I want to know in a metric sense, when is the bow going through the wind? There's actually signs before that. The sail starts luffing. There's a thing called telltales on the sails that will tell you you're starting to execute the maneuver. We want telltales, we want luffing, we want bow going through the wind. To make that less nautical and more literal, what if we looked at the ASP of deals in the pipeline? If that was bigger in mid-market, wouldn't that be a leading indicator of closing deals, right, with a higher ASP from mid-market? Yes, it would, right? That's a leading indicator. That's a telltale. It's not the KR. The KR is to get ASP up for real, but the leading indicator is to look at the ASP of the mid-market segment in the pipeline, right? So I don't have to just say, trust me. 4, metrics culture. This we've talked about. It's really a choice, and the goal is to not have conversation about metrics, but conversations about the business using metrics. My favorite quote on this, it's an old one, but I love this quote. If we have data, let's look at the data. If all we have is opinions, let's go with mine. And if you say that enough in your staff meeting, maybe people will get the hint, right, and start showing up with data. And then finally, agree on strategic trajectory. We haven't talked as much about this as I would have liked, but I do have a blog post on this called "Glide Slope Planning." That's the metaphor I use on how we want to take this company to its exit, and it's all about using models to do basically timing, sequencing, convergence. So we're going to do some Q&A. Let me just wrap up here with our last slide before we go to Q&A. But, you know, SAS Metrics can be used as tools of enlightenment, oppression, or obfuscation. I've seen all three. We've talked about 15 symptoms of a SAS Metrics problem. We've introduced a 5-layer SAS Metrics maturity model, which you can use to both kind of locate yourself and make your roadmap. And we discussed things you could do at each level to build your SAS Metrics maturity. So let's go off perfectly on time, 5 seconds ahead. Let's go off to Q&A. And I don't know how that works. Thanks. I guess there's a mic there. So if somebody has a question, ask it. If you're in the front row, I'll just— I can take it and repeat it. Somebody's got to break the ice. Here we go. Brave metrics person. Thanks, Dave. I'll break the ice with a question about how do you treat exceptions? So think about an unusually large deal or something like that. Yeah, I love that question. So look, exceptions are a weird thing. Like, I'll tell you my perspective as a CEO. This is why I like the fact that I've been a CMO, CEO, and a board member. 'Cause as a CEO, I'll hear the board go, "Well, except for the million-dollar deal that closed last quarter, you're behind plan." And I'm like, "Screw you. We actually, in point of fact, closed the million-dollar deal. We're not behind plan." Right? That somehow the million-dollar deal is getting used against me. Right? So I think that's— basically, I think both answers are kind of wrong. Right? 'Cause if you exclude them and say, "I only want to look at run-rate business," and on a run-rate basis, you're behind plan, that feels very wrong as the person who's ahead of plan. Right? Being told they're behind. On the flip side, you know, if I don't have a bunch of other— do I have 3 $1 million deals lined up for the next 3 quarters in the pipeline? Then maybe it's not an exception. Maybe it's a trend. And maybe I need to get used to the fact that I'm dealing with a bimodal business, and I ran one of these. MarkLogic, we did massive deals, as well as a bunch of regular deals, but we could do massive deals. So we have to figure out a way to kind of incorporate them into our metrics and our pipeline. And the typical way is you just kind of forecast an upside, or you break apart deals. You pick part of the deal and say, I think I can get the baseline. It's not easy, but I think I don't like board members who automatically exclude them because they're not run rate, because they happened, right? It actually happened. I got the million dollars. So I don't want to pretend it didn't happen. Hi, Dave. Big thank you. I'm your big fan. We actually have relied a lot on your work in my previous roles in rev ops and finance. So thank you. Question is, to what extent should someone who works in finance or rev ops rely on their own judgment when it comes to, let's say, setting a forecast or a target? Yeah. Whereas, because, you know, they're generally not in the trenches, they're not selling, they're not speaking to customers as much. Yeah. Versus relying on someone from who actually carries the bag. Yeah. But might have an adverse incentive to send back the target. Yeah. Yeah, it's a great question. Good questions today. So I believe in triangulation. The generic answer to your question is triangulation, which is that the CRO is calling $2 million this quarter. It's their job. It's their call. They will live and die by that call. I'm not going to tell you if they're right or if they're wrong. I will tell you that in other quarters that look like this one, using historical week 7 conversion rates, if the pipeline is graded in the same way, I think we're going to come in at 2, 3. And I'm doing that using a stage-weighted— to make it really depersonalized, I'm using a stage-weighted week a week-driven stage-weighted expected value of the pipeline, and that's saying 2.5. And that's just a fact. So do with it what you will, but you're the CEO. You asked me, right? And they should ask you, or you should burn this into your templates, right? I mean, I would always have CRO forecast, then I'd have some of the reps. I would have some of the managers, some of the managers' forecast. I'd have stage-weighted expected value forecast quarterly. Forecast category weighted expected value. I'd have them all right there. And if you got me more, I'd put more. You have more triangulation points? I want them. So that way I think you make it unemotional. I mean, unless the CRO is really a psychopath. You make it unemotional and you're saying, I'm doing math. My job is to do math and here's some math that I did. And you can ask them about their forecast. So it doesn't work all the time, but that's the way to approach it. Triangulation. Thank you. One more. Yeah, thanks a lot. It was a great talk. My question is, how normal or regular is to have specific internal metrics like you said and others just for external world, you know, for marketing or fundraising, to keep both of them? How normal is that? Or, you know— So the question is, how normal is it to have multiple sets of metrics? I'll give you my favorite example of what I consider normal. And I am a marketing guy by DNA, OK? So finance people, cover your ears. But if you ask me to get a customer satisfaction survey rating that we're going to use in marketing, I'm going to suggest we set up a bunch of iPads in the bar at the user conference. And that's the exact moment we ask everybody how happy they are. And we sit down next to them and say, hey, Joe, could you fill in the customer satisfaction survey? And I will say in the footnote, survey conducted at the 22, Perform conference or whatever. I will footnote it. I'm not going to lie to anybody, but I'm a marketing guy. I want a good number. And so what that tells you is when our customers are drunk and happy at the user conference, they like us with an NPS of 72. Now when they're sober and at home, it turns out that it's 35. But I'm not going to necessarily— I want to see the 35 in the Ops Review. So I don't know if that's capturing the spirit of your question or not, but I'm revealing my marketing colors. Marketing's job is to tell the truth in the most positive possible way. So just think about that. We're not supposed to lie, but we're supposed to tell the truth in the most positive possible way, and let them do that. It's their job. Right? They don't need to say the unvarnished truth. So what's your truth? No, kidding. Thanks, it was a great question. Any other— yeah? Hey, my question is on talent. You talked about FP&A and RevOps people being hard to find, strategic ones. What would be like the common traits you would look at in an ideal analyst profile of someone? Is it investment banking? Is it consulting? Is it— Yeah, that's a great question. I mean, I have my own super bias, so I like FP&A-oriented CFOs. I like ex-banking-oriented CFOs. I like people who are fast with numbers. Right, who can whip through models, who know numbers in their head, who can give approximate answers, which a lot of CFOs are afraid to do, right? Just say, "I'm not sure what it is, but I think it's here," right? I like those people. So I'd say it's the more FP&A style. A lot of people for different reasons prefer the more controller style. I do think they're different animals, for whatever it's worth. Like, because I sold to both at my last company. FP&A was customer and controller was a customer. And believe me, you could tell them apart at the user conference, right? I mean, they wouldn't even like talk to each each other. They'd hang out separately. So that's my personal bias. It's probably got some stage on it. But I think those— look, I like the finance person as the business partner who could act— I thought that was a cliché for decades. I can't even believe I'm saying it. Be a business partner. You know, cliché, horrible. It's actually not. Certain CFOs can actually be a business partner to the CEO. 'Cause they're smart and they're fast and they understand the business. They can keep up. And that's who you want. You don't want CF No, right, as they call him. Right? Dr. No, who just says no to everything. I worked with one CFO once. This is a public company that we had hit like 30 quarters in a row of profitable growth. And we missed one quarter by like 2 pennies on the EPS. And of course the stock got crushed. And he goes, I am not here to explain your losses. It's like, Jesus, we've built this amazing business. So yeah, I personally don't love that style. I'd rather have the forward-looking FP&A type. Yep. Hello. Thank you for the outstanding lecture. My question is, sometimes we have some SaaS metric that is most commonly calculated in a way, but if you look closely, sometimes better to calculate another way, but the markets use it to— yeah, in a most common way. For example, if you look at Rule of 40, the majority of companies report that in revenue growth rate plus EBITDA margin. And I just saw in another lecture using free cash flow margin, which I believe it makes more sense. Yeah, but I could totally imagine, uh, venture capital or another investor to looking at it and saying you are not calculating correctly. How do you think we should balance that? So to me, I mean, look, there— for a lot of these metrics, there's no official right answer, right? And I say this with my friend Ray Reich wincing at me because he runs a thing called the SaaS Metrics Standards Board. So talk to Ray, you're 10 feet away from him. But for the most part, that work is not yet highly popularized, let's just say. So a lot of VCs won't hear about it, and some VCs might say you're doing it wrong. Frankly, if you ask me, I'd say you're doing it right. My definition of Rule of 40 is actually free cash flow margin. So there you go. I think the answer to me is always very simple. Just footnote stuff. Just tell people what you're doing. And if they say, "Why are you doing it free cash flow?" Be ready to answer why you think it's meaningful. Either why you think it's standard, because I don't know, because Meritech does it that way on public comps, which I think they do. So you could argue it's standard. Or you could argue it's more meaningful for your business for whatever reason. But the main thing to me is footnote it, because I don't want anyone thinking you're trying to pull a fast one. To me, it's all about trust. That layer 2 trust is so important. And to me, templates build trust, footnotes build trust. So thank you very much. Yeah, thank you. I think we have time for one more. Thank you for your talk. It was very enlightening. I have a question about companies that have multiple products and assessing the performance of those products. Yeah. How do you look at that from a metrics perspective, especially when you think about shared expenses that are hard to allocate to a specific product? Yeah, yeah. Okay, glad we got an easy one with 8 seconds left. I'm going to give a short answer. Contribution margin, right? That's the short answer, but I think you need to figure out— and I'm not sure it's the right answer, by the way, because contribution margin is technically defined as variable cost, right? Because it's supposed to be a, do our variable profits cover our fixed costs, right? Price volume cost analysis. Sorry for doing the MBA finance class, but that's what contribution margin is. I would personally start with that sort of concept. I would define a contribution margin for us. I would decide what to put it in. I am not a big fan of allocating fixed costs in an arbitrary way across products, right? Maybe some people are. Some of the more finance people I know are, so you can ask them. But my answer is just exclude it from the analysis. If we're going to have a CMO either way, if we're going to have a PR firm either way, if we're going to have a website either way, and I'm trying to figure out the contribution margin of product 1 versus product 2, just ignore it. Don't allocate it. But I guess it doesn't change things that much if you do the allocation. But I think— and we're going to— so I'll give a plug, Ray. You're probably happy. Ray and I do a podcast called SaaS Talk with the Metrics Brothers. On our list is contribution margin. So there's an upcoming episode, hopefully. Thank you. All right, everybody, thank you very much. Have a good day.