Fighting Churn with Data with Zuora | SaaStr Software Community

My name is Carl Gold, chief data scientist at Zora. Zora is a subscription management platform. My PhD is in biology, but, you know, now I'm a subscription economy guru. All companies can benefit from fighting Churn with Data. When you look at a SaaS product, they're typically tracking dozens or even hundreds of events. If you can find a few key relationships, that's what really makes the difference. Churn is hard to fight because it's owned by different parts of your organization. So the Churn fighters in your business, they need insights from the data. It's a multi team effort. The product creators, the customer support people, they're all thinking about this. In general, more observations can never hurt you. You're not going to find easy fixes for Churn. You really have to deliver more. Thank you so much, the Saster team, for making that. It actually makes my nerdy data scientist life look kind of cool, which is quite unbelievable if you think about it. But hey, it's true. My name is Carl Gold. Welcome to today's session at Saster. Scale Fighting Churn with Data is the name of the session. I'm the chief data scientist at a company called Zora and I'll be telling you more about all this in seconds. And. And at the bottom is my Twitter handle, Carl24K. Just to dive right in. I'm going to dive right in with the basics. I'm going to level set for everyone. I don't know if I need to do this at Saster, but I start out all my presentations with a little intro. What is Churn? If you don't know, Churn is customers quitting. And it comes from the term the churn rate, which means the percentage of customers which is quitting or dropping out on you. You typically measure the Churn rate in a month or a year time period and we'll see more of that in just a second. So churn is also now a noun and a verb. It's not just a rate. So you can say the customer churned or make a report of last quarter's churn or even a churn report. It's also, you can also say, oh, I'm churning from Hulu or Netflix. So there's a lot more uses for Churn now for those of us who are already in the SaaS business. But the fundamental thing that you all must know is that Churn cancels your growth because Churn is your new customers dropping out the door. Did I really have to tell you all this at sasterscale? I think this is one conference where I don't have to explain what Churn is and what we're here for. Although to finish off the level setting, I want to say that please feel free to ask questions. You know, I don't want to set a high bar. Maybe if any of this is new for you or if you have any in depth questions, you know, please ask them in the Q and A. I'm not going to try to answer questions during the presentation, but we have a lot of time for questions at the end. So that's the basic level set on Churn. Let me tell you a little bit more about my background and why I'm here talking to you about Churn. What is Fighting Churn with Data? That's actually a book that I wrote and it's just been published. It's just come out in hard copies. It's actually a book of data science and analytics. It's not focused on the business case, but it's all about Churn reducing strategies and it includes step by step instructions and open source code. So it's really a book for your tech team. But still it's based on my learnings over the years as zora's chief data scientist where I worked on Churn with many different companies. So this book is really all about the lessons I learned and to help you avoid the mistakes that I made along the way fighting Churn data. So just really briefly, if you don't know what ZORA is, ZORA is a subscription management platform. We say our customers from Avalara to Zoom. I'm not really here to talk to you about zora. I just want to by way of explanation, you know, how did I come to work on Churn for so many different companies? It was because of my role as chief data scientist at zora. It brought me into contact with all these companies. We have a product at ZORA that was allowing them to analyze their churn. So that was made my role unique in comparison to a data scientist at a typical company where they only work on Churn for their own company. As data scientist at zora, I worked on Churn for many, many companies and that was where I got the experiences that let me write the book. So I'm going to go back to kind of level setting and just talk about Churn rates, which is something that people often want to know about. So this is showing what typically typical monthly Churn rates look like and you can see there's a big range. So this is based on ZORA data. There's a big range of churn rates. And it depends a lot on the business model and company maturity. For example, we generally see that consumer products have significantly higher churn rates than business products or B2B products. Overall, we see a median of around 2% for Zora customers. And you do see different numbers coming out of different sources. For example, another, you know, there are other subscription management platforms more geared towards small businesses and those typically have somewhat higher churn rates just because of the nature of being a small business. So for details on this data, we published it in a report from ZORA called the ZORA Subscription Economy Index. And you can find it. Well, it's pretty easy to google for subscription economy index. So this is the typical monthly churn rates and this is how you're going to look at churn if you're a consumer company because you sell your product in typically monthly or short term contracts. So you'll look at monthly churn. If you're a business to business company or you're selling annual contracts, you want to look at the annual churn rate. And this is showing how those monthly churns translate into the annual loss to your growth. And so you can see the median churn of 2% per month translates into 22% per year. 5.7% per month translates into 50% per year. So that's if you're in that group, that's half your growth walking out the door each month. So for ZORA customers, the median annual churn is 22%, as you can see here. And as I mentioned, this is typically how you would look at your churn if you're a B2B company. So if you're lucky and you're in the top half of this group, you know, above the median, that's around 20% of your growth disappearing every year. So churn is a huge problem, you know, for all companies. I'll mention that public companies are kind of like the best in class for churn is typically around 10% or less per year. And that's what you see kind of in the top customers analyzed in this cohort. So like I said before, where you are on this chart, it really depends a lot on what type of business you're doing. But I have seen that all companies that every level of churn that they start out with, if they do make an effort to use their data to fight churn, there's really a big benefit that comes from it. Okay, so another thing people have been asking me a lot about is what happened this year due to COVID 19. Here is some data on that. This is showing the annual churn rates. Again this is from the ZORA subscription economy index based on customers on the ZORA platform. We saw churn peaked in Q1 at 27% average annual rate. And that is the highest churn rate. These are annualized churn rates shown here on a quarterly basis. And that is actually the highest churn rate we've seen in quite a while. 27%. I don't think it's been that high. Yeah, since like 2015 or something, but it's not that high. That was my first reaction. I mean sure it's the highest rate we've seen in a while, but I mean really, is that as bad as it gets? And churn fell back down again in Q2 and Q3, coming back down to the typical levels from 2019. So the impact on churn overall from COVID 19 has not been that strong. Now that's not to say there are not some businesses that have been very disrupted by what's going on this year, particularly companies that where their product involves face to face meetings or travel live events. Those companies have obviously had much more difficult times. But there's also been as you know, a rush to adopt digital services and products as so many people are working from home and so many companies are going digital. So there's contrasting trends in Churn and overall the net effect in the SaaS market has been pretty muted. So that's how Covid has impacted Churn. Now let's stop talking about the churn numbers and talk about the actual churn reducing strategies. This is what fighting churn with data is really about. It's data driven churn fighting strategies. So this is the top four churn fighting strategies that I see companies use and that I provide analysis for. The first one is making a great product. Did you realize that would be a churn fighting strategy? Well, having a great product is actually your best churn fighting strategy and you can use the data that you collect to make a great product without relying on biased surveys. Surveys of your customers are great but they mostly the customers who love you or hate you are the ones who fill it out. But using data in the way I'm going to explain will let you see what are your best product features in a way that uses all of your customer data and gets information from all of your customers. Okay, next is marketing or particularly engagement marketing where you re engage with your existing customers. And here the key is to use the data to make the communication to your customers actually valuable. Actually telling them about features that they don't know about or telling them tips and tricks for features that they really value. That's the kind of engagement that your customers want and not you spamming them. Okay, next is customer success. I love being at Sastra Scale because I don't have to explain what customer success is. There's been a lot of great presentations about customer success these past couple of days. And so using your data is the best way to see what customers need training and onboarding and when do they need it. So again, you want to take a targeted approach with your training and onboarding by using data to get the customers that really need it. Now, the fourth way to fight Churn is actually in your pricing. And here what I tell people you're trying to do is give customers value with a multi tiered pricing plan, but don't give discounts. A lot of people mistakenly think that giving out discounts is how you reduce Churn. And there are examples of that, mostly from the cable companies where you call up and say you want to cancel, they throw you a discount. Well, SaaS leaders, if you give out discounts to reduce Churn, you're undermining your pricing. So what you want to do is use your data and design a multi tiered pricing strategy where, where people can get value at different levels of involvement with your product. Now, there's actually a fifth way to reduce Churn, which I talk about in the book, and that's to find lookalikes of your best customers. So figure out where your best customers are coming from and really double down on those channels. I'm not going to go into that today and I consider that kind of a lesser way of reducing Churn because there you're not saving your current customers, you're just looking for new, better customers. I'm not saying you shouldn't do that, but again, it's finding better lookalike channels is kind of an extra method we're not going to go into today, but it is in the book that I wrote. So let's see. So these are the main ways to reduce Churn and how does data contribute to them is the next subject that we're going to talk about. Well, first actually, let me back up and say why Churn is hard to fight and why do you need to use all this data? Well, the first thing about Churn is that even with the best data, it's going to be hard to predict. Churn is relatively rare for most companies. Also, it's based on the subjective opinions and Thoughts of your customers, which you don't have, you can't read their minds. And even about the objective facts relating to your customers, you're going to have limited information. For example, you'll never know what their ability to pay is for. If you're a B2B product, you don't know what your customer's profit and loss is unless you're an accounting product. But that's a special case. And lastly, Churn is always really random, which you can see in your own behavior. Probably the last time you were churning from a product like Netflix or Hulu, you probably weren't watching it for a while, you got sick of it. But it probably took you a while to make up your mind or maybe to choose a replacement product. So even when Churn is pretty obvious, it tends to be hard to predict. So Churn is hard to predict. But even worse, Churn is hard to prevent. These are customers, they already use your product, they know your product and you actually have to deliver more value. There's no silver bullets, you know, for churn reduction, if you know the expression, this is a problem for lead bullets, not silver bullets. Which means you have to make an ongoing investment, get down in the trenches and fight with it. It's not just going to be like a once and done solution. Now also, the other thing that makes Churn hard is that it's a multi team effort. And here if you think about those churn reduction strategies that I was telling you in the previous slides, I said make a great product remarket to your customers, customer success and support, pricing and packaging. I just named every department in your company. Hello. I mean if you're big enough to have departments, but that's like the entire company. And all those people have different methods and different tools and different ways that they think about Churn. And so there's going to be communication and alignment issues when you try to tackle Churn. It might be, you know, finger pointing. I thought they were going to do it. Or different definitions of Churn even, or, you know, different ideas about what are the most, the best strategies to pursue. All right, so now that we've talked about how, why it's hard to fight, let's talk about how you fight it with data. A lot of people ask me if you should use AI for Churn. And there are some AI use cases. I mean, they ask by AI, I mean artificial intelligence if you don't know the acronym. I'm sure everyone does now. And since I'm a data scientist, people always expect. I mainly work with AI. Now I do work with AI, but not so much for churn. AI is a little helpful for Churn, but it's not really the focus. I mentioned that CHURN can be hard to predict and it's hard to use the data. So you would really only be doing this if you were an organization which is already advanced in analytics. But the problem, it's not just that it's hard to predict CHURN and you need to be very good with your analytics, it's that the interventions we talked about all need to be targeted and designed to particular segments. There's no one size fits all CHURN intervention that you're just going to unleash on all your customers. So whatever intervention you're going to make, you're going to design it using the behavior and demographics that are particular to that customer segment. And yeah, so AI, it helps with some use cases. If you want to figure out your most at risk customers with very high degree of accuracy, that's when you're going to use AI. But I'm mostly going to talk about a different set of methods, more traditional analytic based methods. And really what I'm going to focus on telling you about is your customer metrics. How do you find term with data? It's all about the customer metrics. And by customer metric I mean a measurement of your customers that you're going to make individually for all the customers. So I don't mean an aggregate measurement, I mean at the individual customer level you're going to make metrics and, and I want you to scientifically design them. And that's what I teach. And if you do this, those metrics will be easy to understand and usable by all those different departments in your company. And the metrics actually predict CHURN and retention. And without using AI, if you do well designed customer metrics, every single metric in your arsenal will predict churn and retention. And I'll show you examples of this very, very soon. Two slides away from case study data showing how this works. So the customer metrics that you make are going to enable you to do targeted segmentation when you take the interventions like marketing or onboarding training. And of course, if you make intuitive metrics like this, it'll be usable by all your teams and it'll become a focus of communication between the teams, so will be the metrics. So that's actually the background. I think I'm about halfway through the prepared material now. I'm now going to go into case studies. I'm going to introduce some companies that you may know about first up is broadly, broadly, broadly is a SaaS product. And it ensures that businesses look great online and are found and chosen by potential customers. So what that means, I guess in layman's terms is that broadly helped businesses manage their online reviews. And so when broadly collects data, the data that they collect includes events which are things like a customer promoter, which means that a customer made a review and it was positive. Or another event might be a customer detractor, that's an event that they track in broadly system among many, many others. And a customer detractor is a customer who actually gave a negative review. And from these events, which are kind of the underlying data, you make the customer metrics. And here I'm mentioning some basic metrics which would be for broadly, how many promoters per month did a particular customer get? And you would update that metric continuously over time as you watch your customers. And another metric might be how many detractors did the customers get per month? Now I said that you can use even simple metrics to predict churn and retention. And now let's look at an example. This is showing how you can use a simple customer metric to predict churn. And it's using a technique that I call a cohort analysis. Now you might heard of cohort analysis before and it was probably based on using cohorts of customers that signed up in a particular month or quarter, for example, and that would be a time based cohort analysis. I'm going to look at time based cohort analysis in a little bit. But this is actually a metric based cohort analysis. So all the customers have a metric which is their customer promoters per month metric that I just mentioned. And we group the customers based on that metric. So the bottom cohort is the first decile, or the bottom 10% are those customers that have the lowest promoters per month. Then the next group will be those that have somewhat higher promoters per month. And that's what's shown in this slide. On the horizontal dimension you can see the average promoters per month in each of the cohorts. So the bottom cohort has just about one or zero promoters per month and the top cohort has on average more than 80 promoters per month. Now on the vertical dimension of this graph, I'm showing you the relative churn rate. So this is not the actual churn rates for broadly. As you can see, there's no numbers in the scale. And that's for a protection of the strategic interests of my case study customers. They have all generously allowed me to show you this kind of data from their real live businesses. But we're not going to show you their churn rates. But the bottom of this axis is the zero. So you can tell on a relative scale how churn decreases. And what you can see here is that customers with more than around 15 promoters per month are churning at about one third the rate of those customers that have around 0 or 1. And that after around 15 promoters per month, the churn rate isn't really affected. So it's very easy from a chart like this to go to answer the question, what's a healthy customer? In this case, a healthy customer would be one with more than around 15 promoters per month. And any customer with less than that, they would be a reasonable target for customer success training or something that will help them get more promoters. And so this is the first example we're going to look at of one of these curves where you see how churn decreases with increasing use of the product. And to me, these are the bread and butter of churn analysis. You're going to see a lot of these charts and you'll see they all have this very similar shape where churn decreases from a very high level for the lowest cohort, the cohort that uses the product the least, and then it tends to flatten out at higher levels. Let's look at another example now, which is actually for the customer detractor metric from broadly. So this is how many bad reviews the customer gets in the last month, or it's an average. So in an average month, and again, you can see that the customers with the least bad reviews churn the least. They have the highest churn rate, and the more bad reviews they get, the churn rate goes down until it kind of levels out when you're above around one customer detractor per month. Now, that all sounds well and good, but isn't something wrong here? Aren't detractors and bad reviews bad? Why does that make people not churn? From broadly, you'd think that a product that's giving feeding you up a lot of bad reviews would make you churn. Well, to understand this, we actually have to go to a more advanced kind of metric, and that's what we'll look at next. We're going to look at similar analysis, but now we're going to look at churn versus the percentage of bad reviews, or what I call here the detractor rate. And this is an example of how you can use a more advanced metric to disentangle the effects in a disengaging behavior. So what this shows is that the bottom cohort in percentage of bad reviews shown here on the left has practically a zero churn rate. There's hardly any churn among those who get zero bad reviews. But if you go up to the top cohort that has more than 25% reviews, there's a much, much higher churn rate. So now what you're seeing is that too many detractors for this product really is a churn risk, but you have to look at it on a percentage basis or a relative basis. Now why is that? It's because this last chart, where we only look at the count of bad reviews is missing something. It's actually missing how big the business is that's using Broadly and how many reviews they get. So a big business using broadly is going to get a lot of reviews. They're going to get a lot of good reviews and a lot of bad reviews. And the fact that they get a lot of, well, they get the most number of bad reviews, maybe two or three per month, that's not a reason to churn because they're doing good, they're getting a lot of good reviews. It's only a reason to churn if they get a high percentage of bad reviews. And this is a very important technique for disambiguating these kinds of effects, which applies to all products with users that act at different levels. It's for B2B products, your big customers are your enterprise customers who do a lot of everything. And for consumer products, you tend to have a similar effect with power users. Certain power users are going to do a lot of everything and they tend to churn less. So when you look at a disengaging behavior, it's important to look at it on a relative basis. And we're going to see some more examples of this in a few minutes. But first, let's kind of wrap up the story by looking at Broadly's top churn fighting moves. Great and priceless advice here. Number one is that the customer success managers use the metrics to talk to the customers. So when they get on the phone with a customer, they pull up the screen. I think they have this imported into their salesforce. They have all the most important customer metrics so they can have a conversation. Oh, I see you got, you know, two detractor reviews last month and 10 promoter reviews or, you know, or what have you. So it really makes it easy to get into the conversation with the customer when you have these kind of metrics. Next, very important is to focus on the Mid range and churn risk. Ignore those most at risk since they're probably not savable. I'm going to back up briefly here to illustrate. If you consider this top cohort with more than 25% bad reviews, you know, it might not be worth helping them. I mean, you know, it could be really a poorly run business that deserves the bad reviews. Who knows? The point is, if you want to help customers focus on the mid range, in this case they would focus on those with between 7 and 15% bad reviews and help them reduce the number. So focus on the mid range. This applies to almost any measure of churn risk. Those who are in that bottom cohort with the highest churn rate, they might not be worth your effort. So really look at it carefully. Lastly, figure out your stickiest feature. And for broadly this was integration of the business's booking system. They found a much lower churn rate whenever one of their customers integrated their booking system. So now that's a top priority for the setup. And most businesses will have some similar sticky features where if the customer gets that set up, it'll really increase the integration and the stickiness of your product. Okay, gonna move on from broadly now and look at another case study, a company called Clipfolio. Clipfolio is a data analytics cloud app for building and sharing real time dashboards of mission critical company metrics. And so in English I guess you'd say they're a BI tool, you know, a dashboarding tool for company data. So they have events in their system that are things like dashboard views and edits. And from those events, one metric they calculate is the active users per month. Another basic metric they look at is the account tenure, which is what I call the age of the customer, I call it account tenure. So it's not confusing with the demographic age of a consumer. An advanced metric we'll look at for Clipfolio, which is based on the active users, is the seat utilization. So that's another ratio there. So we'll look for that in just a minute. Let's start by looking at the time based cohorts based on account tenure. And so this is from the real data from Cliffolio system, but this is very typical for other companies. And again, we're not showing the actual churn rates here. These are churn rates on a relative scale. But you can see there's a very significantly higher risk in the first part of the first year, peaking at around one half a year. The churn risk comes down again in the third quarter but is up again at the one year renewal. But for customers that make it past the one year renewal, churn gets much better. And by year three, churn is less than half what it was in the high risk period at the front. Now this really shows you where to target with the onboarding and training. It's in that crucial, you know, first half year. And yeah, every company should definitely look at a chart like this. You can find one technique to do it using what's in my book. This can even be done in a spreadsheet. Lots of people do this in spreadsheets. I'm not really an advocate, you know, of doing this all in spreadsheets, but it is doable. But absolutely. Look at the churn rate based on the length of time your customer have been customers. Okay, turning back to the behavior on Clipfolio, here's another very typical relationship between product use and churn. And I say it's typical because that standard behavior, those customers in the bottom cohort, in this case it's one active user, have a much higher churn rate. Then you see a rapid decrease in churn rate in the next cohorts that have.