SaaStr AI Day: Tiering our Entire Customer Base in Days, Not Weeks

Thanks so much, David. Hi everyone, it's great to be here. Um, as David mentioned, I lead the customer success team here at Backstory. I've been with Backstory for about 5 years, but prior to that, my, uh, history and experience has been in sales and sales leadership. So the scenario we're going to talk about today is not something that's unique to my position now and here at Backstory. It's an exercise that, um, I've gone through at previous companies, and I'm sure anyone here in go-to-market or strategy has been involved in at some point or will be involved in at some point when it comes to your go-to-market strategy and essentially determining the focus of your go-to-market team across your customer and prospect base. So the, the goal of today is really to have you walk away with a perspective and some ideas on how you can bring together, you know, key data sets and also fill in the gaps where there are perhaps missing points of data that you need across your entire customer base to be able to create a comprehensive picture on what each key account has that's unique, and then how to measure your customer base or a portfolio of customers against that same threshold to really build a tiering framework that's data-driven and also incorporates your own human perspective along with the qualitative conversations and context that can only be uncovered through the raw conversations that have happened with your end customers and prospects. So really, like, today we're focused on bringing together a whole plethora of perspectives and data sources into a workflow that allows you to tier your accounts in a way that previously in an old life used to take from my team plus 4 other teams approximately a quarter of work. And today, I'm going to talk to you about how we were able to turn this around in a matter of 3 days. So like I mentioned, this is something that came to me honestly from an ELT ask from our board. We had a short turnaround time, and we needed to be able to look at a subset of our customer base and provide a tiering mechanism that really dove into the characteristics of what our best customers looked like and reflected across these 141 accounts, the key factors and where each one stood against those benchmarks. Out of the outcome of this really was we were able to make some really great decisions on how we redistributed our go-to-market teams in terms of redistributing seller portfolios, reestablishing SLAs, creating playbooks for specific tiers of accounts, and really aligning our growth targets to the specific tiers and what we believed we could achieve across each tier. It also allowed us to make some difficult decisions about customers that maybe sat at the lowest tier and forced us to reflect, looking at real data, as to whether or not those customers were ones we believed would truly grow with us and stay with us, considering all the factors we were looking at. So to start off, we had to first define, right, what does the golden customer at Backstory look like? And this one started with not just a perspective on who was our longest standing customer or who was our largest customer, We really, you know, took the perspective of the full account teams and senior leadership on what made one of our— one or two of our customers truly ones that made us feel like they saw Backstory as a core part of their tech stack. They were building systems around the Backstory platform. They were planning 5 years into the future with Backstory at its core. They cared about our roadmap, wanted to inform it. And can continue to partner with us to find new and unique ways to push our product that served more and more use cases across their like ever-evolving go-to-market needs. So we had a couple of golden customers that we called in mind, and that's when we started to strip away what made those specific customers unique. And I call this finding the signals that matter. Within the golden customer, there were a few things that we came to identify that were, that stood that customer apart from the rest. There were some characteristics related to things like that customer's go-to-market process maturity, their tech stack mix, perhaps their AI maturity. So how inclined was the customer to be investing and moving up an AI maturity curve and doing the change management internally to become an AI-first company? And how quickly was that happening? Whether the customer was deeply embedded or leveraging partner motions. These are all sort of internal to the customer's process factors that we recognized were unique to some of our golden customers. And then more specific to us and Backstory, we also understood the track record of these customers. They had really fast deployment velocity. Which means that, you know, we landed with a small and key use case, but we very quickly grew the customer to start leveraging our full stack. And that happened over a compressed period of time. Executive visibility, our initiatives, the Backstory initiatives within this customer organization had executive-level visibility. They were being leveraged by executives as well to make key decisions. And we had strong executive engagement there within the organization. And then lastly, a key factor was obviously the total addressable market within the customer account from when we landed it to where we were now, as well as the remaining white space. So looking at these, these characteristics and these factors, I knew we had a lot of this data sitting in silos across a variety of tools. But I also knew there were certain points of certain signals listed here that we had not repeatably measured across our customer base. So that was where we needed to start first. We needed to fill the gaps on the signals that we knew we didn't systematically have across our entire customer base. So an example of that is this concept of AI maturity. So like I mentioned, within our golden customers, these were customers that were quickly moving up what we called the AI maturity curve. And we had actually created internally a framework where we measured a customer's level of AI maturity against a scale of 5 levels. And this was a combination of factors based on their internal culture, their level of investment, their technology stack, the talent that they had, and also their willingness to engage with us on certain problems and likelihood to help us push the roadmap for our product in a more AI mature direction. So we created this framework internally, but it's a pretty grueling process to be able to continuously and manually maintain, like categorizing and asking the account teams to categorize their customers in this way. So to add this signal to sort of our repertoire before we did this broader tiering analysis, we started by creating what we call a Backstory signal. So in Backstory, and I can actually hop over so we can look at this live because that's always much more interesting. So within Backstory, we have what we call signal builders. And these signal builders are actually like a mini LLM that basically allows us to pull together specific data sources that Backstory has access to and almost run systematically a prompt against each account or, or deal within the CRM to produce an output or a value. So with Backstory, we created what we call the AI maturity level signal. Within the signal, we're looking at a variety of data sources. So we called out specific fields in the CRM that we feel are relevant to helping us uncover where a customer sits their AI maturity curve. But we also have things like public company information. So, you know, public news about new investments or entering new markets with AI-forward products, for example, is a signal that we could leverage here. But then most importantly is a lot of the richness and the intel about the customer and where they're at, how they're engaging with us on our AI roadmap specifically. Comes from direct customer conversations, right? Those are like the emails that we're having across the account team. Those are the meetings that have taken place. Those are the Slack messages that have gone back and forth. And all of that is comprised into what Backstory calls a chronicle, which is a timeline of events that occurs on a specific account. So across this, in this set of data, we created a signal that is looking at, like I mentioned, those 5 categories, and we provided instructions to the actual signal builder to look for key characteristics across those data sources to categorize the customer in that way. We predefined a set of outputs, which is the 5 levels, and we're able to then run it actually across the customers that we select and produce an output. I'm not going to run it against our customer base live in front of you, But what you'll be able to see is that as I go through and select our customer base, I'm able to get an actual maturity level output as well as a rationale as to why that specific maturity level was, was selected. So this was a precursor to me being able to actually start doing sort of this comprehensive analysis. And I didn't do that for just the AI maturity. I also did that for uncovering what the customer's tech stack mix was. We had some scorecards that we collected maybe during presales that gave us a hint to what the customer's tech stack was. But things may have evolved since then. It might be this customer may have been around for 2 or 3 years, and we wanted to pick up from those conversations what has changed in their tech stack and how that customer has evolved. So that's the identifying the signals that matter and filling in the gaps. Then we get to the arduous task. So this was probably the task that in a, in a previous life would require a project manager or a key project owner to go cross-functionally across the product team, the BI team, finance, several functions to start to collect key data points related to the customer's TAM, health, revenue, renewal rate, delivery risks that came up, product feature requests, adoption levels, etc., right? The beauty of us being a cloud shop today and having all of these connectors available, available to us on hand made it so that I didn't actually need to ask anyone. I could actually pull all of that information myself and supplement it with the signals that I just built. So that's really what I'm going to walk you through live today on how I ran that workflow and what the output looked like. So like I mentioned, the only sort of manual pull I did in this process was I did pull our customer list directly from Salesforce. Along with the signals that I had just built through Backstory. So I aligned all of our customer revenue numbers, renewal rates, as well as other key information that I'll show you with those key signals that we built. I then ran it through Claude and enriched that with the public data, open feature requests, product adoption metrics, custom signals, and real context from customer conversations. Cloud did its thing. It reconciled that information. I used CoWork because I thought it was important for us to use a series of specific steps in order. And then I was able to produce this 4-tier model that actually segmented our customer base and helped me understand where the growth potential was across our top-tier customers using real TAM information and extrapolating the other dataset that I provided it. So one of the things I want to call out, a human in the loop is always a very important factor in any of these workflows, right? This workflow that I'm going to show you today, it didn't happen, it didn't come from my first iteration. It took 3 or 4 iterations for me to revalidate you know, the confidence or the weighting that I wanted to give each of these data points. And so I'm going to show you the final output today, but just know that there were 3 or 4 iterations that came before this where I learned that, you know, through the way that the customers were tiered at the end, maybe we didn't actually need to rely on feature requests as such a strong signal that a customer was a more engaged or a higher tier customer than, than others. So just wanted to bring up that caveat that there was a lot of iteration in this process. That being said, I was able to do it without pulling in several cross-functional teams, which is great. So I'm going to go ahead and switch over to my trusted cloud environment. So like I mentioned, we are CloudShop here at Backstory, and we have several of our connectors set up. The ones that were relevant for my project specifically were Amplitude for our customer utilization and usage data. Atlassian held our Jira and GAPS. So these were the customer feature requests or gap requests that have come in throughout the years. We have the Backstory MCP. Now that is the historical context. It's all the conversations and engagement that's happened with the end customer. What was said in a chronological order. And then lastly, Slack. We use Slack a ton here internally. We actually have a customer channel an internal customer channel for every single customer. And that channel is sort of the, the source of truth for any internal alignment that we have on account strategy, next steps, the first place we call out risks, et cetera. So there's a lot of richness on internal team dialogue within Slack channels that I also wanted to pull into this tiering analysis. Because it could shed a ton of light on the perspective of the customer when it comes to backstory or risks that we have personally perceived and called out over, over our time as an account team internally. So I'm going to go ahead and drop my, my prompt. I have already created a markdown that we'll go through together, but I'm going to kick off this workflow because as we know, These things can take a little bit of time. So as this is running in the background, we'll check in on it periodically. I'm going to go back to my skill. So I'm going to talk you through some of the key steps and the order that I've asked the Claude project to run. So starting with— ingesting and normalizing the CSV. Actually, that reminds me, I didn't actually add my CSV. That's what happens when you start moving too fast. So let me actually rerun this. I'm gonna drop the CSV this time and start running the task again. We're gonna be uploading this CSV specifically. This is actually what I exported from Salesforce, what I mentioned earlier. This is just a small subset because, uh, 150 accounts might take quite a bit of time for us to run live. But I have the account name and then I have a couple of the key backstory signals. So the backstory executive engagement level, the predicted account health, the AI maturity level that I had created, and then upcoming renewal date, renewal ACV. These are some of the things that I pulled directly from Salesforce and that's what I have in my CSV that I dropped in there. Now going back to the actual project, when we look at the markdown, this is the order that I eventually got to through many forms of iteration to be able to pull together this tiering framework. So we're starting with ingesting and normalizing the CSVs, ensuring that Claude is able to connect the dots between the accounts that I've selected in that list and then all the other data sources that I'm adding. The very first data source that I like to start with is the Backstory MCP, and that's just because that is the— I think of it like a primary data source. It's the source of truth. It's the conversations that have happened, and it tells me the most about what is the most recent engagement, how, how have recent conversations gone, what risks have come up and what opportunities has the customer presented to us. And there's a lot of richness in this context. So we're starting with that engagement to help us build that framework. From there, we're moving on to utilization. So like I mentioned, our golden customer is a customer, or are a few customers that have quickly landed with a small use case and expanded rapidly into several other use cases and across our tech stack.. And that is something that's visible through our usage data. So I'm able to pull that directly from Amplitude and through the connector. The third that's really important is the growth potential. So, you know, backstory, backstory, we, we have a per-seat model that comes specifically from data subjects that are part of the go-to-market team. So the size of a go-to-market organization is a key indicator for us on the total addressable market within a customer account. Between the Salesforce data that I've given and the public research, I'm able to uncover how much of that go-to-market customer base, that go-to-market team within our customer base has actually been secured and how much remains as white space for us. So that's the public research that I've added in as the third step. The fourth is the Jira and Gap boards. So our customer success team and sales engineering team are extremely diligent in logging customer feedback, and our customers are not shy in giving us feedback. So there is a ton of richness within our Jira boards that help us uncover if customers are asking things that are pushing the envelope on the AI maturity of our roadmap, or are they stuck in some of our maybe lower-level, less AI-mature features, and they're asking for things that we know potentially indicate that they're still leveraging, or they, they aren't as high on the AI maturity curve as we would like for our higher-tier customers. There is a reconciliation step here between adoption signals and feature gaps that through iteration I learned was important. Some of them contradicted one another, so I needed to provide an additional perspective on a lot of gaps doesn't necessarily mean that the customer is unhappy. Again, there's nuances in what the customer has been asking for based on the types of features that they're requesting. And then lastly, and most importantly, was the actual scoring model. So for us, The, the 4 main categories when it came to tiering the customer base came down to these 4. So the growth potential in the account, the AI maturity and velocity of AI change in that customer account, the level of engagement, so their perception of backstory, as well as who we were engaged with at what level in that account, and then the overall health of that account today. So I, this is another area where I went through a couple iterations to be able to really nail down which of these levers should, should we lean on more for creating our Tier A to Tier D ranking. So this is where we got to. Our Tier A was a really small composite of 8 accounts actually. And these were accounts that we truly believed we could grow over 10x in the next 2 to 3 years. And these were customers that scored highly on all of those factors that I mentioned earlier. And there was strong growth potential and engagement. We redistributed and are looking at how we reassign sort of our account team focus on these tier, tier A accounts, and we're currently building really specific and deep playbooks on the types of engagement we expect to see for these tiers. For Bs, we have a longer tail of about 21 accounts. These are customers that we believe we can grow to this 10x potential as well. However, on a longer time horizon, and this will have a separate playbook. Tier Cs are customers that could go either way. They are sort of at a critical juncture where they will either continue to partner with Backstory and see us as a core part of their tech stack, or they will not mature quickly enough to meet where Backstory is headed and will likely turn into a Tier D customer. And Tier Ds are the ones where we needed to reflect and come to the sort of hard realization that maybe these customers aren't going to be the ones that are going to be with us in a couple of years. And that's because we as an organization have evolved and these customers have not necessarily perhaps evolved as quickly, or there are specific key characteristics around those 4 categories that I mentioned that have been insurmountable over the past several years. And we believe that we may need to cut our losses and and think about how we service these customers differently. So I will stop there. I know we're running low on time. I'm going to just pull up the final output from the tiering project so that we're— you're able to see what that looks like. And then I will open it up for questions. Um, actually, while I'm pulling that up, um, since it's taking a minute, I don't know if David or Amelia, you wanted to start fielding some questions.

Sure. Yeah. So first question being, um, for folks that maybe want to tier on more signals. Did you find in your workflows that there's maybe a case where there's too many signals or triggers tripping up the ranking system? Like, what did you find was the most ideal?

Yeah, so what you'll, what you might have noticed in the markdown that I finally built, I started with about, uh, 8 different signals and I ended up, um, through many iterations of this process. Kind of having to narrow down or rationalize a lot of discrepancies or contradictions that would happen between signals when Claude was producing the output. Um, and then I ended up actually getting down to 4 main buckets. So 100%, I think, uh, you can start with a lot, but you're probably going to have to narrow down and reconcile, um, conflicting data points because, uh, Yeah, the, the narrative can get lost with, with a little bit when we have a little bit too much that we're working with.

Yep. Related question from the livestream was, can you explain more to why you decided to omit customer feature requests as part of that dataset and signals?

Yeah, so I didn't actually omit it. I actually combined it with adoption because It was actually interesting. It was an interesting learning through this process. We had a strong correlation between our highest adopting customers and the number of feature requests that we were getting from that customer. So when I initially ran this analysis, we were actually like dinging points, so to speak, on the customer engagement and health based on the number of feature requests that were coming in. Because we were perceiving that as something that was negative or an unhappy customer. But we reckon when we reconciled that with the adoption data to say like, if a customer has a high adoption in this feature set but is also requesting a lot of AI-forward features, that's actually a positive signal for us in this case.

Awesome. Next question is, how long does this usually take to run and how much of it is directly connected and passable to Salesforce?

Yeah, so this project that I had running in the background, it is still running. So I think it is going to take about— last time I did it, it took about 20 minutes to run. Sometimes it's faster, sometimes it's slower. But collectively, like, to turn around the full analysis end to end, I had only approximately like 3 or 4 days. So I was able to build it in a really short amount of time. That being said, this was only like the first iteration and we have feedback from ELT that we'll incorporate. But when I compare it to doing this exercise several, several years ago, it was a full quarter milestone that we had to hit as a cross-functional team.

Yep.

So yeah.

Two more questions we'll try and hit before we lose you. Next question is, how often do you plan to re-tier the accounts? Is this something you just mentioned, you know, maybe prior to this workflow taking days, you would do it maybe once a quarter. Like, how often do you guys plan to rerun this? As of now?

Yeah, quarterly, because it's really important for us to understand what's changing, right? If our customers that we believed were Tier A are dropping into Tier C, for example, after a quarter, it's important for us to keep track and like also refresh our our tiering definitions if that's the case. But we also want to see those Tier C customers either move up into a higher tier like I talked about, or move out. And being able to track that change closely quarter over quarter is going to be helpful for us to understand how we're going to be on track to achieve our growth targets.

Yeah. And are you using the same ranking system for new versus existing customers, or do you have different signals that you're maybe more so looking at when they're a new customer?

Yeah, so right now what I showed today was exclusively for our existing customer base, uh, and a lot of that is because obviously I'm leaning on things like utilization data, uh, existing customer conversations, feature gaps, etc., right? So it's, it's relying on a history of a relationship there, but it is something that we have also considered building and replicating with different data points for pre-sales.

Last question. How has this tiering impacted the sales team and any actions they've taken against the data so far?

Yeah, so this is all very new to the Backstory team, but some of the ways this has really changed how our sales team really engages with our customer accounts is, first of all, we've established a much tighter and stronger QBR cadence, obviously for our Tier A, Tier B, and Tier C accounts. Tier A accounts, and we developed a new executive sponsor program, an alignment program specifically for those accounts to help AEs engage deeply and more quickly at a senior level within our top tier accounts. We've also been able to ensure that the growth targets that we've identified for each of the top tier accounts aligns to the— is a path to revenue for the AE to their targets. So like, In terms of their pipeline goals, they've been adjusted and they're also being looked at alongside the tier, the spread of the tiering that they have in their book of business. There's more to come. Like we have really prescriptive playbooks that we're looking to finish developing and deliver to the sales and customer success team on how servicing these customers looks different. And we have a lot of really great signals we're going to start building to help AEs actually uncover, okay, where do they go next? If there is, if this is a Tier A account and this is what they've deployed, this is what they have with us today, based on all these same signals, where could they, where are they most likely to grow pipeline next within that account? Cool.

Thanks for answering. For folks who want to get in touch with you for any additional questions, what's the best way for them to do that?

Yeah, you can reach out to me on LinkedIn. I'm happy to connect there. And my email as well. I don't know if there's gonna be any follow-up material, but my Backstory email is hia.kamal@backstory.ai. So feel free to reach out. I'm always happy to chat.

Awesome. Yeah, we'll share that with folks as well as like your presentation and some of your prompt things so that they can hopefully duplicate it. With that, thanks, Hia, for hosting the session. Thanks everybody for joining our AI Day session. The next one starts at the top of the hour, so we will see you guys there. And see, thanks, thanks again, Hia.