Running Growth Experiments at Scale with Growth Ex Machina | SaaStr Software Community

So I'm Guillaume Caban and today I'm going to talk about the playbook to running growth experiments at scale. So the first thing I want to put out immediately for growth people and for founders is that I want to trash out all the KPIs and we're going to focus just on revenue. That seems obvious, but you see, most companies don't do that. But before I go into the dough here, who am I, what have done very quickly? I worked at a few of those recognizable companies, most notably lately in the Bay Area segment and Drift where I've driven a significant growth. And if you wonder what significant means, I've been at those companies while we grew in about 18 months, from 50 to 200 and 300 people in 18 months. And my, my focus is SaaS and my, my job is to create a defensible go to market moat. Right. My focus is only revenue. And when we talk of, when we talk of hyper growth and I go back to Drift and segment to give you an idea of how fast and how much growth there is, we did 3x revenue at segment in 18 months and we did 5x revenue at drift in 18 months. And so you wonder like that sounds good, but how good is it? Right? Well, if you look at the KeyBanc Capital 2018 distribution of growth, so you get all companies here that are surveyed for like their growth over one year. Right. What you see is that segment is up here and drift is off the charts in terms of year over year growth. And these days I do advisory. We don't care about that. That's fine. So the first question that you usually ask yourself is how do you compete? Okay. And the thing is most of you are founders here. Can we have a show of hands here? Who's a founder? Cool. Most of you are founders. Okay. Within those founders, who is like a product technical founder? Like okay. And who's like a more like marketing or business founder? Okay. So more technical and product people kind of makes sense. The issue with like most, I'd say technical, product driven founders is that they don't see the value. They don't value marketing as much as they value the product. Which means they under invest in marketing. They don't see marketing as the way to compete. And I'm going to show you, I'm going to show you what the difference and how that works. But first I want to focus on how I compete. Right? And I compete by doing insane innovation on the marketing side, investing as much on marketing through engineering as you can invest on your Product and through creating an insanely good user experience at the very first step at the top of the funnel. Okay, but if we go into details, what that means is that when I build growth teams, I always get the commitment from the founders to get engineers on that team. Right, because that is how you're going to be able to create a differentiable marketing, how you can compete differently. That team is going to be independent. And I'm going to talk in a few minutes of growth team structures. I split out traditional marketing, so events branding, that's great, that's just not measurable on the revenue scale. So I split it out completely. We trash KPIs for marketing. We keep only revenue and then we have a high pace experiment framework. And I'm gonna cover those elements. Okay, but first, why growth team? You know, we talk of competing, we talk of like being better than others, and then we say, well, we gotta have a growth team. But why, like what's the benefit of having a growth team? What do they do differently than product? What do they do differently than marketing, for example? Well, if you want to compare like growth and product, for example, is that if you think of product, products has to succeed every time. If you're a founder and you have a product leader like a head of product or a pm, and that person fails two launches or three launches successfully, that person is out, that person's fired. Right? Because that person failed at doing that job, which is launching products successfully. That's not growth. The difference is that growth is like a VC model. Okay? We're not trying to be smarter, we're just trying to think. We don't know what's going to work. So we need to experiment. We need to throw a lot of things at the wall. If you know what's going to work, then do it. Don't experiment, don't try, do. Okay? The principle of growth is that you don't know what's going to work and you want to find stuff that your competitors haven't looked at. Okay? So you've got to come with that new paradigm. And that paradigm is that just like a vc, you're going to have nine out of ten experiments fail. And so the only way that you can win is by testing a lot, by lowering the cost of testing and, and by being really good at measuring what's happening. And if you look at the typical, like I'd say hype cycle, this is like the typical curve of hype cycle. As a founder, your company's gonna do like that, you're gonna launch A product or a company, and you're gonna have some early success, okay? And you're gonna think everything's going great. And then you're gonna reach the maxima of that first, let's say a strategy or channel or target market. And then the conversion rates are gonna plummet, the CAC is gonna go up. And then you're gonna be in this, in this valley of fear here in the middle, right? And that's gonna repeat over and over and over again, all right? And so if you think like, when do I need a growth team? This is when you need a growth team, okay? When you're going up, you have no product market fit, okay? And you're gonna, you're starting to find that product market fit. Once you have the product market fit and you've reached the maximum of the easy to close customers, then you can hire a growth team. Okay, let's go into innovation. We talked of doing like insane innovation. But who wins when we don't innovate? I'm gonna tell you who wins. Who wins is the big brand advertising networks. That's Google and that's Facebook. They drive around 70% of all the ad spend. So what that means is that you are transferring your hard, let's say acquired VC money directly into their pockets. That is not innovation. That is not what your VC wants, okay? You're not building anything sustainable if you're just competing on Facebook on cac. That is not a good strategy. That is not what your VC wants. And when you look at the challenge that we have here is that the CAC has grown year after year, both on B2B and B2C. Right? We're now at 70% higher CAC than we were five years ago. And that trend is not on a slowing down path. It's not going to stop. Which means that if you think of the ACV in the market, how much you sell is more or less stable, you. Your CAC is going up. What that means is you're gonna hit that kind of like sound barrier where you're gonna crash. Cause you're just not making profit from those leads anymore. Okay, cool. So the smart CEOs, what are they doing? The smart CEOs, they are transferring their budget to technology. So if we look at this slide here, what we have is that all the CMOs are transferring their budget to, from Martech, from agencies to technology, even labor is going down. What I wanted to explain is that the budgets. I have a graph on Gartner here. I'm going to explain the Graph. So bear with me. So what you see is that there's all those categories of spend, and the spend budget is going down in all verticals for the top 100 CMOs. But Martech, Martech is the only one going up. Well, that means that the smart CMOs have understood that to compete, advertising is not a good solution. Hiring more people is not a good solution. Being smarter through Martech is the solution. Okay? And what that comes with is actually engineering, right? You can do stuff by hiring engineers and in your, in your team, in your marketing, demand, gen growth, however you want to call it, right? By having engineers to build products, like, stuff, you can do stuff that you can't say just with marketers. One thing that you got to think is like, we got to have, like, got to calm down our egos, right? That's what really we need. When you think about it, you're trying to compete with, you know, most of you have competitors. Who has a direct competitor here on the product, okay? So for those of you who have a direct competitor, even if those have like an indirect competitor, think about it. If your marketing team has like Marketo and then does ads on Facebook, like, how are you gonna win versus your competitors? You think you're just gonna have, like, better branding, you're gonna have, like, better ads, you're just gonna have better marketers. That is like a weak proposal. It's very weak. Okay? There's only two things. One, you can rely on products. So that means that your marketing isn't doing a lot. And so either you rely on your product exclusively, but that means your marketing team isn't driving a lot of value, okay? If you want them to drive value, it means they should be able to sell a product that is no better than your competitor. That is a good marketing team. That's how you should evaluate them, okay? And the thing is that growth strategies, growth tactics diminish over time. Which means whatever we talk about today has a shelf life of like, if not a couple months, at least a year, right? Because you're first to a strategy, you're first to a channel. Eventually, if that works, everyone goes to that channel, that strategy. And the benefit, the conversion rate up that you had actually goes down. To give you one example, you look at that ad banner at the bottom right here. That ad banner is the very first ad banner that ever existed. Okay? That is 1994 for Hotwire. That banner ad had a conversion rate of 74%. That's 74%. I've been doing marketing for the past 15 years, I've never seen something click for 74% ever. Right? And so yeah, that was the first mover strategy that worked well. And the same thing is true for video right now that we have in emails or like all the videos and stuff like that. It works really well. Eventually that's gonna diminish and also revert to the mean. So we talked of why you wanna create a growth team Innovation. When do we want to build it? Easy after product market fit. Okay. Once we have enough engineers that we can at say, afford to have engineering dedicated to building demand generation. Okay. And when the initial growth starts for the all the founders in the room here that are wondering, like, how should I structure the growth team? Who should they report to? I got a couple of examples for you. So the first model is the cross functional model, okay? And so that model has, let's say, a growth team that is like kind of like peppered around your organization. And as you can imagine, it doesn't work that well to create innovation. It works really well to, let's say, minimize the rifts and conflicts because there's basically no growth team. But because the product and the engineering are incentivized to minimize risk, you don't have a team who is incentivized to take risk. So there's little innovation. The second model is the independent model, the one from Facebook. Okay. That one has a dedicated growth team with a growth part that's completely separate, that creates really good innovation. Fantastic innovation. It has the opposite problem. Lots of cultural, fashion and rifts and conflicts. Why? Because you got people in product engineering and design and marketing owning basically your entire business. And now you have a team that is going to say, hey, we're going to like work on that one thing, right? And we're gonna own it and we're gonna like hack with it. And then once we're finished playing, whomever owned it before can get it, can get it back. Obviously that didn't work. I did that. I did that. I took the sign up from the marketing and engineering team and said, hey, we're gonna, we're gonna kill that conversion. We're gonna be much better. When you do that, you are humiliating whoever owned that part of the product by taking it, by saying that they did a poor job. And then there's only two outcomes. Either you succeed so you humiliate them a second time by showing that you were right and they actually did a poor job, or you fail and they're gonna point the finger and say, you see these folks in growth, they Thought that they were really smart. They're not. Right? So that doesn't go well. So that is like, that can work if you have a really clear charter throughout the organization that the growth team is here to experiment, fail, and help find stuff that is unique. Okay. If you don't empower the team through that very strong charter, you are setting them up for cultural failure. This model, the embedded model, is a very common one. And unfortunately we don't have the animation here. But basically what happens with this model, which I don't recommend, is that it's a power war where the growth org is going to move from product to engineering to marketing, depending on what, which exec has the most power with the founders at a given time. And you can see those teams are just moving back and forth, which means their KPIs incentives, their strategy is basically aligned with which org they belong to and not the entire organization, which means. The last model I really like, the one I've been implementing, is this one. So this one was created by my friend Sean Clauset, Atlassian. It has this kind of mixed model. We have the engineers and the designers reporting into their guilds and a few core functions going to growth. This optimizes for both. And the real good reason why I like it, Think like, I need engineers in my team. Am I an engineer? No. Can I help an engineer grow in their career? Can I help an engineer debug code? No, I can't. Which means that person needs help from the broader engineering team, the broader product team. And same for designers, which means they need to be embedded in some way in those teams. Okay, so this model works really well. So we talked of, when we talked of how. Let's talk of, like, how I fail because my ability to fail defines me better than my ability to succeed. And to be honest, it's actually easy to reproduce failures if you don't, like, look at how other people have failed. It's really hard to reproduce successes because they're really unique. So when I think of a new role and when I consult these days and I help CEOs, my first question is, what does the boss want? So boss is CEO, and so the boss wants good KPIs and a good roadmap and basically not breaking the culture and being reliable. To be honest, I've failed at all of those four, sometimes multiple of those. Okay. When you think of growth teams, most growth teams fail at being able to communicate what they're doing, why they're doing, and what impact they have on the bottom line. So let's start with KPIs and KPIs. When you think of KPIs, it means, well, growth teams is here to build demand generation, so does marketing. And it ends up with something while you have like sales versus marketing, then it doesn't end well. Okay? So if you think of sales, sales are incentivized to do a good job because they have a clawback clause. A clawback clause, for those who don't, who don't know, is basically that if you oversell to a customer and that customer churns within a given amount of time, three months, six months, the commission is lost. Who in marketing has a clawback clause? No one. Which means we can push horribly bad leads on the funnel and have no consequence. Right? And that's how we, that's why we have a rift in conflict with sales. We're not incentivized to long term quality, we're not incentivized to ltv. Right? We as marketers, as dimension people, as growth people, we should be incentivized financially incentivized to long term customer success, okay? The same is true for forecasting when you look at how growth teams work. This is one example. This is a growth team tool to have all of your projects and your ideas and list which ones you're working on and why. There's a couple of issues here. The first one is that it's a laundry list of ideas. The second one is there's no revenue, right? And so when the boss says, hey gee, what are you working on? And say, I'm going to work on that project because it has a 6.7, what does that mean? It means nothing. What are you committing on? Nothing. Really? And look at the other tools. The same thing. I asked a lot of my friends which tool they use. Same problem over and over again. There's no revenue, there's no commitment. You're not committed to revenue, you're not aligned with the sales team, okay? I tried implementing that ice, so impact, confidence and ease that I score three times. I failed three times, both with my team and with my CEOs. And the last thing is visibility and forecasting, okay? Because you don't know what's gonna work, okay? You, you can't give that visibility, you can't give the forecast. And so you have the problem of not being able to commit. You have the problem of not being able to forecast what you're going to do and what impact it's going to have. The new model I want to implement is trashing all KPIs, trashing the systems that are laundry list of ideas and moving to revenue. And if you want to move to revenue, this is how you do it. Okay. This is from Point9's founder, Christoph Jens. So they are a VC firm in Germany and he has proposed this model where your entire funnel is just revenue. And revenue flows from stage to stage. Just as in sales. It's true in marketing it means the maximum value you will get from your prospect is at the beginning. Each stage after that you just destroy revenue by losing people by not being at the at the most efficient that you could be. And so the revenue flows from one stage either to the next stage or to close lost. And so you start having that mindset that you could use the revenue as a metric for marketing, even the very early stages. And so you convert all of those KPIs, such as New visitors and returning visitor into revenue metrics. You divide how much revenue has gone through that stage by the number of people. You get a revenue per unit object. And now you say, well, this is how much I have at each stage and you can start having those metrics. I'm going to show you now how I have my actual dashboard. This is my dashboard for one of my companies, okay. And that dashboard has the future value, okay. That's added to stock weighted by the quality. So what you have here is that the top, I'd say leftmost number here, which obviously is blurred out. It, it tells me how much I'm adding to my, to my pipeline. Okay. And you gotta think of this like a shop, okay? Just like a shop, we have revenue that comes in revenue that is in stock and revenue that goes out either in close one or close lost. And each of those bubbles shows one of the stages, how much I'm adding, how much I have, how much I have closed, how much I have lost. And that is a marketing dashboard. And so you can run your growth team just like that. If you want to think of your sales team best metric, which is sales velocity, which is the number of leads, times the average contract value, times the conversion rate divided by time conversion. The sales velocity is how you measure your sales team's efficiency to add revenue over time. You can do the same for growth, which is how much revenue potential revenue your growth team is adding over a period of time. Right? And you can do that at all stages of the funnel. Let's look at how we do it. You take the metrics, as I said, you convert visitors into revenue, right? And so you track all your conversion Points you now have, let's say visitors is like each visitor is worth $10. And then you have a new metric, which is qualified visitors. And you're going to have qualified signups and so on and so on and so on. Then you have all of your projects, all of your ideas, like this. Now you have all of the revenue, okay? The total per column, and you're gonna have all of the ideas. And you can say at each stage of the funnel, I know exactly how much potential revenue I have. So here you have in the gray area, impact approved. You have the total value of all ideas at that stage. I know also how much time I'm gonna take to work those ideas, okay? And so I can say, well, if it's gonna take us like 5 days to drive 100k, I. I know my value per day. And that's how we prioritize all the ideas. It's revenue per day. Now, one interesting thing is that of course we track week by week, okay? And so we know the performance of the team on a revenue basis, right? And you can start having. There you go. You can start having stuff such as when the boss asks, what project are we doing next? You know? Well, I know exactly why. Because those rank as the highest grossing per day of work. What impact are we going to have? I can commit on $270,000 out of 5.5 days of work. And that is highly predictive because it's based on past historical data. So now just to nail the point, the questions we can answer is like, yeah, what are you working on? How do you prioritize? How do you forecast? If you want to think about it, it means that I'm just like a sales team. I have a number of people. And those people bring revenue from the top to the bottom, some of which will be closed won, some of which will be closed last. It is marketing, but the organization is the same. That's super important because all CEOs that I have met understand basically two things really well. Revenue and sales team organization. Okay? The rest is like, marketing wise is muddy. And so instead of trying to change the mindset, I'm changing how I structure my team and how report by having marketers commit on future revenue. By having marketers actually structure the team as sales teams, you have amazing results. And one thing I want to say is I actually, I did that at drift. We had engineers who had bonuses based on revenue per day achievements. Okay? It's very important on the marketing demand gen side, okay? Just like salespeople. Why do I do that. And how does it work? Well, it's very easy. As I said, I have a VC model. I don't know what's gonna work. My only impact is moving experiments faster through my pipeline. Okay. Moving faster means that I want to be able to have my engineers actually code faster and ship experience faster. I don't care if the code isn't great. So I'm trying to incentivize them to go faster. And money is a great way. They don't really care about the money. They care about winning. Right. And so I have engineers winning with money to move things faster. And so that's what we have in the end. Right. We have a process aligned on, let's say on the sales structure. We have a predictable output and we have a commitment on impact. And that's it. Thank you. I think we have a couple minutes for questions. So the question is, what was the tool? So the tool is. It's built on airtable. There's actually a template of that airtable that was shared by my friend Darius Contractor. The link I believe is bits Ly Airtable Evelyn. If you search on Google airtable Evelyn, you're going to find it. It means experiment Velocity engine. Yes. For your. The revenue projections for each step. How are you determining that? For example, the award reduced the friction. It was assigned like maybe five figures. Yeah. How do I forecast the value? Unfortunately, we had the wrong slide deck here. But basically I use a tool called Madkudu. So they are like a AI machine learning company based in Mountain View. And they, they do like advanced regressions based on past data to predict the value of, let's say leads at each stage of the funnel. And so I leverage their insights into my model. Okay. And so it's pretty reliable. Other questions? Yes, we have questions back there. What are some examples for typical SaaS company that maybe engineers might create in order to help with growth? That's a good question. So I have. If you look on Internet, you'll find a couple of videos of me on like tactics and strategies. Mostly what I try to do is to try and build products for acquisition that are tangent to your core product. Okay. That would push people to register low friction drive value. Say my goal is to deliver as much value as possible to people with as little friction as possible. So a couple of examples. If people need to sign up to get the value, the question is, well, can you create, can you pre create all the accounts, screenshot the value and. And send it into an outbound email to prove the value to the future customer? I've done that once. It worked really, really well. I worked at a company called Mention. We're doing news aggregation. And so we thought, well, news aggregation is really good for people who are just launching a product, but they don't have the time to sign up and set up the tool when they're launching the product on Product Hunt on day one. So how can we help with that? Well, we pulled the API feed from Product Hunt. We created the news aggregation account for each of those launches every day. And then we outbounded an email with a screenshot and a link with the token to like, claim your account. Okay. Which means as a future customer, you have in an email the proof of the value. Very little friction, just like click here and claim your account and the data is already there. That obviously requires engineering, but it really helps. I'd say the customer on the right path. That's one example. I've done stuff at Drift, for example. Drift is a live chat bot. And so the principle of live chat bot is to get your sales to move faster, people to engage with you faster. And so we did a tool as an acquisition that was meant to measure the response time of your sales team. So there was a website which is now down, but there was a website called getmyresponsetime.com where VP of Sales and sales leaders could go sign up with the domain and the email. And we would have contractors go and automatically request a demo with generated email and would measure the time between the demo request and the demo follow up in hours. And we then measure that against the industry standards to tell them how long they took and how much conversion they're losing. And then the VP of sales, damn, we're losing opportunities. We're not converting as high as we should. And the obvious follow up is like, maybe you need a live chat, but it's tangent. Right. So a lot of those things require some level of engineering and it's much more than just building a landing page. Right. It usually requires some level of like backend engineering and stuff like that, but it creates value. I'm trying to use engineers to push value up the funnel. That I think is the core strategy. Push value up the funnel. Cool. I was wondering if you could quickly.

Elaborate on how you compensate the engineers for hitting the certain growth targets.

How I compensate? Yeah.

Is it like a person, like a commission or a flat rate?

Just what's worked for you? Yeah. So basically the target is to have them produce at least $1,000 per day worth of experiments. The thousand dollar mark is there because it's more or less what the cost in San Francisco. Like 200 days of 220 days of work, you know, 150, 180ks cost plus like bonus and whatnot. It's like they're more or less the same. And so can they add $1,000 per day worth of value to our pipeline? Okay, and if they do that, they get bonuses. Now if you think of the mic, what's going to happen? Okay, so basically they want to win. There's this laundry list of ideas that are rated on a scale by experiment, value per day, projections of how valuable is that idea. Obviously they all want the one at the top. Okay, so how do they compete for the best experiment idea? Well, someone's going to say, well, I can do this project in five days. And the other one's going to say I can do it in three days. And I'm going to say commit. Right. And so now they're committing to going faster and it's forcing them to make some shortcuts and to work extra hard, which is exactly what I want. Because I don't care about the code quality at that stage of the process. I care about having the answer really fast. Right. Because we're trashing, remember, we're trashing nine out of ten ideas so we can take care of code quality later. Okay, so that's how we do it.

Smaller stage companies, it sound like, and I'm sorry I couldn't quite see the slide there, but it sounded like you were saying dedicated people for this kind of like experimental team. Smaller, smaller stage companies that can't really dedicate a four person team across those different guilds. And then a follow up to that. How do you think about reporting and who's driving that? Who's typically in that seat?

Yeah, it's a good question. So at a small stage it might not make sense. You might not need a growth team as a separate function. You probably don't actually. Right. Because the question is, have you hit product market fit and have you exhausted the obvious channels? If you haven't exhausted the obvious channels through obvious marketing strategies, you don't need to experiment. You should just execute on those first channels. Okay. The need for growth team comes when you have exhausted the obvious things and you're getting like, you're getting trouble continuing on the same growth at say path in terms of like month over month growth and your CAC is going up now you need to experiment. That's the first thing. The second thing is for me, if you really want growth as an experimentation engine to figure out stuff that is not obvious. You need two things. One, you need a very clear charter throughout the organization that this team is taking risks and they're gonna fail often. And two, that team needs to report to the CEO, okay? Because stuff's gonna break and stuff's gonna fail, and they need air cover. Right? Otherwise, that team's not gonna survive longer. Cool. Thank you very much.