Running Growth Experiments at Scale with Growth Ex Machina | SaaStr Software Community

[FOREIGN LANGUAGE] So I'm Guillaume Cabane, and today I'm gonna talk about the playbook to running growth experiments at scale. So the first thing I wanna put out immediately for growth people and for founders is that I wanna trash out all the KPIs and we're gonna focus just on revenue. That seems obvious, but you're gonna see most companies don't do that. But before I go into the dough here, who am I, what have I done? Very quickly, I worked at a few of those recognizable companies, most notably lately in the Bay Area, Segment and Drift, where I've driven significant growth. And if you wonder what significant means, I've been at those companies while we grew in about 18 months from 50 to 200 and 300 people in 18 months. And my, my focus is SaaS, and my, my job is to create a defensible go-to-market moat, right? My focus is only revenue. And when we talk of When we talk of hypergrowth, and I go back to Drift and Segment to give you an idea of how fast and how much growth there is, we did 3x revenue at Segment in 18 months, and we did 5x revenue at Drift in 18 months. And so you wonder, like, that sounds good, but how good is it, right? Well, if you look at the KeyBank Capital 2018 distribution of growth, So you get all companies here that are surveyed for like their growth over 1 year, right? What you see is that Segment is up here and Drift is off the charts in terms of year-over-year growth. And these days I do advisory. We don't care about that, that's fine. So the first question that you should ask yourself is, how do you compete? Okay? And the thing is, most of you are founders here. Can we have a show of hands here? Who's a founder? Cool, most of you are founders. Okay, within those founders, who is like a product technical founder? Like, product, okay. And who's like a more like marketing or business founder? Okay, so more technical and product people. Kind of makes sense. The issue with like most, I'd say, technical founders, product-driven founders is that they don't see the value, they don't value marketing as much as they value the product, which means they underinvest in marketing. They don't see marketing as a way to compete. And I'm gonna show you what the difference and how that works. But first, I wanna focus on how I compete, right? I compete by doing insane innovation on the marketing side, investing as much on marketing through engineering as you can invest on your product, okay? And through creating an insanely good user experience at the very first step, at the top of the funnel, okay? But if we go into details, what that means is that when I build growth teams, I always get the commitment from the founders to get engineers on that team, right? 'Cause that is how you're gonna be able to create a differentiable marketing, how you can compete differently, okay? That team's gonna be independent, and I'm gonna talk in a few minutes of growth team structures, okay? I split out traditional marketing, so events, branding, that's great. That's just not measurable on the revenue scale. So I split it out completely. We trash KPIs for marketing. We keep only revenue. And then we have a high-pace experiment framework. And I'm going to cover those elements. OK? But first, why growth team? We talk of competing. We talk of being better than others. And then we say, well, we've got to have a growth team. But why? What's the benefit of having a growth team? What do they do differently than product? What do they do differently than marketing, for example? Well, if you want to compare like growth and product, for example, is that if you think of product, product has to succeed every time. Okay? If you're a founder and you have like a product leader, like a head of product or a PM, and that person fails 2 launches or 3 launches successfully, That person is out. That person is fired, right? Because that person failed at doing that job, which is launching products successfully. That's not growth. The difference is that growth is like a VC model, okay? We're not trying to be smarter. We're just trying to think, we don't know what's gonna work, so we need to experiment. We need to throw a lot of things at the wall. If you know what's gonna work, then do it. Don't experiment. Don't try, do. Okay? The principle of growth is that you don't know what's gonna work, and you want to find stuff that your competitors haven't looked at. Okay? So you've got to come with that new paradigm, and that paradigm is that just like a VC, you're gonna have 9 out of 10 experiments fail. Okay? And so the only way that you can win is by testing a lot, by lowering the cost of testing, and by being really good at measuring what's happening. And if you look at the typical, like, I'd say hype cycle, this is like the typical curve of hype cycle. As a founder, your company's gonna do like that. You're gonna launch a product or a company, and you're gonna have some early success. Okay, and you can think everything's going great, and then you're gonna reach the maxima of that first, let's say, strategy or channel or target market. And then the conversion rates are gonna plummet. The CAC is gonna go up, and then you're gonna be in this valley of fear here in the middle, right? And that's gonna repeat over and over and over again, right? And so if you think of like, when do I need a growth team? This is when you need a growth team, okay? When you're going up, you have no product market fit. Okay? And you're gonna— you're starting to find that product-market fit. Once you have the product-market fit and you've reached the maximum of the easy-to-close customers, then you can hire a growth team. Okay? Let's go into innovation. We talked of doing, like, insane innovation. But who wins when we don't innovate? I'm gonna tell you who wins. Who wins is the big brand advertising networks. That's Google and that's Facebook. They drive around 70% of all the ad spend. So what that means is that you are transferring your hard, let's say, acquired VC money directly into their pockets, right? That is not innovation. That is not what your VC wants, okay? You're not building anything sustainable if you're just competing on Facebook on CAC. That is not a good strategy. That is not what your VC wants. And when you look at, let's say, the challenge that we have here is that the CAC has grown year after year, both on B2B and B2C, right? We're now at 70% higher CAC than we were 5 years ago. And that trend is not on a slowing down path. It's not gonna stop, right? Which means that if you think of like the ACV in the market, how much you sell is more or less stable. Your CAC is going up, what that means is you're gonna hit that kind of like sound barrier where you're gonna crash, right? 'Cause you're just not making profit from those leads anymore, okay? Cool. So the smart CEOs, what are they doing? The smart CEOs, they are transferring their budgets to technology. So if we look at this slide here, what we have is that all the CMOs are transferring their budget from martech, from agencies, to technology. Even labor is going down. What I wanted to explain is that the budgets, I have a graph from Gartner here. I'm gonna explain the graph, okay? So bear with me. So what you see is that there's like all those categories of spend, okay? And the spend or budget is going down in all verticals for the top 100 CMOs, but martech. Martech is the only one going up. Well, that means that the smart CMOs have understood that to compete, advertising is not a good solution, hiring more people is not a good solution, being smarter through martech is the solution, okay? What that comes with is actually engineering, all right? You can do stuff by hiring engineers in your team, in your marketing, demand gen, growth, however you want to call it, right? By having engineers to build products-like stuff, you can do stuff that you can't, I'd say, just with marketers. One thing that you got to think is like, we got to calm down our egos, all right? That's what we really would need. When you think about it, you're trying to compete with— most of you have competitors. Who has a direct competitor here on the product? Okay, so for those of you who have a direct competitor, even if those have like an indirect competitor, think about it. If your marketing team has like Markito and then does ads on Facebook, like how are you gonna win versus your competitors? You think you're just gonna have like better branding? You're gonna have like better ads? You're just gonna have better marketers? That is like a weak proposal. It's very weak. Okay? There's only two things. One, you can rely on product. So that means that your marketing isn't doing a lot. And so either rely on your product exclusively, but that means your marketing team isn't driving a lot of value.

Okay?

If you want them to drive value, it means they should be able to sell a product that is no better than your competitor. That is a good marketing team. That's how you should evaluate them, okay? And the thing is that growth strategies, growth tactics diminish over time, which means whatever we talk about today has a shelf life of like, if not a couple months, at least a year, all right? Because your first strategy, your first channel, eventually if that works, everyone goes to that channel and that strategy. And the benefit, the conversion rate up that you had actually goes down. To give you one example, you look at that ad banner at the bottom right here. That ad banner is the very first ad banner that ever existed. That is 1994 for Hotwire. That banner ad had a conversion rate of 74%. That's 74%. I've been doing marketing for the past 15 years. I've never seen something click for 74%, ever. Right? And so, yeah, that was a first mover strategy that worked well. And the same thing is true for video right now that we have in emails, or like all the videos and stuff like that. It works really well. Eventually, that's gonna diminish and also revert to the mean. So we talked of why you wanna create a growth team, innovation. When do you wanna build it? Easy, after product market fit. Okay? Once we have enough engineers that we can, I'd say, afford to have engineering dedicated to building demand generation, okay? And when the initial growth starts. For all the founders in the room here that are wondering, like, how should I structure the growth team? Who should they report to? I got a couple examples for you. So the first model is the cross-functional model, okay? And so that model has, I'd say, a growth team that is, like, kind of, like, peppered around your organization. All right? And as you can imagine, it doesn't work that well to create innovation. It works really well to, let's say, minimize the rifts and conflicts because there's basically no growth team. All right? But because the product and the engineering are incentivized to minimize risk, you don't have a team who is incentivized to take risk, so there's little innovation. The second model, is the independent model, the one from Facebook. Okay? That one has a dedicated growth team with a growth product that's completely separate. That creates really good innovation, fantastic innovation. It has the opposite problem. Lots of cultural clashes and rifts and conflicts. Why? Because you got people in product engineering and design and marketing owning basically your entire business. And now you have a team that is going to say, hey, we're going to work on that one thing. And we're going to own it, and we're going to hack with it. And then once we're finished playing, whomever owned it before can get it back. Obviously, that didn't work. I did that. I did that. I took the sign-up from the marketing and engineering team and said, hey, we're going to kill the conversions. We're going to be much better. When you do that, you are humiliating whoever owned that part of the product by taking it, by saying they did a poor job. And then there's only two outcomes. Either you succeed, so you humiliate them a second time, right, by showing that you were right and they actually did a poor job, or you fail and they're going to point the finger and say, you see, these folks in growth, they thought that they were really smart, they're not, right? So that doesn't go well. So that is like, that can work if you have a really clear charter throughout the organization that the growth team is here to experiment, fail, and help find stuff that is unique. Okay, if you don't empower that team through that very strong charter, you are setting them up for cultural failure. This model, the embedded model, is a very common one, and unfortunately we don't have the, the animation here, but basically what happens with this model, which I don't recommend, is that it's a power war where the growth org is going to like move from product to engineering to marketing depending on which exec has the most power with the founders at a given time. And you can see those teams are just moving back and forth, which means their KPIs, incentives, their strategy is basically aligned with who, which org they belong to, and not the entire organization. Which means the last model I really like, the one I've been implementing, is this one. So this one was created by my friend Sean Cross at Atlassian. It has this kind of mixed model, where you have the engineers and the designers reporting into their guilds, right, and a few core functions going to growth. This optimizes for both. And the real good reason why I like it, think like, I need engineers in my team. Am I an engineer? No. Can I help engineer grow in their career? Can I help engineer debug code? No, I can't. Which means that person needs help from the broader engineering team. From the broader product team, and same for designers, which means they need to be embedded in some way in those teams. So this model works really well. So we talked of when, we talked of how, let's talk of how I fail. Because my ability to fail defines me better than my ability to succeed. And to be honest, it's actually easy to reproduce failures. If you don't look at how other people have failed. It's really hard to reproduce successes, 'cause they're really unique. So when you think of a new role, and when I consult these days and I help CEOs, my first question is, what does the boss want? Okay, so boss is CEO, and so the boss wants good KPIs, okay, and a good roadmap, and basically not breaking their culture, and being reliable. To be honest, I've failed at all of those four, sometimes multiple of those. Okay? When you think of growth teams, most growth teams fail at being able to communicate what they're doing, why they're doing, and what impact they have on the bottom line. So let's start with KPIs. Okay? And KPIs, when you think of KPIs, it means like, Well, growth teams is here to build demand generation. So does marketing. And it ends up with something where you have like sales versus marketing, and it doesn't end well. Okay? So, if you think of sales, sales are incentivized to do a good job because they have a clawback clause. A clawback clause, for those who don't know, is basically that if you oversell to a customer and that customer churns within a a given amount of time, 3 months, 6 months, the commission is lost. Who in marketing has a clawback clause? No one, which means we can push horribly bad leads on the funnel and have no consequence, right? And that's why we have a riff and conflict with sales. We're not incentivized to long-term quality. We're not incentivized to LTV, right? We, as marketers, as Dimension people, as growth people, we should be incentivized Financially incentivized to long-term customer success, okay? The same is true for forecasting. When you look at how growth teams work, this is one example. This is a growth team tool to have all of your projects and your ideas and list which ones you're gonna work on and why. There's a couple of issues here. The first one is that it's a laundry list of ideas. The second one is there's no revenue, right? And so when the boss says, "Hey, G, what are you working on?" I say, "I'm going to work on that project because it has a 6.7." What does that mean? It means nothing. What are you committing on? Nothing really. And look at the other tools, the same thing. I asked a lot of my friends which tool they use. It's the same problem over and over again. There's no revenue. There's no commitment. You're not committed to revenue. You're not aligned with the sales team. Okay? I tried implementing that ICE, so impact, confidence, and ease, that ICE score 3 times. I failed 3 times, both with my team and with my CEOs. And the last thing is visibility and forecasting. Okay? Because you don't know what's going to work, okay, you can't give that visibility. You can't give the forecast. And so you have the, you have the problem of not being able to commit. You have the problem of not being able to forecast what you're gonna do and what impact it's gonna have. The new model I want to implement is trashing all KPIs, trashing the systems that are laundry lists of ideas, and moving to revenue. And if you want to move to revenue, this is how you do it. Okay, this is from PointNine's founder, Christoph Jentz. So they are a VC firm in Germany, and he has proposed this model where your entire funnel is just revenue, and revenue flows from stage to stage. Just as in sales, it's true in marketing. It means the maximum value you will get from your prospects is at the beginning. Each stage after that, you just destroy revenue by losing people, by not being at the most efficient that you could be. And so the revenue flows from one stage either to the next stage or to close lost. And so you start having that mindset that you could use the revenue as a metric for marketing even in the very early stages. And so you convert All of those KPIs such as new visitors and returning visitors into revenue metrics. You divide how much revenue has gone through that stage by the number of people, you get a revenue per unit object. Now you say, well, this is how much I have at each stage, and you can start having those metrics. I'm going to show you now how I have my actual dashboard. This is my dashboard for one of my companies. Okay. And that dashboard has the future value, okay, that's added to stock weighted by the quality. So what you have here is that the top, let's say, leftmost number here, which obviously is blurred out, it tells me how much I'm adding to my pipeline. Okay. And you gotta think of this like a shop. Okay. Just like a shop, we have revenue that comes in, revenue that is in stock, and revenue that goes out, either in close won or close lost. And each of those bubbles shows one of the stages, how much I'm adding, how much I have, how much I have closed, how much I have lost. And that is a marketing dashboard. And so you can run your growth team just like that. If you wanna think of your sales team best metric, which is sales velocity, which is the number of leads time the average contract value times conversion rate divided by time to conversion. The sales velocity is how you measure your sales team's efficiency to add revenue over time. You can do the same for growth, which is how much revenue, potential revenue, your growth team is adding over a period of time. Right? And you can do that at all stages of the funnel. Let's look at how we do it. You take the metrics. As I said, you convert visitors into Revenue, right? And so you track all your conversion points. You now have, let's say visitors is like, each visitor is worth $10. And then you have a new metric which is qualified visitors, and you're gonna have qualified signups, and so on and so on and so on. Then you have all of your projects, all of your ideas like this. Now you have all of the revenue, okay? The total per column, and you're gonna have all of the ideas, and you can say, at each stage of the funnel, I know exactly how much potential revenue I have. So here you have in the gray area, Impact Approved. You have the total value of all ideas at that stage. I know also how much time it's gonna take to work those ideas, okay? And so I can say, well, if it's gonna take us like 5 days to drive $100K, I know my value per day. And that's how we prioritize all the ideas. It's revenue per day. Now, one interesting thing is that, of course, we track week by week. Okay? And so we know the performance of the team on a revenue basis, right? And you can start having— there you go. You can start having stuff such as when the boss asks, "What project are we doing next?" You know, "Well, I know exactly." Why? Because those rank as the highest grossing per day of work. What impact are we going to have? I can commit on $270,000 out of 5.5 days of work. And that is highly predictive because it's based on past historical data. So now, just to nail the points, the questions we can answer is like, yeah, what are you working on? How do you prioritize? How do you forecast? If you wanna think about it, it means that I'm just like a sales team. I have a number of people, and those people bring revenue from the top to the bottom, some of which will be close one, some of which will be close last. It is marketing, but the organization is the same. And that's super important, okay? Because all CEOs that I have met understand, like, basically two things really well. Revenue. And sales team organization. Okay, the rest is like marketing-wise is muddy. And so instead of trying to change the mindset, I'm changing how I structure my team and how we report. By having marketers commit on future revenue, by having marketers actually structure the team as sales teams, you have amazing results. And one thing I wanna say is, I actually, I did that at Drift. We had engineers who had bonuses based on revenue per day achievements. OK? It's very important on the marketing demand gen side. OK? Just like salespeople. Why do I do that, and how does it work? Well, it's very easy. As I said, I have the VC model. I don't know what's going to work. My only impact is moving experiments faster through my pipeline. OK? Moving faster. Means that I want to be able to have my engineers actually code faster and ship experiments faster. I don't care if the code isn't great. So I'm trying to incentivize them to go faster, and money is a great way. They don't really care about the money, they care about winning, right? And so I have engineers winning with money to move things faster. And so that's what we have in the end, right? We have a process aligned on, I'd say, on the sales structure. We have predictable output, and we have a commitment on impact. And that's it. Thank you. I think we have a couple of minutes for questions. So the question is, what was the tool? So the tool is, uh, uh, it's built on Airtable. Um, there's actually a, um, a template of that Airtable, uh, that was shared, uh, by my friend Darius Contractor. Uh, the link I believe is, uh, bits.ly/airtable-evlyn. If you search on Google Airtable Evelyn, you're going to find it. It means experiment velocity engine. Yes? For your revenue projections for each step, how are you determining that? For example, the award reduced the friction. It was assigned maybe 5 figures. Yeah, how do I forecast the value? Yeah. Yeah, so unfortunately, we had the wrong slide deck here. But basically, I use a tool called MADQUIDU. So, they are like an AI machine learning company based in Mountain View, and they do like advanced regressions based on past data to predict the value of, let's say, leads at each stage of the funnel. And so I leverage their insights into my model, okay? And so it's pretty reliable. Other questions? Yes, we have questions back there. What are some examples that, for a typical SaaS company, that maybe engineers might create in order to help with growth? That's a good question. So I have a— if you look on internet, you'll find a couple of videos of me on tactics and strategies. Mostly what I try to do is to try and build products for acquisition that are tangent to your core product. That would push people to register low friction, drive value. See, my goal is to deliver as much value as possible to people with as little friction as possible. So a couple of examples. If people need to sign up to get the value, the question is, well, can you create, can you pre-create all the accounts, screenshots the value, and send it into an outbound email to prove the value to the future customer. I've done that once. It worked really, really well. I worked at a company called Mention. We're doing, like, news aggregation. And so we thought, well, news aggregation is really good for people who are just launching a product, okay? But they don't have the time to sign up and set up the tool when they're launching the product on Product Hunt on day one, right? So how can we help with that? Well, we pulled the API feed from Product Hunt, We created the news aggregation account for each of those launchers every day, and then we outbounded an email with a screenshot and a link with a token to, like, claim your account. Okay? Which means as a future customer, you have in an email the proof of the value, very little friction, just like click here and claim your account, and the data is already there. That obviously requires engineering, but it really helps, I'd say, the customer on the right path. That's one example. I've done stuff at Drift, for example. Drift is a live chat bot. And so the principle of live chat bot is to get your sales to move faster, people to engage with you faster. And so we did a tool as an acquisition that was meant to measure the response time of your sales team. So there was a website, which is now down, but there was a website called getmyresponse.com where VP of sales and sales leaders could go, sign up with the domain of the email, and we would have contractors go and automatically request a demo with a generated email, and we would measure the time between the demo request and the demo follow-up in hours, and we then measure that against the industry standards to tell them how long they took and how much conversion they're losing. And then the VP of Sales can say, "Damn, we're losing opportunities. We're not converting as high as we should." And the obvious follow-up is like, "Maybe you need a live chat." But it's tangents, right? So a lot of those things require some level of engineering, and it's much more than just building a landing page, right? It usually requires some level of like backend engineering and stuff like that, but it creates value. I'm trying to use engineers to push value up the funnel. That I think is the core strategy. It's push value up the funnel.

Cool.

Yes?

I was wondering if you could quickly elaborate on how you compensate the engineers for hitting the certain growth targets?

How I compensate? Yeah, is it like a commission or a flat rate? Just what's worked for you? Yeah, so basically the target is to have them produce at least $1,000 per day worth of experiments. The $1,000 mark is there because it's more or less what they cost in San Francisco. You know, like 200 days of, 220 days of work, you know, 150, 180Ks cost plus, like, bonus and whatnot, it's like, they're more or less the same. Um, and so can they add $1,000 per day worth, uh, of value to, to our pipeline? Okay, and if they do that, they get bonuses. Now, if you think of the mind, what's gonna happen? Okay, so basically they want to win. There's this laundry list of ideas that are rated, uh, on a scale by, uh, experiment value per day. Projections of how valuable is that idea. Obviously they all want the one at the top. Okay, so how do they compete for the best experiment idea? Well, someone's gonna say, "Well, I can do this project in 5 days," and the other one's gonna say, "I can do it in 3 days," and I'm gonna say, "Commit," right? And so now they're committing to going faster, and it's forcing them to make some shortcuts and to work extra hard, which is exactly what I want because I don't care about the code quality at that stage of the process. I care about having the answer really fast, right? Because we're trashing— remember, we're trashing 9 out of 10 ideas. So we can take care of code quality later, okay? So that's how I— that's how we do it.

Smaller stage companies, it sounds like— and I'm sorry, I couldn't quite see the slide there, but it sounded like you were saying dedicated people for this kind of like experimental team. What about smaller stage companies that can't really dedicate a 4-person team across these different guilds? And then a follow-up to that, how do you think about reporting and who's driving that? Who's typically in that seat?

Yeah, it's a good question. So at a small stage, it might not make sense. You might not need a growth team as a separate function. You probably don't, actually, right? Because the question is, have you hit product market fit? And have you exhausted the obvious channels? If you haven't exhausted the obvious channels through obvious marketing strategies, you don't need to experiment. You should just execute on those first channels. Okay? The need for growth team comes when you have exhausted the obvious things and you're getting, like, you're getting trouble continuing on the same growth, I'd say, path, in terms of, like, month-over-month growth, and your CAC is going up. Now you need to experiment. That's the first thing. The second thing is, For me, if you really want growth as an experimentation engine to figure out stuff that is not obvious, you need two things. One, you need a very clear charter throughout the organization that this team is taking risks and they're going to fail often. And two, that team needs to report to the CEO, okay? Because stuff's going to break and stuff's going to fail and they need air cover, right? Otherwise, that team is not going to survive long. Cool. Thank you very much.