Sean Ellis @ GrowthHackers Building a Company Wide Growth Culture

So we're going to go right into the next session about building a company wide growth culture. I'm Alison Wagenfeld, operating partner at Emergence Capital, a venture firm focused on enterprise cloud companies. And now I'm pleased to welcome Sean Ellis, who's founder and CEO of Growth Hackers. So he will share with you some background about himself. He had great roles in companies that you'll know well, like Dropbox. And now starting his own company to share a lot of the learnings that he's had over the years. He's going to take you through some slides, then I'm going to ask him some questions and then we'll open it up again for the last five to seven minutes on questions from all of you. So with that, I'm going to turn it over.

Thanks, Alison. So, hi everyone. So, yeah, I'm going to give a quick introduction to building a company wide growth culture and then we'll go into some of the challenges of doing that with Alison following that. So everyone here who isn't talking is looking to build a growth engine that gives them explosive growth. Of course, that's something that we all want. It's actually gotten a lot harder in recent years than people realize. One of the big reasons that it's gotten harder is that 10 years ago when I was at LogMeIn running the marketing team, we were able to spend millions of dollars a month driving growth, but there was about a third or less of the money chasing each person online compared to how it is today. So we could invest money in growth, get a fast return on investment, keep cycling that money, build a team around primarily external customer acquisition. And it worked pretty well. But over this period of time, you've got a lot more dollars pouring in online. And at the same time, you've got this just massive fluctuation in channels. And so the things that work today are probably not going to work in another month or two. So for growthhackers.com for example, one of our big growth channels is Twitter. And this weekend there was an announcement that they're going to start to introduce an algorithm to Twitter, which is going to change that channel. There's probably going to be new opportunities that emerge from that, but there's going to be new challenges as well. And so ultimately you've got to just. Growth has become a lot more challenging. And to be effective with it, you've got to start to bring in your entire company into the growth process. So instead of just looking externally at driving growth, you want to look across all of the Key levers of growth. So Dave McClure has what he calls pirate metrics, which is an R framework that goes from acquisition or all the way down to activation, retention, revenue referral, and improvement in any one of those areas is gonna accelerate your growth trajectory. And the challenge is that each of those areas generally fall within different groups within the company. So a lot of those are gonna be product driven levers and they're gonna sit in the product team. Customer support's gonna have a lot of impact on activation. And sales obviously affects growth, business development does. And so the challenge is coordinating those efforts across all of those teams. And that's where a growth team comes in. Who really helps to coordinate those efforts to be successful. What you need. There's really key, three key things that you need to be successful in driving growth across a full team. First, you need broad participation, which is not easy. I'll go into a couple things to help with that. Then you need a weekly cadence or maybe every other week cadence, but you need to have a pretty good cadence of testing. And then finally, you need transparent learning that you can communicate out to the rest of the company as you figure out what works and what doesn't work. So let's look at broad participation first. This is not just within your company that you're driving participation. So this is actually, as I said, all of those levers are controlled by different people within the company. You want ideas from people who have unique insights from whether it's the support team. Engineers are going to have some insights. I'll show you an example in a little bit of a test idea that came from one of our engineers, basically all your internal people. But then externally there's going to be. You may not have a person on the team who really understands how to leverage Pinterest, but Pinterest might be a great growth channel for you. So you want to be able to work with outsiders and bring them into the process so that as you explore some of these channels, you can at least use an expert to explore them. If it works, then you can figure out, do I hire somebody? Do I continue to work with outsiders? But you constantly want to get as much broad participation as you can. And your goal is to build a big backlog of growth ideas. So my team, actually, we just crossed 700 ideas in our backlog, untested ideas. And I'll talk about some of the benefits of having a big backlog in a minute. But ultimately what you want to do is take each of those ideas rather than just have something that's written on a scrap of paper. You want to actually turn it into an experiment doc. That includes things like your hypothesis. Why do you think that idea is going to work? Do you have supporting data that says that that hypothesis is likely to come true? Or is it a total guess? Sometimes a total guess actually is a good thing and works out well. Target lever, where is it focused? Is it a specific channel? Is it deeper within the product? And then you want to be able to score each idea. So we use something that we call an ICE score, but some people use a pie score. But ultimately for us, the ICE score is looking at it across a few different dimensions. So impact. If this thing works, is it going to be really high impact or is it something that's just going to move the needle a little bit? Confidence. Do I have a lot of data supporting this? And does it give me a high level of confidence? And then ease? Is it something that's relatively easy to test, or is it going to take weeks of engineering just to figure out if it works? And the ideal experiment is going to be something that scores really well across all of those. So scoring on kind of a scale of 1 to 10, 10 being something that's really good. If it's 10 impact, 10 confidence, and 10 ease, that's probably a really good test to run. So this is just an example. As you start to think about what's the realm of ideas out there, you can find inspiration anywhere. So my time at Dropbox, when I was there, we only had one thing. So when you look at your computer, there's a little Dropbox icon up in the top of the computer. If you have Dropbox, which most of you probably do. And when you hover over that, originally it was only giving you a shortcut to your recently saved files. But somebody had the good idea of making a share link when you hovered over it. There's a whole onboarding with optimization that was able to happen through that share link. And then over time someone else said this is a great place to merchandise different ways to drive retention or additional monetization. So testing a message to get people to start saving their photos to Dropbox, and that's going to get them better lock in and they're going to hit the limit faster and need to upgrade and then there's just a pure upgrade button. So over time they've added a lot of things. And as you look across the web, you'll start to see other tests that people are running. LinkedIn has a ton of them, for example, but just across these different sites, you'll start to get ideas and some of them are going to be relevant for you, your business as well. So once you have a big backlog of ideas, then you want to look at running a weekly cadence of testing around those ideas. And that's where your growth team comes in. So growth team should start with a growth master. A lot of times people think growth team. If you don't already have one, then you think, okay, I got to go out and recruit and hire a growth team and maybe plan about six months to get them together and then we'll be able to hit the ground in and really start doing a lot of testing. But that's gonna stall a lot of what you could be doing starting right away. So at the very least, you just start with a growth master, somebody sometimes called a product manager of growth or head of growth, but someone who's coordinating the efforts, and then make the rest of the team ad hoc. And so people on the ad hoc team might include some designers, some engineers or an engineer, analysts. And then at my company, we also have a lot of the execs. So head of product shows up to our weekly growth meeting. Head of engineering shows up to our weekly growth meeting. I show up to the weekly growth meeting. And the purpose of the team is to really manage the testing and learning that happens. So here's just a quick look at what our weekly growth agenda looks like. We start out with some of the high level. What are the growth metrics? Where do we see issues? Where do we see opportunities for growth? Review the last week's testing Sprint. And this is where the ad hoc piece comes in nicely, that if you're finding that you had a bunch of things you planned to launch but you couldn't get design resources, it's probably a pretty good argument to go and have a dedicated growth designer or you couldn't get engineering resources, get a dedicated growth engineer and then just continue to be responsive. A key part of the meeting is anything that you've tested and it's run long enough and you can now start to analyze the results, capture that learning and talk about that learning. And that learning is going to be really important for informing future tests. This is also when you plan out the test that you're going to be doing the next week. And so my team, we have everyone nominate a couple of ideas. They kind of pitch the ideas on a one minute pitch to the rest of the team and then we shortlist it down to three to five ideas that we're going to test each week. And then finally we check and make sure that the backlog is continuing to grow. So the goal of the growth team is to test at a high tempo. So you're testing across each of the vectors that we've talked about that are framework running those tests. The more testing you do, the more learning you're going to get. And it's all about that learning to figure out what's going to work and what's not going to work in the business. So a good example of that is Twitter. We all know that Twitter's had some growth problems recently, and that's probably part of the reason why they're open to the idea of adding an algorithm to it. But it's not actually the first time that Twitter had growth challenges in late 2010. They had almost a flat quarter in late 2010, and at that time they were running less than one test per week. And brought in a new head of growth who pushed to get that testing frequency up. They got it to 10 tests per week, and you can see they had years of consistent growth following getting it up to those 10 tests per week. So it really does come down to testing. You obviously want to be smart about what you're testing. So we talked about the ICE score earlier. You can use that to start to prioritize, to figure out what are those things that score well across impact, confidence and ease. Some of the areas where you see opportunities to grow, you can kind of drill into those areas and try to pick, as you're nominating the test to run that week, try to pick some that are scoring well and then manage the testing. Really important, Appoint a product manager or a project manager to each test. So if it doesn't get launched, you got somebody to be able to describe why it didn't get launched. It could be anyone from the growth team. Ideally, you're going to want to have just somebody who actually either nominated that test or came up with it in the first place, because they'll be at least passionate to see it go live and really want to see it work. And then you want to run the test for enough time to actually see if they worked. And so you want to follow the testing best practices where you can find a lot of stuff online about that. But don't run it from a Monday to a Wednesday and compare it to something that ran from a Thursday to a Saturday. You want to run for full weeks, get a sufficient sample size. But there's a lot that's written about that, so I won't go into the details. And then finally, analyze the results. So always ask why, if a test did not perform as expected, don't just accept that. Figure out why. Figure out did you have a clear hypothesis? And if it didn't deliver on that hypothesis, try to understand what happened. And if you can understand what happened, you can generally inform much better the test that you're going to do after that. And then once you've, once you find stuff that works, you want to make sure that you actually start to act on that information, get those as part of your growth engine. So if it means hiring someone to manage that channel, if you can automate that channel, whatever it is, you just want to have it in the mix and continue to monitor how effectively it's working. And sometimes you need to run it as a separate test again a few months later to see if it's still working, because again, those channels are in flux. And then the last piece, transparent learning. You've run the analysis, you've run the testing. If you don't share that learning externally with the rest of the team, you're gonna have a really hard time getting the rest of the team to submit ideas because they don't know what's been tested and what's not been tested. So it's not just about sharing the wins, but it's also sharing the things that didn't work. A lot of times the things that didn't work are what lead to the big breakthrough ideas that end up working really well. And so gotta communicate that, make that available to everybody. Here's just an example of a test that we ran on our team based on sharing some information externally. This is really data that showed us that when we had media embedded on the site, that led to longer sessions. It's pretty intuitive. You could have figured that out. But essentially growth hackers is linking often to resources that will help you grow. But there's a lot of great YouTube videos and SlideShares that there's no reason to link off to SlideShare or YouTube. Those properties are designed for media that can be embedded to where we can then recommend other content that's related to that host discussions around it, kind of take it to the next level. And our challenge was that we had to actually do that through our admin, set that information up in there, and we missed a lot of them. But when we presented it to one of the engineers, he said, oh, there's the thing called oembed. And with oembed, we can automate that. So as soon as our system Detects it's a YouTube URL or a SlideShare URL, it'll automatically pull that media in. And that led to 41% more embeds, which led to longer sessions. So again, ideas can come from anywhere within the organization. So this is the last point that I have, and then we'll have some conversation. So it's really easy to get super process oriented or super numbers oriented and kind of lose track of the purpose for what you're doing as you try to build a growth culture. Ultimately what you're trying to do is that the last session was all about product market fit. And product market fit is about creating value for people. And once you've created that value, you need to deliver that value. You got to find the right people for that and get the right people experiencing the, the product in the right way. And as you obsess on that, and as you get the rest of the team to obsess on that, it's a lot easier for them to want to participate in the process that they, they're not going to look at it as, oh, that's just like spammy marketing stuff. But instead it's about really figuring out how do we connect with the right people, how do we clear away the friction that prevents them from having a great experience with the product, how do we set the right expectations and drive momentum and bring them in? And so that obsession on value will get a lot more people interested in actively using the product and get the rest of your team actively participating in the process. So that's it for some slides and then we'll have some conversation around it.

Great. So thank you. So that was an outstanding framework to think about growth hacking. And I guess when I was reading about you, it looks like you're one of the people who coined the growth hacking term.

With Heaton, actually. Yeah.

So with Heaton. Okay. I didn't realize it was all coming full circle here. So I want to dig into some of the more, like tactical questions around this. Is there a certain size that a company needs to be before you layer this team in?

Yeah. So what's interesting, when I was at Dropbox, I think we were like eight or 10 people. And so at that point, again, it's less about the team or like an external team being layered on there and more about everybody at that point was part of that growth team. By having the early people see some experiments that we were running. Once they saw some wins from those experiments, it got contagious and everybody started quantifying code a lot more and quantifying features and it relates to lean startup type things, but it also is just, it gets addictive once you start to see those numbers. So I think early on you may not need a big team, but this ad hoc approach can work with a pretty small team.

Small team. And is there like certain budget that was allocated or did some of the companies you didn't have budget allocated to run these experiments or was the budget coming from each of the kind of functional areas or elsewhere?

Yeah, most of the stuff that ended up working for Dropbox for example, was not, not dollar driven. And we tested a little bit around AdWords and some things, but ultimately the stuff that worked really well were just natural sharing in the product and incentivized referral program where we gave away free space which wasn't free for us, but ultimately. So there is costs associated with it, but wasn't necessarily budget, it was performance based. If it didn't work, you they didn't do it.

Yeah, makes sense. And it looks like one of the key elements, I mean the fact that you have kind of 500 kind of inputs, you must have some kind of system that everyone's like tactically, where are people putting their ideas? How are they getting put in there? How does everybody even know in a company to submit ideas?

Yeah, the systems are evolving. So I think probably the majority of companies today are still using spreadsheets and Google Docs and kind of a hodgepodge of things. Trello is getting better for it. And then my team is actually trying to build a system. And so some of the things that you saw there is a private beta product that we're working on.

Okay. Yeah, because I couldn't recognize what that product was. Okay, well then you can help sell your product now. And what are some times that things haven't worked? I imagine that you've tried not just experiments, but kind of culturally in a company there's probably some aversion to this until you've kind of spread the gospel a little bit. Walk us through some times where that hasn't worked as well and what you've done to overcome any opposition.

So I think one of the things that tends to throw off growth testing is big projects. So whether it's like in my own company, when we were convert, like we started super MVP on like a WordPress hosted version of growth hackers and then we rebuilt it all in Ruby and so you're suddenly like, oh, we won't do testing until we get this thing back out. And then there was a redesign and like, oh, we won't do Testing until we get this back out. And it's really easy to break that rhythm. But every company I talk to, it's when the rhythm breaks, it's hard to start it back up because everybody has big product initiatives. So they start pulling the growth engineers into big product initiatives. And it's, you know, marketing and growth tends to be a proactive thing rather than a reactive thing. And so it's easy to take something away from proactive. But at the end of the day, if you're not growing, you're not building a valuable company. And so it takes a lot of discipline to continue to invest and drive the process.

So if the CEO is not a champion, can this still happen? Growth hacking, or does it really have to come from the very top?

I think it has to come from the very top. The CEO. I mean, what's the interesting thing that I found is that the CEOs that are really good participants in the process are they don't want to just throw out ideas because they know their ideas may not be as good as other people's. So if there's not a good way of scoring them, like the ICE scoring system, it's just basically, the CEO said we should test it. Drop everything and go test this. And by having a systematic process of nominations and kind of being able to hone in on what are the best opportunities based on what the data says, it's a way for the CEO to be able to participate and actually not dominate the process, but be an active participant in it.

And you talked about the concept of a growth master, that you need somebody who's kind of running the meetings. How would you pick for those people here that want to kind of set up a growth master in their company, what are some of the characteristics of that person? What function might they be in? How long should they have been at the company? What's been successful in your experience?

I think the most successful ones tend to come out of product. Organizations tend to have a lot of really good project management experience. They need to lead the growth meeting, so they've got to have some authority, a little bit, or some level of being able to manage that. A lot of them have pretty good analytics background. Like ours is a woman that we hired out of a bigger company, Broadcom. But she was. She was used to navigating around challenges in a bigger organization. And so she's been really good for managing the process with us. But I think the biggest thing is someone who's dedicated to the process and just is going to be relentless about keeping it on track.

Yep, that makes sense. So I want to make sure that we get time to get some questions from the group. Anybody have any questions right now? Go right ahead there. Yeah.

Yeah. Hi. My question centers around what type of experiments actually to run at the moment. I'm struggling. We have a growth team but I'm struggling on the experiments we do with paid acquisition channels or organic acquisition channels versus the experiments we do in product with the flow. Have you got any like percentages? I don't want to create, create a culture where we're just over experimenting, to be honest with you. Have you got any advice on that? Yeah. So in kind of picking the area to experiment within, it does change company by company. So if you've done almost all external testing then you probably have a lot of low hanging fruit around retention and activation. But the challenge with retention and activation experiments is that generally you can only do one at a time or you start creating too much noise around the data. Where acquisition experiments you can run 20, 30 on top of each other and it's not going to mess things up. And acquisition experiments, you can often see your ROI much more quickly. So you don't need to be quite as deliberate about running between these days and that sort of thing. And you can also, if you end it early, you can still, you can model out your return on investment, but then you can go back and check it maybe a month later and see, okay, once those people in a SaaS product, if you've got a long sales conversion cycle. Did we get the ROI that we thought we were going to get based on the early models of getting people to a certain point in a trial, say, but three months later, did those people convert at a value that made that a cost effective channel for us.

Right over there? Yep. We'll go back to you.

Okay, sorry. Hi there. Jordan H. Franklin from Grover Learning. I wanted to ask a question about your framework for prioritizing your tests. I see that most of it's focused on upside. And what are we going to get out of this test? How will it benefit our users? How do you assess risk and downside of what's the worst that could happen with this sort of test? Yeah, I mean that's a really good question. And essentially it's not all about the upside, particularly when it comes to pop ups and things that can kind of start to create a bit of a spammy experience. And so I think that's why it's really good to have your VP of product in the meetings, have your CEO in the meetings. That your growth master should be more aggressive than your head of product and your CEO, your growth master. It's like there should be a little bit of friction there and a time where people sort of say, hey, that's crossing a line. And then part of that is in the analysis then is sort of saying, well, I think the risk here is that people aren't going to be coming back. Maybe we get a much higher registration rate, but we're ruining that first user experience as a result. And so the downside is I don't think they're going to be coming back at a good rate, but the data will tell you that. And that's something that we specifically in my company looked at recently was an experiment that was more about a wall bringing people in. But the calling, the success based on what does the six week retention cohort look like on people who hit that on their first experience?

Got it. All right, we can go back over.

Ishan, big fan of what you and Anuj and Hilya do at Growth Hackers. Thank you. Quick question. My forte is top of the funnel, getting people to the site, getting people who are qualified, potentially going to convert. Where have you seen some of your biggest wins come? Is it top of the funnel, middle of the funnel as you're warming them up, or is it closer to getting them to conversion in terms of roi, where have you you and your tests? Yeah, so I'm really glad you asked that question because it is interdependent and I'll give you a specific example from LogMeIn. So we had, we early on, in the early days were able to scale cost effectively to about $10,000 a month in spending. But when we actually looked at, all right, we got them to the site, we got them to sign up, but we had so much friction in actually getting them to do a remote control session and start using the free product that if they never use it, there's going to be no conversion to premium. And so we basically, we pretty quickly said we're going to hit a wall on acquisition unless we figure out the deeper funnel challenges. And so we actually hit a pause button on customer acquisition. Spent four months of documenting the funnel, testing like crazy everywhere within the funnel and like qualitative feedback. When people abandoned at this stage, why did they abandon use that information to drive a bunch of experiments. We got about 1000% increase in the percentage of people who signed up and actually used the product in a four month period. And what that did externally to channels is that it took the same set of Channels that previously scaled to $10,000. When we went back and tested everything again, it scaled to a million dollars with a three month payback on marketing investment. So there's a ton of interdependence on on that.

So let's hope everybody else can enjoy that same kind of success. Awesome. Any other questions? Are there any specific tools? I mean given that we're in the tactical stage, specific tools, features, products that you've been leveraging within your website or others that you've relied on?

Yeah, so particularly around the testing itself. It used to be back when I was logmein, what took four months to do a lot of that was pretty challenging. Launching each test with something like Optimizely where I relied on a team of three or four people that logged me in. I can do it myself now with Optimizely unbounce the same sort of thing. So page tests can be much easier. And then usually those aren't necessarily analytics products, they're they're testing facilitation products. If you kind of think that way with light analytics, but then being able to integrate into something like a kissmetrics or mix panel, that's more of an analytics specialist where you can get much better retention cohorts tied to that experiment over time. So yeah, there's a whole stack but I think it's the testing and the analytics are the two big areas that. And then all the stuff around email is well, so mailchimp for a lot of like AB testing and then even marketing automation for being able to then automate and manage programs once you figure out what works and what doesn't work.

And so maybe as a final question for those of people who are here are going to go institute this in their own company, what is kind of an expected period of time that it might take to kind of get it up and going if you can, once you identify the growth master, to actually start seeing some results.

So what I like to do, if you're just getting started with it, I like to script my first 10 experiments basically so that if you're deliberate enough about what the first 10 experiments are, the likelihood that you get enough wins that you get kind of organizational buy in, get the CEO and everybody else saying that, that it was worth the effort makes sense. And so like going in and doing the user testing and doing the feedback when people abandon at different points within your funnel and finding out how do they discover a product like yours and just getting some of that data up front to being able to be really deliberate about your first 10 tests. And then based on those first 10 tests, you'll find that one of them's going to way outperform the others. And then that's going to be a vein of gold in which you want to keep doubling down and doing a lot of other tests like that. But I can usually script and run the first 10 Tests within 30 days for a business.

Okay, great. That's excellent. Well, we are out of time. Thank you so much, Sean. Round of applause for.