How to Break Out and Really Scale

Oh, yeah. Cool. All right. How's everyone doing today? Awesome. Well, you guys are in for a real treat here. We've got Dan Soroka, the founder and CEO of Optimizely, and we've got a great session here coming up. By way of introduction, my name is Ajay Aggarwal and I'm a partner with Bain Capital Ventures. We're an early stage venture fund that focuses Almost exclusively on B2B software, both infrastructure as well as applications. And many people, when they hear the name Bain, they assume we're later stage guys. But in fact, the majority of what we invest in is early stage Series A and Series B. And we have offices out here in San Francisco and Palo Alto, as well as the east coast in New York and Boston. So it's been great to meet many of you here over the last couple of days. I had the pleasure of meeting Dan for the first time, I think, in 2012 when Optimize was raising their seed round. And then we had the good fortune of getting formally involved as part of the A in 2013. And it's been a real pleasure to be part of this rocket ship over the last four years. So excited to get into it. Dan, maybe a good place to start is introduce yourself and perhaps tell the crowd some of the inspiration behind why you started Optimizely.

Sure. Well, thanks for being here and can we give the band a round of applause? They've been great all day. All right. Well, I used to work at Google as a product manager in 2007, and in that year, I snuck in to see Barack Obama speak when he came there and he gave this talk about how he wanted to take what we're doing at Google with evidence and science and feedback and data and bring that to the government. So inspired me to leave my job. Two weeks later, I flew to Chicago and signed up as a volunteer for his campaign. And that eventually turned into a job as the director of analytics, where they tasked me with figuring out how to use data to help make better decisions, which led to AB testing. And at the time, we used products from Adobe and from Google, and we saw the power of a B testing. We also saw the pain. It really required a developer to be part of the process. And so the original inspiration for starting Optimizely was to try to make it easy for anybody to do AB testing. And we've done that. So in 2012, we became the number one most adopted website optimization platform in the world. We actually had Obama and Romney using us extensively, which was great. And now We've grown substantially since then to do much more than just AB testing and really help organizations use data to help make better decisions from testing to personalization to really optimizing the experience they give to each of their customers.

That's great. And so Dan, I'm curious, you know, when you started the company back during the Obama days, or at least had the inspiration in the Obama days, how did you think about the fact that, you know, in this space you had this giant competitor called Adobe, you had IBM in the space, Google had a free product of some kind of. And as a founder, why that space? You know, on the surface seems crowded, seems like it's already been done. What caused you to think there was an opportunity there?

Well, for us the key initially was we deeply understood the problem. And I think in almost any software business, the ones who build the best solution are the ones who deeply understand the problem. Having been the practitioner in the Obama campaign in 2008, I saw firsthand all of the ways that I would do things differently if I were a vendor trying to enable us to do testing. So that approach and that sort of perspective was invaluable and that helped us really focus in on what to do and more importantly what not to do. In the early days we didn't have features that people assumed were must haves, things like multivariate testing. We didn't have that for a year and a half. But people found that the key differentiator of ease of use and enabling non technical business users really resonated and the ability to enable mere marketing mortals to drive the process of experimentation was really a catalyst to our growth. So you know, with respect to Adobe, when we saw they had a product in the market that sort of emerged from an acquisition through an acquisition that was originally Affirmatica, then turned into Omniture and eventually into the Adobe marketing cloud, they were great at sales and marketing and so we said, all right, we'll let them be great at that, we're going to be great at the product. And so we took a differentiated approach and then now over time has sort of hollowed them out from within where the product is actually the thing that that helps us win.

And I'm curious, you know, as you thought about differentiate on product and you mentioned the fact that, you know Adobe's great sales and marketing, you know, I think my understanding of their product back then was huge teams of it required to customize it, to implement it, to run the tests. How did you think about designing this In a way that was truly Democratic. I mean, you know, one of the hallmarks of Optimize is the fact that a marketer who's not technical can start using the product. And when did you sort of figure out that that was the real insight behind getting this business off the ground?

Well, I think that the opportunity was always there. We did not invent AB testing. AB testing is a good idea. We just made it easy for anybody to do. And so that catalyst of the enablement of the marketer, I think is what really helped. Adobe has a product that has been evolved through acquisitions, which turns out what they're offering into what our customers tend to describe as a Franken suite. It's a bunch of products that don't really integrate well with one another. And for our customers, who even these large enterprises, what they want are products that just work. They want something that abstracts complexity from their business and it's easy. And so the ability to enable that practitioner and then grow from there within the entire team, I think is what's helping us win.

So you guys have actual marketers using the product every day, no IT involvement. They're making changes, they're running tests and they're going ahead and implementing those tests into production.

Absolutely, yeah, we know. Great example is IBM. They're one of our best customers. And what's exciting for IBM, it's a company that's been around for over 100 years. I think they have done amazing things in transforming their business every decade. And today I think they've really embraced this idea of that. In order for them to be successful in the B2B world, they need to enable their customers from a marketing perspective, to interact much sooner than the sales process. They've seen 70% of the decision gets made before salesperson is involved. So they use Optimizely. They've got a team of hundreds of people doing AB testing optimization every single day that don't require developers to be involved in the process.

Very cool. Now, one of the interesting things about Optimizely, you know, Dan raised a Series A when I think you were already $10 million run rate. He went from a million to 10 million on almost no capital. You know, we're entering an environment where funding is potentially going to be harder to get. How did you get to that kind of scale one so quickly and also with so little capital?

Yeah, well, I think, you know, looking back at it, we had a lot of benefits that enabled us to get tremendous leverage in the business early on. The key to focusing on the product and making a product that people wanted to talk about and share with their colleagues. And the fact that marketers tend to talk to other marketers about best of breed solutions meant that we had this tremendous growth through evangelism. And earlier today, the box panel, they talked about how important it was to turn your best customers into evangelists. That's a very cost effective way to get word of mouth marketing and grow. And I'll say honestly, some of our biggest weaknesses early on became some of our biggest strengths. We were not great at marketing early on. I remember my co founder and I, we would sort of sheepishly set ourselves goals to like write one blog post a quarter a year later. This may resonate with some folks. A year later we look at our blog and realize it's been a year since we posted a blog post. And so we weren't great at doing sort of that outward kind of marketing. And so the product had to be the real marketing channel. That's what helped us grow in those early days. And now that's catalyzed what we were.

Able to do today makes sense. And you guys really early on cracked the code on a way for your customers to try the product before they had to spend any money, which was unheard of in the testing space. And so day one, the customer came to your website and they could actually see the power of it and the real promise of optimizing that, hey, a marketer can go do this on their own was right there, you know, for them to see, you know, before they even purchase it. Maybe talk to the crowd about how you guys pull that off. I mean, I think it's a really powerful way for you to acquire customers.

So I think it fits very well with our competitive differentiation against Adobe. Our initial tagline when we started the company was AB testing you'll actually use. Which was a reflection of the experience that firsthand I had of using the products from Google and Adobe and seeing so many businesses who knew that AB testing was a good idea, they just weren't doing it. So that philosophy is what translated into the early product experience for Optimizely, which is essentially just a homepage. You type in any URL, it loads into our visual editor. If you haven't implemented optimizely, we would actually proxy your website and inject Optimizely into it so that you get that sort of visceral, magical feeling of making a change to your website without requiring a developer. That was kind of the earlier we talked a lot about crack and cocaine. I think that was the original drug that got people hooked up and got them to become our biggest evangelists within the organization. Because now the marketer says, look, you don't have to worry about doing code pushes to make changes for these A B tests. Empower me as the marketer to run the experiments, to run the A B tests, so that the engineers can focus on what they're great at. And that's really, I think that focus on usability and getting in without having to go through a sales process really helped.

That's awesome. And so when you went out for your a round in 2013, you know, clearly you and Pete saw something where you said, okay, we're, you know, we're feeling escape velocity here. What were the signs to you that, you know, we have a breakout opportunity here and it's time to, you know, put fuel in the tank.

Part of it was we started to see that this wasn't just a small business thing. You know, I think we early on, I remember when we first started thinking about our pricing. We had put the precious metals up on our website, bronze, silver, gold, platinum. And we thought, should we put $2,000 a month on the website? No, no one's ever going to pay $2,000 a month for this.

That's right.

And so we held back from that. And then every month since then, customers have paid us more and more as they get more and more value from the product. And so it really validated to us that this is not just a small business thing. This is also not just a lone wolf thing. This is the whole team at large enterprises like IBM who can adopt this and really transform their business into being much more data driven. So that really to us said, well, now it's time for us to really invest in our growth, invest in the future of the business. And that's why we raised the money.

And you guys also, from early days, built the product in a way to truly be horizontal. Right. Did you try and target certain verticals or did you really try and say it doesn't matter? Anyone with a website, anyone running any kind of business on a website needs optimize it. That can be a commerce company, a media company, what have you. Certainly other startups in your space went more vertical. Was that a conscious decision or is that just where the market took you to go horizontal?

I would say it was an unconscious decision and partly it was due to, you know, we didn't do a great job of deeply understanding the vertical specific problems. I think we've changed a lot since then. If I had to give myself advice to a past version of Myself, I think a lot about that. And when I think about optimization, I think that it would have actually served us better to be more narrowly focused on a set of customers and deeply empathize with their biggest problems. Not just one vertical or one size, but take big, meaningful verticals for us like B2B companies, like retail, travel, media companies, and understand what are their biggest problems. How can we as optimizely solve them in scalable ways that then generalize the broader market? And so that's something we've embraced this year, is a real, much more deep empathy to a set of customers. And hopefully the solutions we build for them will be broadly applicable to the rest of the market.

Makes sense. So you raise your Series A, you now had capital in the bank, you had a business, you had a machine that was working, went from one to $10 million with almost no capital. How did you make decisions on how fast to grow, how hard to hit the, the accelerator? How did you know? You know you're in this rocket, you know, the fuselage is starting to shake, you know, maybe, you know, part of one of the rockets has disintegrated. You're going so fast, at what point do you realize, shit, we're going too fast, you know this rocket's going to explode and how hard do you push it? And how did you think about some of those trade offs, particularly when you didn't know what you didn't know way back when?

Well, I didn't know what I didn't know now. Now I know a little bit more. So let me try to share some of that wisdom.

That's right.

I don't think we got this right from day one. I think I remember a day when we started to describe ourselves as a rocket ship. And it always made me feel a little uncomfortable because I thought it really focused on the wrong things. It focused on growth at any cost. And the truth is growth has a cost. It has a cost to your culture, it has a cost to holding true to your values and really thinking through the right decisions as opposed to the decisions for right now. And so I initially was pretty hesitant to describe ourselves as a rocket ship. But I think in the last year I reflected on what a rocket ship can uniquely do. Well, we talk a lot about metaphors and symbols, so I'll explain why I love actually a rocket ship is a right way to describe a startup. A rocket ship doesn't necessarily have to just go fast. We at Optimizely, we want to go far, not just fast. And we think a rocket Ship's a great metaphor for that. So it's a question of how do we build a foundation today to build a business that can hopefully one day outlive us? And that's the kind of question we ask ourselves when we make trade offs between what's right for maybe hitting a goal today versus what's right for our customers in the long term. That foundation, I think, has served us really well. But I had to give myself advice early on. I would say be really thoughtful. Make it a conscious commitment, not an unconscious commitment, to grow at a pace that enables you to build an enduring company, not just one of the top, fastest growing companies of the year.

Are there examples when you look back, are there certain things or stories you can share where you said we did this and in hindsight it didn't adhere to that value or we could have grown faster, but we chose not to in order to be a rocket that goes far as opposed to just fast. Any stories you can share?

Sure. I think the biggest is probably around hiring. I think I interviewed the first hundred people at Optimizely and that means I also interviewed probably the other 200, 900, whatever people that we didn't hire. And I think I shifted in that very hands on approach into not just delegation, but I think really abdication. I didn't do a great job of instilling the things that were important to me in the hiring process into my leaders. I sort of gave them ownership and delegated that to them. And it didn't mean we necessarily hired the wrong people, but I think it made us not be as disciplined in saying, you know, maybe instead of hiring somebody, let's try to build a system that can do this better. So that's an area where I think it's easy to sort of fall into this trap. If you raise a lot of money, it's really easy to spend it. That's not something that I realized at the time.

Yep, makes sense. Well, speaking of culture, you know, optimizely, you know, one of the strengths of the company has been the culture. From day one, you have a set of values embodied by the acronym optifi maybe share for the group. You know your thoughts on culture and some of the things you did in the foundational years, you know, because you always talk about how the reason you've been able to scale and achieve breakout. Scale is the culture.

Yeah.

You know, share your insights on that front.

Sure. Well, I'll start by admitting that I think there are a lot of culture cynics. I think people think of culture as a nice to have. It's a touchy feely thing, but I actually think it's much more important than that. And I'll start off by just saying, why did we start so early codifying our culture? It started really with this pragmatic need to be consistent. We would interview a bunch of people on a day, and at the end of the day, the interviewers would get together and do a sync meeting. And soon I'd start to hear people say, oh, that person wasn't a culture fit. And I would ask why? And their answers were always different. So I knew at least early on this was like 15 people. So I knew early on that at least we should be consistent. We might be wrong, but at least we're consistent on what it means to be a culture fit. And so that effort then transformed into really talking about the values. What are the things we care about in our co workers? What are some things we care about in aspirational values, Things we hope to achieve down the line. And so we articulated that now through this process as really this living document, which is our culture document. It articulates our values as these six key values that you alluded to spells at optify. Ownership, passion, trust, integrity, fearlessness, and transparency with a Y. And I can go up. I can probably speak for hours on Optify, But I'll just. Instead of just explaining what those mean, I'll just tell you a story of somebody who, on their third week of the job at Optimizely, really lived up to all six of these values. You'd hate that I told this story, but his name's Jordan. He's a software engineer on our team. And one of the things we do at Optimizely is we have a weekly all hands. We call it a show and tell. And at this all hands, we had an update. The marketing team says we were about 400, 300 people at the time. The marketing team says, hey, we had this bug on the lead form. We lost a couple thousand leads at the time. We were getting tens of thousands of leaves. We're like, okay, that's too bad. They sort of didn't brush it under the rug. But they sort of admitted the low light. Then during the Q and A session, one of our teammates asked me, hey, so Dan, I heard that we lost a couple thousand leads. What happened? And I asked our head of engineering to come up, and he said, you know, we'll do a full postmortem. We'll get to the bottom of this. Could have ended right There. But Jordan, in the back of the room raised his hand, took the microphone, which is a weird thing to do, but he's three weeks in the job, takes the microphone, and in front of the whole company, says, I was the one to introduce the bug. Here's what went wrong, here's why it went wrong, and here's why things like this will never go wrong again. And in that one moment, Jordan exemplified ownership. He owned the success or failure of what he worked on. He demonstrated passion. He had a deep, authentic excitement about the work he does. It wasn't about being a rah rah cheerleader or passion. It was like he deeply was excited and had to own that. The amount of trust I built in him in that one moment is probably more than anything anybody else can do. Integrity, you know, having the willingness to be intellectually honest. Fearlessness, being willing to take a risk in front of the whole company. Three weeks in, and transparency, really sharing, being candid about what went wrong. So we gave him close to a standing ovation for being willing to admit that. And those are the kind of values we care about wisely.

And how do you, you know, you mentioned that early days where you had this culture fit. You know, startup I was with, we always used to measure capability and culture fit, but we never codified it. It was just a yes, no, binary decision. How do you guys filter in the interview process, particularly now that you're 400 people, where it's much harder. You're not interviewing all 400 people or all thousand people that are interviewing for jobs. How do you guys filter for that at this stage?

Well, we had the same problem at Google. At Google, when I was interviewing there, we would interview people and have to assess them on their googliness, but we didn't ever define it. So it's really this amorphous, basically, do I like the person?

Yeah, exactly.

So we have a couple ways we do it. You know, one of the things I have a handful of questions I like to ask to really assess their level of passion. You know, I like to ask people, you know, if they had 10 years to live, what would they do? And I don't have a right or wrong answer. But if a lot about what they wanted to do, if they have 10 years to live, doesn't have to do what they're about to do at Optimizely, I don't think they're deeply there for the right reasons. We try to seek missionaries, not mercenaries. So people who are really focused on title as part of that process, to me, is a red flag. It means that they're about trying to make their LinkedIn profile look good, not how to make our customers wildly successful. So those are some of the things we use.

And as you go back to the early days, are there things about the culture that by definition have had to change? And how do you decide as a founder which of these things are not scalable and which of these are non negotiables? And I'm willing to even constrain growth if I have to, because if we lose this, we lose the built to last foundation of the company.

Well, the cultural values are designed to be the ones that will help us achieve our vision. If we don't believe that these cultural values are the right ones to help us achieve our vision, we'll change our cultural values. So it's not about a nice to have. It's really we're rooted in. These are the shared values that we care about and these are the ones we believe are the best ways for us to achieve success. Now, the values are different from the behaviors. Values of transparency, for example, I think are pretty universal behaviors within the organization, such as an early one that we had which was that every time we'd have a meeting of two more people at Optimizely, we would send the notes from those meetings to the whole company. That's a behavior that worked well at 40 people, does not work well at 400 people. So finding ways to scale the behaviors of our culture of transparency have been things that have to change over time. And here now we're using slack, we have a lot of much more pull, not push communication. So that's one of the great things that I love doing, is thinking about how to scale the behaviors so they still live true to our values. Even though what worked for us well when we're smaller might be holding us back now.

And how about symbolism? I mean, when you're 400 people, the stories you shared, the story you just shared about Jordan, how do you get, you know, those stories out there? Do you look for opportunities to hold someone up? Do you send a message to the whole company about something that happened in a sales call or in engineering that you want everyone to know about? How do you as a founder decide, okay, I want to hold this example up to the whole company? And how do you get those stories ingrained in the culture?

I see that as one of my most important solemn commitments to the company is to tell those stories and both internally and externally. And we do a couple things that facilitate that. We have this show and Tell, which I actually think is one of the most powerful ways for us to adhere to cultural values. It's a weekly all hands for an hour every Friday at 4. Where we don't just have me as a talking head talk about what we should be doing or not. It's the edges of the organization who present. It's the people, the Jordans of our company who present. And they're the ones that tell the story. At the end of that, I'll nominate. We have a lot of. You might think this is weird. We have a lot of words that we use internally. For example, we describe ourselves if we're a rocket ship as a company and optimizely employee, we're an optonaut. And so I have optonauts of the week where I actually tell a story for the people who've really lived up to our cultural values. And it helps. Praise is free to give and it's a way for them to be recognized for going above and beyond. It's also a way for us as a tribe to really recognize the people who live up to our values. So I encourage you all to really think about the ways that you can do that. I think a weekly all hands is a powerful way. And also another thing I do is for all the new employees on their first week on the job, I spend an hour with all of them talking about our culture, talking about the story. Hey, here's what you missed in the last six years. And really helping them understand the stories that really live up to our cultural values.

That's great, Dan. I want to shift a little bit to go to market. I mean, you've obviously a lot of companies here are scaling go to market. You've built an unbelievable sales machine. I think when we invested at the A, your payback period was three or four months. You know, I think in 2013, Dan told me a story at the end of the year that his number one performing salesperson the prior year had been a Nordstrom shoe salesman. You know, you had folks from non traditional backgrounds come in and just crushed it at optimizely. What was it about the model in those days and today that have allowed you to create this insane machine? I know a lot of it starts with Travis who was there in the early days and helped build that. But how did you make that happen?

Well, I'll tell you a couple secrets. One, we have a great sales team at Optimizy, but our real sales team is our customers. Getting our customers to be our evangelists and telling other people about how great Optimizely is, has been much more effective than the best shoe salesmans we can hire to sell our product. The second is really, I want to just agree with Keith Robart from earlier. He talked about hiring people who might not be proven. Doesn't mean that they're not great, but they're at a point in their career where they haven't demonstrated the success to get to where they ultimately will in their career. And so this salesperson, Julio on our team is one of those people. He at the time hadn't graduated from college. He joined as an intern. He had this tremendous raw capacity to empathize with the customer, to articulate our value proposition in a way that was using the language our customers were expecting to hear. And so we bet on him. We gave him the opportunity to go in and meet with some of the biggest companies in the world. And that helped harness and really helped him reach his full potential. And so we early on we hired much more for potential. And even if they were unproven, that's actually better. I'd rather somebody with a chip on their shoulder who wants to have an opportunity at our company to prove that they can do great things, than somebody who has the pedigree or resume that says they've done it before.

Makes sense. And have you had to shift that philosophy now that you're moving more to the enterprise? I mean, over the last couple years, you guys are now going after the largest companies in the world. Some of the biggest companies in the world are using Optimizely today. Six and seven figure deals. Has that caused you to shift the profile? Do you now have salespeople, more of the traditional Oracle salesforce kind of backgrounds? How do you think about that evolution and doing so without losing what you guys had in those early days?

Yeah, I think it's a real balance because the last thing I want to do is the same thing every other company has done. And so there are a lot of great leaders who have been through the journey from tens of millions to hundreds of millions to billions to tens of billions in revenue. And I hope that their perspective and their experience can help lens to the efforts we're undertaking to, to help us make sure we don't fall into the same mistakes most companies fall into. But we are in a new world. There's a consumerization of enterprise going on. And what's it gonna take for us to be successful at this next echelon of deal size and customers is not to take the playbook from Siebel, but to really think through how do we give people the opportunity to use our product, demonstrate value? AB testing, as I said, is a good idea. It's a very clear ROI sale. You get a lift. How do we demonstrate the value and then grow those relationships over time? So the way I view it is our job is to hire people who have that experience. But at the same time I never once in a meeting will allow somebody to get away with saying, well at Salesforce we or Dropbox we different things they've done at other companies, that's great as a lens, but it's not the way we're going to run things at optimizing makes sense.

The other interesting thing you guys have done from a go to market standpoint is you've ab tested your own pricing. You mentioned the precious metals pricing you had early on, which I think a lot of companies in this audience probably have some version of bronze, silver, gold and you guys have iterated on that and I think done so in an incredibly analytic, data driven way which I think would be really instructive for folks to hear about.

Yeah, well, we eat our own dog food and so we AB test a lot. One of the things we've been a B testing over the last few years is our pricing and packaging. And this is a really hard thing to get right. I think early on and even to this day we're very underpriced. People look at our product and you know when they're thinking about renewing, they're looking at the millions of dollars of lift they're getting and maybe the hundreds of thousands of dollars they're spending on optimizely. And so that trade off is I think very much in the favor of our customer. And I'm okay with that. I'd rather be in the world we're in today than one where people feel like we're gouging them on price. That said, for us to build a sustainable business and an enduring business that can go far, not just fast, we have to be able to earn revenue from our customers. That revenue from our customers is what we want to fuel our business. I'd much rather that than other people's money.

Exactly.

But thank you for that anyway. I think so for us, as we think about pricing, we went from this traditional SMB self serve, simple to understand pricing, 1979, 399amonth to what we have today through several iterations to two models. A flexible model gives you maximum flexibility. It's pay as you go, all the features, all the functionality, don't need to talk to A single human. At Optimizely, we give you a $500 credit to get started, to sort of prove the value. That's a great way for us to land at large organizations. It's also a great way for small startups to try the product and get value. And then once we've demonstrated the value, then all of the conversations we have with our sales team aren't about gouging you or as I said earlier, shenanigans. It's really about how do we get you a discount off of your essentially pay as you go price that maps to the value that we can create. And we might say take some features out. We may not give you all of the integrations and functionality that you need eventually on day one. We'd rather on day one, you go on a journey with us. And so that's been a huge success for us.

And I think the other thing you guys have accomplished, I mean, this pasture is a huge milestone for optimizing lots of launch the personalization product. And, you know, I think what's interesting, if you look at the history of enterprise software and SaaS companies, you know, most companies rarely build a second product, you know, second product that's kind of as meaningful and as big and has as much potential as the first product. And part of the reason and, you know, optimizely had this going for it. You have a machine that's working, you know, for your first product, and certainly a B testing Optimizely is nowhere near saturated. You have a huge market ahead of you. And so as a founder and CEO, how'd you think about taking enough resources away from this machine that if you put 100 more people against it would just grow faster and take those people and say, hey, we're gonna go do this entirely different thing. I mean, that takes. I mean, it's hard to do. This is why most companies haven't pulled it off. How did you guys pull it off?

Well, I wouldn't say we pulled it off. I think that we've learned a lot in this journey. And so today we have two solutions, testing and personalization. And they're incredibly powerful. But the path we took to go from just testing to now testing and personalization, I think I learned a lot. First, in December 2014, I stood up in front of the company and I made the case for a second product or a second solution. And unfortunately, when I made that case, I think I really did ourselves a disservice by underplaying how valuable testing in itself was. I said things like Testing inherently for what it is is not unturnoffable. It's not something that is so sticky that it's a must have, it's sometimes a nice to have. And I think I made the mistake of overgeneralizing. I didn't think deeply and empathetically for the companies like IBM for whom we are essential, we are an essential product in testing.

That's right.

And so by sort of underplaying our existing product to sort of make a case for the new product, I think we didn't appreciate how great things were with just testing. The second thing is I think we over invested early on in personalization before it had product market fit. Looking back, what I wish I had done differently is have a small team of folks working on getting six reference customers who are using the product, paying for the product and are willing to be public spokespeople for the product as a way to really define the SLA between our engineering product efforts and the go to market. We had made that clear commitment say hey, the goal of engineering and production is to get six reference customers and then we're gonna put the gas on with personalization. I think that would have helped us. You know, we wouldn't have this large investment of people still trying to figure out do discovery while they're delivering the product.

But now it's done.

It feels great now, but it was a painful journey. So hopefully you don't make the same mistake I did.

That's right.

You know, and I think it's a reflection that it's hard to build something great. It really is. And you know, I'm really humbled and proud of the growth we've had with testing. But as you alluded to, I think we're only getting started. I think testing and personalization are the first two steps we think of as a hundred year journey toward enabling the world to turn data into action. And it's easy to lose sight of that journey when you see all these other companies that are growing quickly and you sort of get. I think I'll describe this, I've seen this here talking to some folks in the audience. I'll describe this as the startup duck syndrome where everyone talks about just like a pond of ducks. You see the ducks on the surface, they all look great, but, but underneath the water they're paddling really, really hard. If you go around in the audience, I talk to folks and people are all crushing it, they're all doing great. But the truth is underneath the surface you're working really, really hard. And I Don't want anyone to think that doing what you're doing is going to be an overnight success. And certainly when you had some semblance of success, there's always things you can be doing better. I know that for our company, and I'm sure that's true for many of yours. So it's the journey that I appreciate. But. But no, you're not the only one struggling.

You mentioned some of the lessons learned around personalization. Now that you're six years into it, looking back, you've had tremendous success. Are the things that, knowing what you know now, that you would have done differently in 2009, 2010, decisions you wish you had made in a different way. Obviously, overall, it's worked out great and you've built a phenomenal company. But any things that you think about, you know, that would be useful for the founders in the crowd.

Yeah. One of the best pieces of advice I got along this journey, which I wish I knew earlier, was that the role of management is to make sure everyone understands what you understand. And I, you know, I look back at the decision to do personalization or even earlier, I had a very clear sense of having seen the whole company, all the edges of the organization, a clear sense of what made sense and what didn't. But as you get to a certain size when you're 200, 250 people, the vast majority of the decisions that matter for the company are happening in rooms that I'm not in that my management team may not even be in. And so providing clarity and alignment from day one, a clear sense of what's important and what's not. More importantly, ability to give focus and clear direction on success. What does the outcome for success look like? Not prescribe the how, don't tell people how to do their job, but really describe what the outcome for success is and. And holding them accountable to the outcome for success and let the people closest to the decision make the decisions on the how and how to get there. So that's the advice I'd give myself, which is to make sure that I invest hard in providing clarity and alignment and then getting out of the way makes sense.

Dan, last question for you. If you have 10 years to live, what would you do?

I'd do exactly what I'm doing right now, sitting in front of all you.

Awesome. Awesome. Great. Thank you so much, Dan.

Thank you. It.