The Lean Startup, Part I (how to make your startup "ship" software that rocks customer's lives)

So who are you?

I'm Eric Ries.

Yeah.

I write the blog Startup Lessons Learned. Yeah.

And so I wanted to meet with you because you're doing an event and you're getting known, but the way startups get formed is changing, isn't it?

Yeah, I think it really is the most exciting time in history to be an entrepreneur. There's this trend in industry after industry where as software kind of infects what people are doing, it spreads an incredible amount of uncertainty disruption. People thought they had stable industries with solid barriers to entry and all that stuff you read in economics textbooks. And that made sense because there were physical constraints on what could be done. And software has this unbelievable quality that it's like the direct translation of imagination into tangible form. And so it has these really weird nonlinear effects. And so when I meet with people who are in larger companies, that's terrifying. But for entrepreneurs, it's just opportunity, opportunity, opportunity. So it's very exciting. Yeah.

Well, first of all, tell me what the lean startup is.

Okay.

And then we'll talk about why it resonates with so many people.

Yeah, thanks for saying that. I was an entrepreneur for about 10 years, and what I saw in company after company, starting at the dot com bubble, but then even beyond in Silicon Valley especially, were people building companies with way too much money, way too many resources, and a really premature launch. So that if the product was slightly off, or more likely, if the product was something that basically people didn't want, company didn't find out until it was way too late. And so I've had friends, the most talented people I know, spend years of their lives building something that fundamentally nobody wanted. And you think about that in the context of entrepreneurship, we're supposed to be taking big risks and we're supposed to be taking on incredible challenges. And yet when we have companies fail, not because the technology didn't work, but as Steve Blank likes to say, because there were no customers for it at the end, that's a preventable condition. That's an inappropriate reason for companies to fail. And so when I started to work with companies, after I left my last company and started to write this blog, I was looking for how do we create a theory that explains why some companies succeed, why some companies don't, and more importantly, can help entrepreneurs make good decisions about how to test ideas and how to prevent that kind of premature mortality of their company. That's what eventually led me to this thing called the lean startup, which is an application of lean manufacturing principles to the process of innovation itself. The Basic ideas are to test ideas early, cheaply, and with customers way in advance of having built some new technology, and then to get into a learning feedback loop that's company wide and continuously is testing new ideas so that idea failure doesn't have to equal company failure.

Interesting. As an entrepreneur, if you had the impulse to start a company right now, what would be the first thing you would do to set yourself up for success and set yourself up for a lean startup?

We usually start thinking about a company because we're thinking about a particular new product or new technology, and we have a vision for how that could change the world. And lean startup doesn't change. That vision is still the most important thing for an entrepreneur to hold. But what we're going to do in a lean startup, as opposed to the more traditional models, instead of immediately raising money and hiring some engineers and starting to build, we're going to begin with testing elements of that vision with customers as soon as possible. And so what we do is we actually split the company into two. There's one thing. What we used to call sales, marketing, and business, we now call the problem team, which is the part of the company dedicated to figuring out what problem are we really trying to solve for customers. Because we wrote a business plan, we think we know what problem customers have. But if you look at the stories of how companies actually evolve, they do this thing we call the pivot, they wind up solving the wrong problem or having the wrong solution to that problem, and they wind up changing course in this kind of zigzaggy path that ultimately leads them to some kind of success. We got one team that is really focused on talking to customers, beginning just like, who are they? Do they have the problem? Who are the early adopters? What is the business model? Those kinds of questions in parallel with what we used to call engineering ops and qa, what we now call the solution team, There's a set of people trying to build this thing we call the minimum viable product. Not how many features can we cram into a certain amount of time, but what is the absolute minimum amount of work needed to be done to test out the first element, a product's vision?

I think the best practitioner of this was Evan Williams at Twitter. It's amazing because he not only constrained the product, right, it had no features. The thing that was cool about it was it had less features than what had come along. You could only write 140. It was really a blogging tool, but you could only write 140 characters into it, right?

Yeah. And I'm sure. Now of course they knew all along that that limit was actually going to be, you know, a really, really good idea. But at the time, at least as I understand it, that was a constraint of the technology. They said, well, let's just accept this constraint for now while we work on what else we can do with this SMS sharing thing. And they were, I think, surprised at how powerful that constraint turned out to be, which is one of the really cool things about minimum viable product. You see this, you know, see it in Twitter, in Facebook, at Google. Sometimes the minimum version of a product is actually better than the maximum one that has fewer features, are often more powerful than bigger features. And so it's not just that visionaries, my experience is they're really afraid of exposing to the world a tiny piece of their vision because they're worried that it'll be rejected prematurely. But if we have a little bit of humility and a little courage, what we can find as entrepreneurs is sometimes we discover new opportunities that come out of the kind of simplicity and extreme constraints of actually having customers interact with a minimum version of our product. Yeah.

He also and I, when I met with the engineering team at Twitter early on, they admitted that they didn't engineer it for very, very big scale. Yeah, right. In other words, they didn't over build it.

That's right.

And talk about how you would engineer products, you know, and friendfeed talks about this too, that they would engineer for the next 90 days. They wouldn't try to engineer for three years from now. Very, very important because they could only see 90 days worth of usage anyways. So are you going to prepare for 90 million users that are never going to show up and never did in their case.

Right? Yeah, that's right. Oh gosh.

If they had spent all that time engineering for 90 million users, they would have wasted all that effort because they got bought by Facebook and now they're not even allowed to work on that product.

I'm not going to comment on that situation. I don't know anything about that. But we call it the curse of prevention, that when you anticipate a problem too much and you try to prematurely solve it, not only do you waste a lot of time solving a problem that might not arrive, scalability is the worst. Where if your mental model of what customer behavior is going to be is just a little bit off after a year of building heavyweight architecture, and I've made this mistake personally on several occasions, it doesn't even actually scale. That is scalability is something that you really can Only figure out when you're in contact with actual customer behavior, which is actually a metaphor for almost everything else that startups have to encounter. They say that no plan survives contact with customers. So at imvu, the last company that I founded, we practiced this thing called Just In Time Scalability, where we explicitly engineered our software infrastructure for agility rather than for scalability. So it wasn't very scalable. We started off with a very, very crappy, very simple architecture, but we could change it really rapidly. And so at imvu, we were on what we call the Disneyland calendar, because we had a lot of kids who were using our product. So when they were in school, they weren't on imvu, but nights, weekends, holidays. Those were our peak times. So we would come in on a Monday morning, we would look at our server logs for the last weekend. We were on this crazy growth curve where we're growing 20, 30, sometimes 50% a week. And we'd say, all right, what parts of our infrastructure? Not just the physical parts, but the software. Everything. Every system, every subsystem, which parts almost fell over this past weekend. And we would set a goal, all right, for each of those systems, we're going to refactor them and make them more scalable by Friday. So we would kind of make a decision about how much time we're going to invest in scalability on Monday. By Wednesday or Thursday, we'd be deploying the new, you know, horizontally partitioned database schema or whatever it was, and then we'd have it up and running on Friday. It had to be in order to work in the next weekend. And the reason we were able to do that is not because we were scalability geniuses. Actually, most of the small fixes were relatively simple. It was that we had good analytics to know what the problem was. And we practiced a thing called continuous deployment that allow us to put software into production in 20 minutes. So between the time we would write a piece of software and the time it would be live in production would be no more than 20 minutes. That's why MVU is famous for being able to deploy software 50 times a day. We talk a lot about how that's really helpful for running experiments and testing things out with customers. And it's great for that. It's also awesome for scalability because we could make scalability changes with no fear. When you're refactoring a core database subsystem, most engineers in most systems are terrified if you make the tiniest mistake, take the whole site down. But because of this thing, continuous deployment, one of the elements of it is this thing called a cluster immune system. Our servers had introspection during the process of deploying new software. They could tell if you were making a catastrophic mistake and could autonomously revert that software and send it back to you.

Wait a second. That sounds like a lot of engineering. How did you guys engineer that ahead of time?

Right?

Because we did.

That's the most amazing thing. Not only did we do just in time scalability, we built that system also in a just in time way.

So to most people, I mean, Microsoft, when I worked at Microsoft, they shipped software every three years. I know.

That's why you have software with a date and a name, right?

Yeah, exactly. And you're talking about shipping software how.

Many times a day? 50 times a day, on average. Sometimes more, sometimes less.

How does that happen? And is there any chance for bringing a company like Microsoft into that world? Or do you really need to start over with a new team, a new methodology? And there's no way to bring Microsoft Office into that world.

Well, so I can't speak to any specific product. And I worked very briefly at Microsoft. Boy. Well, like 15 years ago. So I don't know what it's like now. I can't speak to that.

It's even slower. Yeah.

That is the pattern in organizations that practice what we call large batch development. So the size of a batch is how large is the piece of software you kind of deploy to customers all at once. In the case of something like Microsoft Office, when you burn it onto CD and mail it to people or put it in a store shelf, it's basically the largest batch you can imagine. It's millions of lines of code. Whereas at imvue, we would deploy five lines of code as a batch. Sometimes no problem, it didn't matter. And one of the insights from the Toyota production system in Lee manufacturing is that when you reduce the batch size of work, you get faster feedback and problems are instantly localized. So if you deploy 3 million lines of code all at once and there's a problem, you spend a lot of time debugging. If everything's running fine, you deploy 20 lines of code and now you have a problem. High confidence. It was the 20 lines that caused it. You revert, you figure out, why did that interact badly. I want to answer your question, but having taught this now for a while, the hardest thing about continuous deployment is it sounds complicated and like magic, and that can cause people to not want to get started. And that's actually exactly wrong. We don't take a year off and say, we'll build a great deployment system and then start. Instead, you let the problems that you're actually having with your whatever it is that you're doing teach you where you need to invest in prevention. It's one of those Toyota sayings that drives people crazy. Every defect is a treasure. I used to hear that.

Well, they have a lot of defects this time.

It sounds like they've gotten a little bit away from the philosophy. I hope that they find a way to fix that. So I believe that anybody can move towards continuous deployment right away. And the reason is that most of there's this great 80, 20 rule. 80% of the firefighting and wasted time that most organizations spend is being caused by 20% of their system. It's just you don't know which 20% it is. So I don't want to get into too much detail unless you're interested, but we practice this thing called five whys, which is a way for doing root cause analysis when there's a problem. Five, five why's. Ask the question why five times.

Give me an example.

I'll give you an example. So imagine a server goes down. Okay, okay. Why did that happen? Well, we deployed some new piece of code and it caused the load average to spike. Yada, yada, yada. Okay, well why did we do that? Well, someone just, in that piece of code, they used an obscure API that has a side effect of taking the server down. It's like, well, why would they do that? Were they being malicious? No, they didn't mean to take the server down. They thought it was going to work. But they're a new engineer and they didn't realize that this API has that effect. Well, why didn't they know? It's because their manager doesn't believe in training new engineers. So you're like, whoa, whoa, hold on, hold on. I thought we had a technical problem of a server being down. And you're telling me we actually have a managerial problem of a manager who doesn't believe in training. And the answer is yes, behind every technical problem, so called technical problem is actually a human problem that needs to be fixed. Yeah. And so the technique of five whys, first part of it is just to realize that even though the symptom is acute, the problem is at a distance.

This is deep. Because at Rackspace, employee made.

No, that's right.

I can only go 14 minutes. FAT file system software. It ruins our lives. Anyways, we were talking about, you know, at Rackspace, we had an employee Made a mistake. And the chairman said, it's not his fault, it was the team's fault.

Yeah, we used to get new engineers when we would hire them at imview and we still do this. We would have them deploy code to production on their first day, maybe their second day of work. But if it took to the third day, we'd be really upset. And when we would bring in some veterans who've been in larger organizations for a long time, they would be really afraid. They'd be like, wait a minute, what if I accidentally take the site down? And we would say, if you can take the site down on your first day of work, shame on us for making it so easy. But our whole idea was to make the system resilient against human errors. Because humans are imperfect, they make mistakes. And so if you think about that five whys analysis, we have a server problem, an API that's too hard to use, an engineer who's not trained, and a manager doesn't believe in training. The second key insight is we want to make a proportional investment in prevention at each of the five layers of the hierarchy. So, so we'll obviously bring that server back up, revert that piece of code, maybe we'll fix that API so it's not so easy to shoot yourself in the foot. We'll talk to that engineer, make sure they understand why what they did was a mistake, and we'll talk to their manager about the importance of training. And the word proportional is really important because we don't like what happens is engineers sometimes will say, oh, we had a minor problem. Okay, well, I got to take six weeks off and invest in a whole new. It's like, no, no, no, no. If it was a minor problem, will do a minor solution. So my classic thing, I use this in workshops all the time. I actually had this argument, go to the manager and say, manager, it's really important to start doing training for your engineers. The manager will say, I totally agree. You're exactly right. In fact, I have an eight week program I'm ready to do to create a new world class training program. So if you can find someone to do everything I was planning to do for the next eight weeks, I'd be happy to do it. Which, as we say, it's manager speak for go to hell. Right? Yeah. And so this gives us a way out of that dilemma because we would say, what was a minor problem? So we've decided we're going to spend one hour in prevention at each of the five levels. Yep, yep. And so could you Please do the first hour of that eight week plan.

Yeah.

And the manager might say, well, what's the point of doing the first hour? You say, well, that's okay. Think of whatever you. Look, before you could do 100 hours, you had to do something first. So do that first thing. Say, okay, well maybe I could create a wiki page where we say things we should train new engineers about. We say, great, let's start there. That doesn't sound very powerful, but it really is. Because if training is a recurring issue in the organization and you do five why's regularly, this is going to keep coming up every time it comes up. We'll do the next hour and the next hour and the next hour until we stop having problems caused by training. And I use that example a lot because if you had come to me at imvu, I was the VP of Engineering and the cto. It was my job to make these kinds of decisions. And you said to me, you need to have a world class training program. I would have said, come on, give me a break. We're a startup. Training is for big companies. We don't have that problem. We hire the smartest people. I would have given you 100 reasons why. That's stupid. And yet, after doing five whys for a couple years, we wound up with a training program so good that we could get people productive on their very first day. We never had to make that decision, like, we should really have a really important training program. Instead, naturally, organically, incrementally, as a result of the problems that were actually happening, we wound up. Sorry, that were actually happening to us. We wound up building this really powerful system. And when people see it now, they're like, wow, that's so advanced. You must have really spent a lot of time on it. But not really. What we did is we borrowed from the budget of time we were spending on firefighting. That was actually wasted time that we could reclaim for a better purpose. And that's the essence of Lean Startup, is to use the mistakes and the failures as opportunities to learn something new and to get better at what we practice.

That's pretty brilliant. And it works not just with software either. I sat in on TechCrunch's one of their first tech TV meetings and Arrington has this bias for action. It's like, just go do it. Go do it and fail. Go do it. Make it happen. Stop giving me excuses about why you're not doing it already. Get out there and do it. And he forced his team to get out of the meeting and Go shoot a show. And that's sort of what you're talking about, is this bias for action. Get something done, and then let's figure out what didn't work about it. But at least we're further along. We had something done.

You're always better off having taken action than talking about taking action. What's interesting is, and this is where I think the Lean Start differs from what I call the just do it school of entrepreneurship, which is more in the kind of 37 signals approach, which I think has a lot to recommend it, or like the release, early release, often school that comes out of open source. Those are really good systems for getting started. But the problem is, okay, you just did it. You put something out there. As soon as you have three customers, you already have 50 opinions about what your product should do, right? Yeah. Well, now what? So what happens is people sometimes confuse, like, forward motion with progress. And first of all, talking is not progress. So getting people into motion and doing things is very important. But then we need some kind of framework for understanding how do we make sense of the information that we're getting back. Because entrepreneurship is classic overwhelming situation. You're always being told, you know, some journalist thinks you should do this. Your customer says you should do this. Your most valuable customer says you should do that. Your early adopters think you should do this. Ah. And so I think it's really important to be focused on kind of having a structure for thinking about what are the underlying assumptions that my vision is based on and how do I prove or disprove that my vision is correct, rather than like taking in feedback and doing what people say. So it's very interesting. Lean Startup is very customer centric, and yet you don't get a gold star for listening to customers. In fact, listening to customers is not in any way important except insofar as it gives you really important information about them. And that's really hard. You know, as human beings, we're not good at holding these two ideas in the head at the same time. My vision is brilliant and will change the world. And it's probably fatally flawed in some way. And that can give you a headache because that's cognitive distance right there. But by getting feedback, by testing the vision against what people will or won't do, by measuring their behavior rather than just listening to what they say, we can then figure out which elements of the vision are right and then double down in those areas.

This is really important and hard to do. One of my bosses used to say, don't listen to the Customers too much. Because if you were an engineer at Porsche and you asked your customers what they wanted, they would tell you more headroom, smoother ride trunk. Well, they just designed a Volvo. Right. And so what you're saying is design the Porsche and then test if they look like the Porsche.

Exactly right. And you don't get a free ride either way. You can't just be like, well, I did what customers said, so I'm good. It's like, too bad. But you also can't say, well, I'm the visionary. It's my job. At Telco, customers what they want. No, your job is to invent the future and then double check to make sure that you're right. And it's that process, you know, we call it a even.

Steve Jobs does this, by the way.

He does it incredibly well.

A lot of people think he's a dictator, and he is, but he also tests. I know people who saw the iPad, you know, a year ago and were in focus groups and he tested out the idea to see if people would pick this thing up and like using it and stuff like that.

I love that example because entrepreneurs get really confused by what they read in the press. And frankly, most of the stories you hear about how companies are run in the press are wildly inaccurate. And so people. Sorry about that. Yeah, listen, people respond to the incentives that they have. And it's a good story to tell about how Steve Jobs is this mythic hero who never makes any mistakes and doesn't need any feedback. But, like, it's just the evidence is overwhelming that that's not the case. People who are on the inside know that. And so, you know, part of what I'm trying to do when we start up with this conference in April is to, as an industry, get us to start putting the practice of entrepreneurship on a more rigorous footing. So we can start to think of it as a profession, as a kind of management that requires, you know, actual thinking about how to do it well. And we don't just assume that the best practices that work for so and so in such and such contexts work for everybody. Yeah. And that, you know, I think that's a challenge. That certainly is. Does it rub some people the wrong way? And I think that's actually a pretty good sign.

I also love that instead of building an entire training program, you just said, do the first hour. That sounds a lot like David Allen's getting things done where he asks you, what's the next action? You need to plan a trip to Europe. What's the next action? That will get you on that path. What's the first thing you need to do? I think that's really brilliant. Give me some more ideas like that. What other kinds of ways do you find that you get teams moving or get people shipping code 50 times a day instead of making excuses about how they have to wait three years?

What's interesting is that most of the techniques and the getting thing done analogy is actually really good because most of these techniques have this really interesting kind of long term, short term pairing, right. Long term vision of I want to go to this specific place, but very short term tactical. What's the first step I have to take or what's the first assumption I have to test? So for example, at Imbu, we were really proud of ourselves that we managed to ship our very first beta in only six months. In a previous company, we had spent like almost five years to get our first product out. So we were like, wow, six months, that's really fast. We killed ourselves to build this first version and basically it sucked. I mean, it was really buggy and we were really embarrassed by it and we thought, oh, people are going to hate it and they're going to think we stand we're a bad company and all this stuff. But then we discovered that nobody used it. And so it was like, wait a minute, hold on. We couldn't even get people to download the product, let alone discover how buggy it was. And that was one of those moments where I said, wait, how can this be? If our goal of this six months was to learn what customers want and don't want, why do we have to wipe all the software to do it? If we just put up a landing page that said please download the software and nothing happened. If you clicked the button, it would have been the same because nobody clicked the button. We were completely wrong in terms of what customers wanted. And so that was the first step towards reorienting my sense that progress in entrepreneurship has to be denominated in learning, not code. And that's like, I think that's a breakthrough idea. Especially because if you read the Agile Manifesto, which is document is profoundly important and really has influenced me a lot, it says right there in the Agile Manifesto that a line of working code is kind of how we're going to measure progress. And that's better than measuring progress in the older way, which is through milestones and schedules and documents and planning. But in situations of high uncertainty, which software is the root cause of so much uncertainty in this world? Code is not progress, learning is progress. And so that's really, it's very freaky.

That's actually really deep. And I think it's why Rackspace funds me to do this, to go around the world and learn. That's one third of my mission. You know, it's just to go around and learn and find great ideas.

It's surprisingly important and it's very hard to be an expert now if you, you know, in any job anywhere, because the rate of change that we're coping with so high. But you ask for specific techniques. So I'll give you another one that helps teams reorient from code to learning as a unit of progress. You've probably seen the traditional Agile thing with story cards where just for those who don't know what you do is you put up a big whiteboard with three sections on it and each card represents a relatively small task, one to two days. They call them story cards in Agile. And what you do is the three sections of the whiteboard are backlog, that is things you might want to do in the future, in progress and done. So it's a visual way for form of visual control, for seeing what needs to be done, what's being done and what has been done. And then one of the rules in most Agile systems like Scrum or XP is once a card moves into the in progress bar, we'll never cancel it. So if you're an engineer, you're working on something, you won't be interrupted to go work on something else. In exchange, the team agrees they'll always pull the top card off the backlog. So the kind of product owner or product manager or whoever's in charge of prioritizing things, they can reordering the backlog every day, all day, because it doesn't bother anybody. No one's anticipating what we might need to do. Everyone's focused on what needs to get done. There's a technique also from Toyota production system called Kanban, where we'll constrain the number of cards that can be in each bucket. And it's a way of preventing us from taking on too many things at once, which is bad for feedback. So for example, let's say you have five people on your team. You might say we're only going to have five cards in progress at any one time. So it forces each person to basically be working on one thing at a time and that way we kind of get more learning. It's not as efficient from a pure code production point of view, but it's better because you get faster feedback. And when a task gets completed, then you pull the next one and that's it. And then the people who are managing backlog realize, oh, we only need to have five cards at most in the backlog at any one time. We don't need to be keeping a list of 25,000 future features because 6 through 25,000 are irrelevant. Only the top five can ever get acted on any one time. It's a way of kind of focusing on.