Harvard i-lab | Eric Ries - Startup Lessons Learned

Eric has really played a role here. Last year for the first time, we developed something called the MVP award, based on Eric's concept of the minimum viable product. And we made awards of about $5,000 to 10 different student teams to develop a minimum viable product as they attempted to launch their venture. That was a great success. We had 88 applications for those 10 awards. We've doubled the program this year, so we'll be making 10 awards this fall and 10 awards in the spring. If you're interested, there's more information on that up on our Rock center website. The deadline for that is the 30th of September. So our format today. Eric will come up and make some remarks. And then my colleague Tom Eisenman, who I've mentioned, who is in addition to leading his course on launching technology ventures, and in addition to leading our first year entrepreneurial manager course, is also the head of the elective curriculum here. Tom will come up and interview Eric a bit, ask him some questions, and then he'll be glad to take some questions from the audience as well. So with that, Eric, welcome. Thanks for being here.

Hello. Wow. Come on down. The water's fine. I don't bite. Thank you. Thank you all for being here and thanks for that very warm introduction. I feel very lucky that during the time I was writing this book, I had the opportunity to test some of these ideas against some of the brightest minds in business and, and to feel very welcomed in the HBS community. So thank you guys for making that possible.

It's been really exciting and also a little bit strange because you gotta understand, when I tell people in San Francisco, where I live, that I am an entrepreneur in residence at the Harvard Business School, it's always the Harvard Business School.

You gotta see the look on their face when I tell them that. They think it's a very strange way to spend one's time. Because as you know, in Silicon Valley, we have a bit of an anti MBA bias and we tell all kinds of jokes when the MBAs are not in the room. So apologies. We do do that. And we have this idea that entrepreneurship is cool. It is really sexy. It's dynamic, it's creative. It doesn't happen in formal, rigid, bureaucratic places like corporations, company boardrooms and business schools. It happens in garages and in dorm rooms. And we have this bias that because it's so cool and interesting and dynamic, therefore it can't be managed, and therefore managers are like a weed that has to be eradicated rather than something that can help startups be successful. I want to be really clear. I used to believe that, too.

There's a few people here who knew me before I started doing this, whatever this is. And you understand, I was a programmer. I wrote software for a living. Probably the best day in my life was when I found out that you could get paid for programming. I mean, really, I programmed because it was intrinsically the most interesting thing I could think to do with my time. So I couldn't believe that you could get paid to do it. It could be a career. You could get a degree in it. That was like getting a degree in ice cream sundaes. I was really. I was pretty excited when I found that out, and it was pretty cool. So I thought I would be programming computers for my whole life, and I thought I would be very happy doing that. And so you can see how this is, like, the tiniest little departure from that track.

Way of context. I kept having this experience that was really frustrating, which was I would build this amazing technology and then nobody would use it. And at first I was like, well, that's just because the technology wasn't good enough. And the next time I'm going to build even better technology and more complicated features, and I'm going to have a better engineering methodology or I'm going to hire smarter people. And I basically was looking for any technical fix for what I thought was a technical problem. And I kept having this experience over and over again with startups that would fail. But even when I worked as an intern at Microsoft, I worked on products that nobody uses today. And I just kept having this, I don't know, bad luck. I somehow kept getting attracted to these projects that didn't work. And so I kept seeking out better and better answers to.

So how many people have seen the social network? Quick show of hands. Okay. Pretty much everyone. I lived through the first half of the social network, unfortunately, the part where you work really, really hard, not the later parts.

And Clark, who worked with me at that company, was one of my partners in building this company that was going to create.

Online profiles for college students for the purpose of sharing. Sounds pretty good, right? But we wanted them to share them with employers for the purpose of getting a job. So we were so close. Park. I'm sorry. Almost. We could have been there. We could have, would have, should have. What's so funny about that is that if you'd gone back in time and you told me, listen, you're almost there. But what you need to do is make this so that college students can poke each other with their Profiles, I would have been like, that is the stupidest thing anyone's ever said to me. Listen, we're trying to build a real business, one where you charge customers money. We want to build no Facebook. So, yeah, it shows you how brilliant I am as a product visionary. But we really believe that that was what it meant to build a real business. To manage a business meant it would look like this very rigid, specific thing about charging customers money. We didn't really understand why you did that because our understanding about entrepreneurship was like this. In all the movies, in Social Network, in every magazine profile you've ever read about a great entrepreneur, the story is always told in three acts. Tell me if this sounds familiar. Act one, the plucky protagonist, their character strengths and defects that will turn out to be really interesting later. How they came to be in the exact right place at the exact right time. How they came up with their brilliant idea, how they recruited the super smart people to work with them, et cetera. Act two is really short. Act two is what we call a photo montage. Lasts about two minutes. There's no dialogue. It's just guys pounding on keyboards or shaking some hands. In the Social Network, they write on the windows with those cool markers. We used to do that too. That's just what you do in dorms. You have these really cool markers. Where do you even get those markers? I don't know. Anyway, you get over the photo montage as quickly as possible so you can get to Act 3, the really interesting part, who sues who, how you divide up the spoils, who's going to be on the COVID of the magazine.

Really interesting stuff like what kind of car are you going to get to drive? And as entrepreneurs, we like to spend a lot of time thinking about Acts 1 and Acts 3. I admit it, I used to spend a lot of time thinking about what kind of car I would drive when the day came, how I would pose on the magazine. And I thought that that's all you had to do, basically come up with a great idea, be in the right place at the right time, cover of magazines. And then I kept having this experience where things would fall apart. And to make a long story short, I now believe that, that all of the important decisions that influence the outcome of startups happen during the photo montage. That maybe 5% is the great idea on how to divide up the spoils. And everything else is in the photo montage. So why is it in the movie? Why is it not appear in the magazine profile? I would like to stipulate it is because the real work of entrepreneurship is too boring to make it into the movie. Entrepreneurship is not cool. It is not sexy. I know right now entrepreneurship is a hip, cool career thing to do that, you know, winter is coming, as they say, so that will pass. Entrepreneurship is gruelingly difficult, incredibly embarrassing and totally uncomfortable and anything but cool. What happens in the photo montage? It is endless product prioritization meetings. Who's been in a product prioritization meeting in their life? Who feels like it would make a great movie? That's my point. It's deciding which customers to listen to and which ones to ignore. It's deciding whether we should pivot or whether we should persevere. They're like really difficult and annoying, frankly humiliating questions of entrepreneurship. And so I believe that if we're going to improve the practice of entrepreneurship, as has been my mission for the past few years, we have to get really focused on the boringest parts of entrepreneurship. And that brings me right back to being here at Harvard Business School. Because what do we need to focus on? We need to focus on two concepts in the book I talk a lot about, which is always a huge disappointment to people who came expecting like, ooh, I'm going to start exciting, cool. Talk about entrepreneurship. No, we're going to talk about management and we're going to talk about accounting. What could be more boring? Entrepreneurship dynamic and cool. Management accounting. HBS guys in suits grip. But think about this for a second. I have met many corporate CFOs and venture capitalists who have the same problem. And the problem goes like this. I gave a team a bunch of money, a million dollars and a bunch of resources and a bunch of time. I said, you got a year to do something amazing. And at the time that I made that decision, they showed me these charts with these hockey stick shaped graphs, amazing spreadsheets that showed how they were going to make just a zillion dollars with my money. And I was pretty excited. So fast forward a year. The team comes back to tell me how it's going. What do we know for sure? Think about this for a second. What do we know for sure? We know for sure that the money got spent. The money always gets spent. We know for sure that everyone was really busy during that time. We know for sure that they hit some very important product milestones. And we know for sure that they have almost no customers and no revenue. Because of course, the plan didn't quite turn out the way they promised. This is exactly what happened to me in every one of My companies. Now, we like to make fun of the corporate CFOs, who then kill those projects and fire those managers because they're so conservative and they're all focused on the bottom line and roi. But I want us to just be a little bit sympathetic for a second because according to the current management standards, the current counting practices that we teach MBAs, the CFO has just no information that can distinguish between a team that is on the brink of breakthrough success and a team that has spent the last year totally goofing off and doing nothing. Both teams will have spent the money. Both teams will have a pathetically small number of customers. The gross numbers that we care about in accounting are all going to be, you know, too small to be important. And by an ROI calculation, both teams will have seen, you know, basically not that good. Think about that for a second. We're not talking about being unable to differentiate between, you know, slightly different degrees of performance of a team. We're saying we can't tell the difference between cataclysmic failure and, like, brink of utter success. So I think that is a dramatic breakdown in our current management tools. It is an unacceptable paradigm to use for managing innovation, and that's what we have to do something about. It doesn't matter if our stakeholder that we're trying to convince is our CFO or a VC or our spouse or ourselves. The really deep question at the heart of entrepreneurship is, am I making progress? Because if I'm building something that nobody wants, why should I be proud of having done it on time and on budget and yet go to startup board meetings? What do we talk about? Are we on track? Are we executing the plan successfully? Are we hitting our product milestones? And rarely are we talking about who made that plan in the first place and what orifice did they pull it out of? How do we know it's any good? And here's the trick. We are now this year celebrating the 100th anniversary of the idea of management. The famous seminal tech. The Principles of Scientific Management by Frederick Winslow Taylor was published in 1911. And here we are in 2011. And in the 20th century, we got really good at this kind of management that deals in plans and forecasts. In the 20th century, if you're a manager who beat plan, you get promoted. And if you miss the plan, look out.

But planning and forecasting only works if we have a long and stable operating history from which to extrapolate the plans. And I would like to posit that the world see if you agree with this I feel like the world is not getting more and more stable every day. People feel the same way. So if the world's getting more unstable, then more and more of our work is being pushed out of the domain of what we call general management and into this domain we call entrepreneurial management. I believe entrepreneurship is the management discipline that deals with high uncertainty situations. And going back to all the startups that I built that failed, the common thread through all of them is that we were convinced we knew what customers wanted, we knew what product was going to work, we knew what our market size was, we built these gorgeous business plans. And yet it wasn't that our business plans were insane or incoherent or stupid. They were just based on wrong facts. So they were like beautiful but false. And yet we diligently executed those plans extremely well. One company I built, we spent five years and $50 million successfully executing the plan. We had 200 employees by the time customers first saw our product. That should be all I need to tell you about what happened next. You know what happened. We see this all the time. These big companies and startups both one day launch this amazing new product. It's the best thing since sliced bread. They spent years developing it and sometimes as little as six weeks later, it's pulled from the market. That is the incredible waste that is plaguing entrepreneurship and innovation work today. And that is the waste we're dedicated to eradicating. So that's all just by way of background. You should understand that at my last company, which is called imvu, we were really determined to do things differently. We felt like we'd been burned. We were like, for God's sake, let's just try to make some new mistakes. And so let me see if this, let me try this idea out on you and see if this makes sense. We wanted to build, we were in the 3D avatar business. We had this 3D avatar technology and we thought we wanted to make it social. And this is 2004. So the hot new social technology at that time was instant messaging. And we wanted to build like the next aol back when that was still cool. And so all right, here was our plan. Everybody knows, and I've got a lot of MBAs here tonight. Show up hands. Who's an MBA? MBAs, raise your hand. It's OK. This is a safe environment to admit it. Okay, Lots. So all of our MBAs know for sure. What do we know about instant messaging? Network affects business, right? The value of the network increases with the square of the number of Participants, the more people who have the IAM client, the more valuable it is. In 2004, everyone already has an IM client. So what else do we know? We know high switching costs, because if you want to switch from one network to another, you have to bring all your friends with you and therefore instant messaging business with high barriers to entry. Right? With me so far, that seems pretty smart, right? So here's our strategic brilliance. To get around this problem, we would build an instant messaging add on that interoperates with all the existing networks and that therefore would cause our product to spread virally through the existing networks automatically. Because the only way you could use it is by inviting one of your friends to chat with you. And we would send a link straight through the IM client directly to the other person. Doesn't that sound good?

I think that's good strategy. I think we really did a good job. I think your favorite strategic framework, you rechecked all the boxes. It's really good. And the only problem, there's like the itsiest, bitsiest little problem with that strategy, which is that everything I just said is not true. But it sounds good. Sounds good. The two most dangerous words in entrepreneurship. Sounds good. So we spent six months building this product that would interoperate with all the incident messaging clients. And to make a long story short.

Customers would not use it. They just wouldn't use it. They absolutely refused. Our brilliant. We would try to explain to them the network effects and the switching costs and they'd be like, old man, what are you talking about? And you got to understand, I was the engineer. I was the CTO of this company. My job was to build the software that did this. And so when we shipped it to customers, it was like super buggy. And I'd barely been able to cobble it together with duct tape and bailing wire. And we just. I was really afraid to ship it to customers because like, God, this product is more likely to crash people's computer than it would be to give them a delightful 3D avatar experience. My career could be ruined. But then we shipped it anyway. And I was actually relieved because no one would even download it. I was like, well, at least I didn't find out how bad it was. And I was like, wait a minute, Wait, what? Why did I just spend the last six months building this thing that nobody wants? And we had done it using Agile software methodology, which is the latest in building software without any waste. And yet we committed the biggest waste of all, building something nobody wants. So to make a long story short, we had to pivot to a standalone instant messaging network. And that was the beginning of Imbue becoming successful. Today, it's a very successful company. I'm very proud of what we've done. But just for a second, if you could sympathize with me personally as the Chief Technology officer, think about all the thousands of lines of beautiful, beautiful code that we had to throw out that did the instant messaging interoperability. I mean, it was deeply painful to me. And I had this thought after a while, which is like, why did I even have to be here? What did the company need me for? Couldn't I have been on vacation this whole time after all of my work got thrown away? And I was like, well, no, if we hadn't done that work, I had an excuse. I was like digging in the bottom of the barrel for an excuse to make me feel better. What's left at the very last excuse when all others are, oh, I can claim to have learned something, because if we hadn't done this thing, we wouldn't have learned that customers don't want this IM add on. Our strategy was flawed. I was like, phew, feel better. And those of you who've worked in big companies know how dangerous the thin ice I've just gone out onto. Because in big company, if you're a manager who claims to have learned something, you are about to be fired. Why? What do we know about someone who learned something? Two possibilities. One, they didn't make a very good plan, in which case, fired, or two, they. They made a good plan and didn't execute it well. So, like, double fired. Yet in entrepreneurship, our plans are totally bogus. They never work. So failure is the inevitable consequence of trying to do anything under conditions of high uncertainty. So why are we using learning as an excuse? And that's what really started to bother me. I was like, wait, wait. This is very upsetting because how come the word learning is only coming up now at the end, after we failed? How come we never talked about learning along the way? So if our goal the last six months was to learn this important thing about customers, why did it take 6 months? Why did I have to support 12 different IM clients out of the gate? Would the learning value have been the same if we only supported six? Yeah. What about three? Pretty much the same. How about just one client? Same value. But wait a minute. That's 10 times less work I would have had to do. And that wasn't actually so bad. I was like, all right, well, that's just. Next time, good learning. But I was like, wait a minute. What if we just created a single web page that had nothing on it but a screenshot of the product we intended to build and a giant download button? Would we even have had to build page two where we apologize that the product's not available for Download, or would a 404 have been sufficient? What did I say before? I said people wouldn't download the software. That means literally, they wouldn't click the button. It wouldn't click it. So what did it matter? It was on page two. And now I was really upset because, like, what did it say in my business card? I had a business card printed up Kinko's for $3, and it said, Imvue. Eric Ries, chief Technology Officer. It did not say, chief crappy landing Page Experiment Officer. I was like, how could it be that my work, the work I have spent my life training to do, could have the same value as some crappy web experiment that some juvenile designer could have done in an hour? That was really depressing. And that was the question that really eventually drove me to do what I do now, which is to start to ask the question, not how do we build the product in minimum time, achieve the spec, minimum effort, which I think is mostly what us as engineers or marketers or MBAs, we're all taught to do, but instead, how can we figure out what we're trying to learn with minimum effort? So I believe that a startup is an experiment. Everything we do in a startup is an experiment. And not an alchemy experiment, a scientific experiment, one which has a hypothesis that can be falsified. So the problem, you know, there's the people who think, oh, the MBA bureaucracy, that doesn't work for innovation. It's too dynamic. There's also this other problem of what I call the Nike school of entrepreneurship, which is like, just do it, ship it and see what happens. The problem with ship it and see what happens is you are guaranteed to succeed at seeing what happens. Something will surely happen, and then you can claim success when that happens. You can't learn if you can't fail. So what we're trying to do is get entrepreneurs to reconceive everything they do as an experiment designed to learn. We call the unit of progress of entrepreneurship Validated learning, scientific knowledge about how to build a sustainable business. And once you make that mental shift, a lot of cool new things become possible. So we can say, instead of, how do I achieve the spec? We can say, how can I perform this same experiment at lower cost? And that's the question that could have allowed us at Inview to make that pivot in days instead of months, because we could have said, wait, what are we really fundamentally testing here? At first we're testing, hey, do customers even want this product? We could find that out way cheaper. So that's at the heart of what the lean startup is all about. Learning to know the difference between value and waste. And the word lean comes from lean manufacturing, which is a methodology that has revolutionized the manufacture of physical goods. It's the culmination of that management project of the 20th century. I think it is the most advanced management system we have devised. It is very good at answering the question, how do we build this thing with maximum efficiency and build quality in from the beginning. But my contention is that if we don't know who the customer is, we don't know what quality means. And so all of our built in intuitions about how to do high quality work have to be adapted to entrepreneurship and take into account that fundamental uncertainty. So in a lean startup, what we do is adapt many techniques from lean manufacturing. I'm sure we'll get a chance to talk about a few of them tonight. The famous Toyota Andon cord that can stop the production line has its analog in something called continuous deployment, the process of putting code into production 50 times a day on average. And we also pull in techniques from direct marketing like split testing and cohort analysis, and all these tools that normally are used to optimize for efficiency. Here we use to optimize our speed through what we call the build, measure, learn, feedback loop. But our goal is to as quickly as possible turn ideas into products, test to see how customers respond to those products, and then learn impacting our next set of ideas.

So that's what the project of Lean Startup is about. And let me come back to why I wanted to spend some time at Harvard Business School talking about it. I've actually been very impressed the last year and change to meet members of the HBS startup tribe who have started to get what I think is a new religion about this new way of entrepreneurship that is very exciting to see. I tell people back in Silicon Valley, the first humble MBAs I've ever met, because they've gotten the message that in Silicon Valley they're a problem and they were actually curious about figuring out why, figuring out how can I as a manager add value in an entrepreneurial setting? So I consider it really a privilege to get to work with future entrepreneurs who happen to be MBAs. But the stakes for this change are a lot higher than that. Because it's not just entrepreneurs who need to understand entrepreneurial management. It's all of the other players who impact the entrepreneurial ecosystem. So for example, in Silicon Valley, when a big brain dead company way overpays for a crappy startup and buys it for $700 million, and then the next year it turns out that that startup is only worth $30 million, that is an epic failure of general management. General managers are not supposed to buy an asset that's worth hundreds of millions and have it instantly turn into tens of millions. That's really not supposed to happen happen. And in Silicon Valley, we celebrate that the entrepreneur made all this money. We laugh at the dumb MBAs who made the decision to buy the company and wash our hands of it. But if you think about it, I think that's a really big mistake. Because we are empowering those bozos to anoint the next generation of leaders in Silicon Valley. Because who do we listen to as the experts in entrepreneurship? They're the people who made the most money selling their startups to big companies or taking their companies public. And so we have actually empowered people who don't know anything about entrepreneurship to make really critical decisions that influence the entire entrepreneurship ecosystem. Those people buy startups, they nurture internal startups inside of big companies, and they also happen to be the LPs and investors in venture capital firms themselves. They become VCs. All of our stakeholders and partners need to be educated about entrepreneurship. So to me, the project of reforming the entrepreneurship ecosystem to have higher success rates, to be more scientific, to be more meritocratic, requires educating everybody about what successful entrepreneurship looks like, so that we can actually start to tell the difference in an objective way between what are the companies that are making progress and who's just putting on what I call success theater. So that's the project that I've been lucky to have some colleagues work on here at hbs. That's why I was happy to come have this conversation tonight and listen. I'm on a book tour. You might have heard, when you're an author, your job basically becomes like, convince every person on the entire planet to buy your book, no matter what. I have spent the last seven days doing nothing but listening to the sound of my own voice. I've been talking constantly, and I'm actually way more interested in hearing what you guys have to say than in just telling you more and more and more things that pop into my head. So if you would be so willing as to have a conversation, I would find that Much more interesting. I hope you will find it interesting. And I thank you all very much for coming. So thank you.

This whole project is Tom's fault, so if you didn't like it, you can blame Tom.

Thank you, Eric.

Oh, man, how embarrassing.

Can I get some data, please? Of all people?

Yeah, that's a good idea.

Do a little market research. So, as Mike said, it's a brave new world here at Harvard Business School in terms of entrepreneurship this year with the iLab, we're reaching out beyond the business school to the whole community, other parts of Harvard, and to the startup community in greater Boston. So it actually would be very useful to Mike and to Gordon Jones and to me to see where people are from. Eric asked how many of you were MBAs, but you barely put your hand up. So how many of you, hands high so we can see it and get the data, are from the college?

Not bad.

Yeah, not a ton. And how many are from a graduate school at Harvard other than Harvard Business School?

Oh, a lot more.

Okay.

And how many of you are from a university other than Harvard? Presumably a lot of mit, but hands high.

Lonely, huh?

Sorry, we didn't get that word out. And how many of you are from the Boston school startup community? Not currently in school anywhere? Yeah, that's cool. So we put out the word to the lean startup. Eric's got a lean startup circle in 100 cities.

Something like that. Yeah. Thank you guys for being here.

We put that word out, and I'm glad it seemed to take hold. And how many of you are Harvard Business School second years ecs, and how many are Harvard Business School first years?

All right, so I saved you for last. You guys are going to have a lot of fun this year. As Mike said, we're going to pull these. You'll launch as part of Field a Micro Business in the winter and spring, and the conceptual foundations on which you build that business will be lean. So I hope you like this stuff. And in parallel with that, you'll take a course. One of your other courses will be the Entrepreneurial Manager. And we're going to pull these ideas into the front end of that course. So the RCS will get a heavy dose of that. And ecs, if you take the course, launching Tech Ventures, likewise will expand on that.

Can I make a request real quick? I'm so excited to have people from the academic and the startup communities here in the same room at the same time. I get a sense this doesn't happen all the time. And so if you happen to be sitting next to someone who's from a different community as you. Before you all leave and vacate, there'll be some time for mingling. Just do me a favor and introduce yourself to somebody who doesn't have the same background as you so they can feel like we actually fostered some real.

Intergroup connectivity.

Yeah. And there's going to be a ton more of that coming this year with the Ilab. Next week, spread the word about the Peter Thiel speech. And the Ilab will be constantly open, so this group will move over, as Mike said, to Kresge. So introduce yourself. Keep the discussion going over at Kresge.

Wow.

So, Eric, what's hardest for people to learn when they start to embrace these concepts? I'll take you through five steps just quickly, so you can choose the ones that are easy to embrace, the ones that are harder for people to embrace. We're trying to use this to change the way they innovate and launch businesses. So there's the decision to launch early, to sort of. To sort of compress the montage or move it forward a little bit. There's the task of taking your business model and exploding it into a bunch of hypotheses, falsifiable hypotheses, which is new for some people. For each of those hypotheses, defining a minimum viable product, the sort of smallest set of activities or features, you need to test the hypothesis in a rigorous and decisive way. There's actually doing something with the data, sort of embracing the results, sort of staring reality in the face, and then there's, if necessary, pivoting, sort of finding a new direction ahead. Which of these are hard and which are easy?

Probably the hardest one, on average, is making the decision to test early because it is so much more fun to be in stealth mode. I can't even tell you how much more fun. You know your head's down, there's no pesky customers bothering you. Everything works according to your mental model, just like it should, should. And every person you talk to, every scrap of data you collect, every research report you read, every single one, confirms that your idea is the best thing since sliced bread. If you've never had this experience, it's really hard to explain. You look like a complete moron from the outside, but everything from the color of the cars you see on the street, everything confirms that your idea is great. So going from that to being like, boy, I'm going to put myself in a position to be wrong, I think, is really the hardest thing.

Analyzing your business plan strategically is actually ironically One of the easiest things to do. It's not necessarily easy to do well, but what I have learned is that every entrepreneur I meet already thinks they know what their strategy is and is willing to elaborate on it in great detail. And they just. It's again, not that they're incoherent, it's just that they happen to be factually wrong.

So turning beliefs into hypotheses is. Isn't that hard for people.

It's not, especially because.

If you do it wrong, the penalty is just not that bad because you still put your minimum viable product out, and then, like, crazy stuff happens that you never anticipated. And you're like, boy, I sure wish I realized that that was one of my hypotheses I didn't state. But, like, the penalty's not that bad compared to spending years in stealth mode. And then the next hardest thing after just deciding to do the mvp is deciding to pivot. You will wish that you pivoted sooner. I guarantee every entrepreneur that goes through a pivot wishes they had done it sooner. But it is really hard to walk away from that sunk cost of all that energy to admit that something isn't working, to take the risk of morale and the risk of embarrassment. There's a lot of reasons psychologically why it is hard, hard to pivot. And yet almost all of our energy, if you read the book, the way the book is structured, almost half of the book is all a gigantic prelude to trying to make that decision to pivot or persevere a little bit easier, a little bit more rigorous, a little bit less fraught with difficulty.

The word pivot's getting confusing because it's gone into the popular vernacular.

And it's almost seen as a badge of honor. Right? It's a thing entrepreneurs do. And we ran into this in this second year course last year when we were teaching these ideas that it's a strange thing. Pivot's not a goal. You're not trying to do one. On the other hand, if you need to do one, you better get around to it.

So.

That does feel tricky. And then defining the MVP feels like defining the hypotheses. Probably not hard because you think you've got it, but you may not have the best possible one.

Yeah. And in fact, there's an easy heuristic for knowing what to do if you're trying to figure out what's the MVP like, calculating the ideal mvp, which is kind of the minimum amount of effort invested for maximum learning. You can imagine some MBA building a spreadsheet of Incremental learning units compared to incremental effort. Okay, that's hard. But guess what? As an entrepreneur, whatever you think the MVP is, is way too big. I already know that for a fact. So all you have to do is take whatever you think the MVP is and cut it in half, and then cut it in half again. And I know that's impossible. I know you couldn't possibly do it. That's the craziest, stupidest thing you ever heard. Because your idea is perfect. I understand. Believe me. I so been there. But the nice thing about MVP is that the penalty for releasing too soon is way better. Like, it's really minimal compared to the penalty for releasing too late. Because here's the thing. Let's say you release half the product and customers hate it. And they're like, you idiot, you glossed with feature A but not feature B. And you're like, ooh, feature B is the thing on my roadmap. Like, congratulations, you were right. Isn't that great to get that confirmation that you're on track, you just double checked, everything's fine. But the truth is, that's actually not what usually happens. Usually you launch without feature A and either customers are like, I can't believe you idiot. Can't believe you launched feature A without feature C. And you're like, I'm sorry, what feature? What? Huh? Oh, really? Yeah, that's definitely now in our roadmap. You know, instantly you already go back and backdate that you always were planning to do that. So that's a big win. But there's an even more crazy thing that can happen. Sometimes customers like the lower quality, crappy version better than your beautiful castle on a hill. Like, now we all say that Google brilliant, genius Google founders understood the power of simplicity, and they understood that the simple search box would be the thing that would disrupt the complicated portals. People have heard this story before. The nice thing about the Internet is we have something called the Wayback Machine, which lets you go back in time and see how it really was. So go back and look at Google 1.0, go look at the early days and read the About Us page. And the About Us page is the most whiny, apologetic thing you will ever see. It's like, listen, we know we're supposed to have these other features and we're sorry. We only have like this one box. And we'll totally get around all this stuff soon, but we just kind of wanted to put it out there and see what happens. They didn't know that that MVP was going to turn out to be better than the complicated thing they planned to do. They just put themselves in a position to have an unexpectedly positive result. So, yeah, for mvp, when in doubt, ship something too small and be right.

Just in the same vein, in the book, you talk about teleporting at imview. Can you relate that?

Sure. Yeah. I love the story. It's still embarrassing to me, even now. Okay, so I said imvue was a 3D avatar product, and you understand 3D avatars. It's like you have a little video game character, and you can. You could interact with other people in a scene together. So imagine that Tom and I are sitting on two chairs on a virtual stage, and our avatars are pontificating in front of a large crowd. And we get to simulate that because it's so fun.

In video games, if you play a game like the Sims, if your avatar. If you decide you want your avatar to move, here's how it works. You click, like, in a seat over there, and then your avatar would get up, smooth animation out of the seat, figure out the optimal path to walk over down the stairs, avoiding all the obstacles, go over to the side seat, sit down with a nice smooth animation, and then their looping animation would play. There's this thing called inverse kinematics, which is the technology for making sure that their feet glide against the floor, and it looks like they're actually stepping instead of just looking like a zombie. It's actually incredibly difficult. You need a lot of physics, and this is a hard technology to build, and we could not afford that for version one of imview. And so we were like, well, what can we do? Our avatars will just be stuck in place. So version one of InView, your avatars could not move around the scene. So we've actually built these scenes with a nice big auditorium or like a lake. You might want to walk around and anchor your avatar to one place, the place we chose for you, and that was it. And customer feedback was unilaterally so clear. They were like, I feel claustrophobic and stuck. I want to move. And they didn't say, I want inverse kinematics. They're just like, please let me move around. And we're like, oh, no, this is a disaster. We don't have to spend a year developing this technology. What are we going to do? And then I remember it was late one night, and I was like, well, what about this hack? And I was so embarrassed to even suggest, but I was like, well, we're really desperate. What if we Just try this and see what happens. And the hack was when you click your mouse to the place you want to go, your avatar would just disappear from the old place and reappear in the new place. And I'm not talking about a puff of smoke and a beautiful sound effect. No, I'm saying for the engineers in the room, literally, you are in one frame, you're out, your hand is over here, and in the next frame, it is like 20ft away in a new place, and you're in a completely different position. And here you're smoking a cigarette. It's just completely different. It's, like, psychologically jarring. It's just in cartoons and movies, they'd be like, you never do that. You break continuity. It's horrible. It's the worst thing ever. And we were so embarrassed to ship this feature that we wouldn't even ask customers what they thought about it. We just did it. And we're like, maybe no one will know this. See what happens. And right away, immediately from that very first day, we did a tracking survey to ask customers, what are your favorite features of imvue? And people immediately started saying, we love IMVU because it is more advanced. In the Sims, we were like, what? We were like, what does that mean? They're like, well, in the Sims, if you want to get somewhere, your avatar has to stand up, walk all the way around, but in inview, you teleport right there.

It's like avatar teleportation. One of our top, still one of our top, most loved features. And we were so embarrassed, but in retrospect, made total sense. We're asking people to suspend disbelief. And who wouldn't rather teleport, then walk? I mean, really, I gotta go to Chicago tomorrow. I would rather teleport. So here's my question, especially for the engineers in the room. Which version of the product is higher quality? The crappy hack or the beautiful inverse Kinematics product?

That's my question.

You want an answer or just keep going? Think about it.

I thought someone's head might explode. I was just waiting. I hope my point is clear.

So, speaking of heads exploding. Vision. Vision versus feedback. When we pulled these ideas into the second year course, we had endless debates about whether Lean Startup was degenerating into the customers leading the entrepreneur around by the nose and the entrepreneur giving them everything they want. And I think students had the perception that the entrepreneur was giving up vision, that there was a lot of sacrifice. So say a little bit about how you think about the balance between feedback and vision.

No, thank you for that. I got this so wrong at first. When I first started developing these ideas at Imbue, my co founder, who was a very visionary man named Will Harvey, who worked in the video game industry for many years and is really, truly a brilliant, brilliant product designer.

We would have these epic, epic arguments. I mean, I think about the thousands of hours we lost to these arguments. I wish I could have that time back. It was such a waste. Arguments would go like this. I would say, let's split test, or let's talk to customers, or let's ask them a question. Let's do a survey. And he would be like, but customers don't know what they want, so why don't we just tell them what to want and we'll build a cool thing and then when they see it, they'll love it. And I'd be like, but what if we're wrong? He's like, but we're not wrong. Our job is to see the future. And we would just go around and around circles like that. And the problem, of course, is that we were both right. Will Harvey was 100% right. Customers do not know what they want. This is like an axiom of any. You look at any psychology research and you look at how do human beings fare when they're asked to make predictions about how they would behave in hypothetical situations, like complicated ones, like, how will you feel about lunch tomorrow? No idea. Let alone, do you think you would want to buy this hypothetical product that I'm vaguely describing to you, you know, three months from now, for 20, $25? Garbage in, garbage out. Customers do not know. But that doesn't absolve us of the responsibility to find out how they behave. Imagine I'm a physicist and I came up here and I was like, I really want to discover the structure of the atom, but I can't because electrons don't know what they want and they won't tell me. I can't survey them. So no science.

That's really dumb. Your job as a scientist is to run experiments to discover how electrons behave. Well, guess what? Lean startup is a scientific approach to entrepreneurship in that the deal is if you don't have a vision, you cannot run an experiment because then you wind up back in that ship in a see what happens mode. What I was trying to get us to do, ask customers what to do, was also a waste of time because it didn't work. We eventually developed a rhythm where we would test our vision to see which elements were brilliant and which ones were crazy. Here's the thing. A pivot is a change in strategy without a change in vision. So we need to stay true to the vision throughout the life of the company. It's just when we encounter difficulty, we have to find another path to get to that answer. So letting data tell you what your vision is is like having your GPS tell you where you want to go. It's backwards. You tell the GPS where you want to go, but if you hit a roadblock and you have to go around, it helps you find the best. So the goal of strategy in a startup is to find the best route you can, given the facts you have, towards the destination, which is the vision. And without it, you really can't make it work.

Another place where the students snagged repeatedly, we did a case on Foursquare. And Foursquare is a classic example of.

The species where you see scale like crazy before you have business model sorted out. And some of this is in lean startup. If we go back to Steve Blank's work, one of his core precepts is don't scale until you've got a clear path for monetization. So our students fought a lot about whether Foursquare was playing it right. One camp talked about the network effects you were mentioning before. The other camp said, no, why would you scale this thing up before you figured out how you're going to make money?

And that problem, you can go all the way back to Google, Facebook, YouTube, Twitter. It's a long list of very successful companies that cranked it up before they figured out how they were going to make money. How do we think about that?

Boy, Steve and I have lost a lot of hours to that argument, too. And Steve, he was very clear with us at Inview, you better charge customers money. From day one, he was on our board, so we listened to him.

And that worked out great for imvu. And we always had this kind of question in the back of our minds, but what if we were trying to build the next Facebook or next Google, would that advice be right? And we used to say, well, those guys were just lucky. So it doesn't count. That doesn't count as entrepreneurship. But I don't believe that anymore. And I think the confusion is about what are you really trying to find out? Why do you want to know how your business model works? I don't think it's because you want to know if you're going to be able to make money. I think it's you you want to know. Two things you want to know, does my product fundamentally create more value than.

I consume in producing it. Which in classical business would be like, is my marginal profit positive or negative? And I also want to know, is there an engine of growth? Is there a way for me to scale this business up? And in some businesses where you're paying for advertising, your revenue per customer is a key determinant of how fast you can scale. But in a viral business like Facebook, it really, it really doesn't matter.

I tell the story in the book of the very early days of Facebook when they raised all this money actually from Peter Thiel. You can ask him about it next week when you come to the talk. It raised all this money from Peter Thiel. When they had only a handful of campuses, they had a few thousand customers and no revenue. And the plot of the social network, if you recall, a big part of it is this debate about should they be focused on growth and monetization. And lucky for them, they chose right. And a lot of people at the time said, I remember a lot of people saying to me, boy, this is too terrible. They're getting this crazy valuation for this crappy product with no business model. It's like.com all over again. And I was one of those people, I was like, eh, Facebook never going to work out. So yeah, don't listen to me when you want to know which ideas are going to work out. I also didn't think Google was going to work out very well either. I turned down jobs at both of them. So I got a really good track record, 100% wrong.

But in retrospect, I think of it like this. Like if I came to you and said, I got a business idea, my business idea is I want to start a dime diamond mine. And so I'm going to dig into the earth and pull diamonds out of the earth. Now there are two fundamental questions you've got to ask for a diamond mine. A, if I dig into the earth in this place, will I find a bunch of diamonds? And B, if I had a pile of diamonds, would I be able to sell them for money? Now, which is the riskier assumption? When we look at it at diamond mine, it's like, well, duh, diamonds are intrinsically, of course, diamonds are valuable. You better go find out, can you dig into the earth? Well, Facebook is in the business of mining customer attention, accumulating this giant asset of attention and then reselling it to advertisers. So the question for Facebook is, if they can build this giant asset, would it be worth something? I think the answer is obviously yes. So then the question is, well, about how much is it going to cost them to build that asset. You're like, wow, if you can accumulate massive amounts of customer attention using viral growth at no cost, is that going to be a profitable business? I think it is. So I think it is not correct to say that Facebook had no business model in the early days. I think it had a very clear, very specific business model, namely value creation by trading attention for money. Value attention, sorry, attention acquired through viral growth through the viral engine of growth. In the book, I lay out three different, what I call engines of growth, viral, paid, and sticky, for three different ways you could do this test. And from that basic decision, then that lays out what are the key metrics you need to do to measure. And that can resolve the argument that Steve and I used to have, because only really in one of the three engines of growth is how much you can make per customer one of the key variables. And the others, you could basically ignore it.

Lean Startup reminds me, my colleague Clay Christianson had, maybe it's fractal, maybe his is bigger than yours, but I hope this gets big. But he had an experience 10 or more years ago that parallels in some way what's going on here. The term disruptive technology is tricky because everybody thinks they know what disruption means, but Clay means something very specific by disruption that isn't what most people just think. It means some kind of radical change. And it turns out Clay was not really talking about technology as much as he was business models. So he had both words wrong. But then the thing exploded and it's had a huge impact on managers and the way we think about business. Lean startup is a bit the same way. Everybody thinks they know what lean means. They think it means bootstrap, but it doesn't exactly mean that. And startup, I know that you've thought very hard about how this applies to companies, Corporations beyond just startups. And so as I watch Clay push his ideas around the world, he was of kind, constantly getting into situations where people were saying, either, wow, your ideas apply in this surprising space, or they don't apply here. I've got a counter example. In the insurance business, it's not disruption, it's something else. So have you started to be surprised? You must be going through the same thing, both in terms of the name you've attached to this and in terms of people sort of challenging where the ideas work and where they don't work. Have you been surprised yet?

Very surprised. Look, I wish I could claim I was some kind of marketing genius and that I deliberately picked this Trojan horror strategy, which, as it turns out, if you're going to name a thing, very useful for everyone to think they know what it is and something that they obviously want. And so they're like, yeah, I want that thing. And they're like, well, I'll find out what it is later. But, yeah, get it in here. And then they're like, wait, what? What did I just sign up for? And a lot that does happen with Lean, a lot. Especially in Silicon Valley, where the people are not educated about lean manufacturing and therefore don't know that history. They think lean means cheap. And it was very helpful. It just happened to be that when I happened to first start talking about the term lean startup, there was a financial crisis going on. And, like, every startup was being told to cut costs by their VC and it was a panic going on. It's hard to remember that was only a few years ago, given what's happening right now. And it was really handy to be known as the lean startup guy at that time, because people who were desperately trying to save money would call me up and they'd be like, my VC said I had to save money. What do I do? And I wanted to be like, let me tell you about the build, measure, learn, feedback loop. They'd be like, the what? I was like, listen, if you just cut cost indiscriminately and you cut the costs that allow you to go through this loop fast, then you're just going to go out of business more slowly. Why is that admirable? And they'd be like, oh, yeah. And they're like, but here's something else. If there are costs that you currently have, wasteful costs that are slowing you down, why do you need a financial crisis to cut them? Why don't we just cut them all the time? And we only had that. The only reason we could even have that conversation is because they called me thinking I was going to help them save money on office chairs. So it was super helpful at that time. It has become something of a liability in some quarters. Like, there was a fat startup backlash that lasted a little while because people thought lean meant cheap and not raising VC and therefore fat meant like, go big, be bold, have a big vision. I hope we've totally dispelled that idea. And actually, now as we're going into more corporate settings, it's proving to be really valuable because most corporate managers know about lean manufacturing. There's actually a lot of companies, like, you know, I spent time with Amazon. I just met some of the senior executives from ge, people who have a supply chain or manufacturing business love Lean and are all about it. So when you tell them, hey, did you know that you're doing lean in this part of your business, but you're doing anti Lean stupidity in this part of your business, they're like, what? Hold on, I'm a black belt, certified, whatever. It can't be right. And they're like, oh, how exciting. And startup, since startup happens to be the most cool, sexy thing in the world, every company wants to bring startup DNA into their company. And the message that I have for them, which is, I think very hopeful, is that you don't need to bring in startup DNA from the outside. You actually already have it in your company. Your just oppressive process is eradicating it. So if you just stop doing that and use this alternate process, we can capture that. We can capture all that time and talent in a book. I give some case studies of that happening. But yeah, it is a funny thing about how things are named and how ideas spread. I admire Clay's work tremendously. Just for the record, it's been very foundational to me.

Just a quick a couple of personal questions before we promised the audience that they'd get a chance to ask some questions.

Three to five years out. What's your business model? And sort of a you can figure out whether you want to tackle either of these or just say, let's turn it to the crowd and ignore this.

You've been a serial entrepreneur. Aren't you tempted to go do? I mean, in a sense, what you're doing here is very entrepreneurial, different than starting a company. Are you tempted to go do that again?

Yeah. Oh, sure.

How do you make money from. You don't make money from selling. I mean, you can't get rich from selling books.

Not really, no.

You get rich from giving speeches to people in big companies and repeating yourself over and over again. Can you handle it?

I don't know. We'll see. I mean, the truth is anyone who claims to make their money as a consultant, an author or a public speaker actually does all three. It's like the holy trinity of. That's my whole business model that self sustaining. So I've been doing that the last few years. It has been very surprisingly lucrative. I've been very blessed. I don't know if I'll keep doing it. I honestly don't know. Right now I feel like two things are true. One, I feel like I'm having an impact and being on book tour. The best part of it is getting to meet people who have put these ideas into practice and are advancing the state of the art about entrepreneurship. That's amazing. And it's been really intellectually interesting. And so as long as those two things are true, I think I'll keep doing it. I think this is a startup, so it's using the same part of my brain that writing code did all those years ago. So we'll see. My life has changed so much over the last few years. I no longer make any predictions about the future. God only knows what's next. Here I am being a professional expert. Who would have thought? Very strange.

Thank you. Thank you.

Questions?

Yeah.

Why hasn't this gene transformation happened already? I mean, given the precipitous economic decline and all the advantages of technology, when you go and speak to people and you point out, hey, over here, this is very wasteful, what interests are overcome?

Well, first of all, this is all really new. I mean, I know now it's considered overhyped, but it went from totally obscure to overhyped, like, without having passed through the intervening stage of people actually doing it, which is. That's our way in Silicon Valley. We only oscillate between the two poles. So you gotta remember, the very first time I had the courage to say the phrase lean startup out in public in front of an audience was just about two years ago. So these ideas are just starting to percolate beyond the early stages. But what's holding it back is what I talked about before. The fact that the prevailing system of management is so attractive is the main adversary that we have. And it's attractive for a really good reason. It works really, really well. If you look at the old General Motors, the companies that were dominant in the 20th century, it's all planning and forecasting, down to the boat phones. And most of us, even in most of our current companies, even the companies that are in major decline, there's a huge part of their business where things are going great. They're the stable part of their business. Management is like rocking it. So we're going to people and saying, the tools that you've learned your whole career, that you've seen firsthand work, amazing, actually don't work at all. And most people are like, I'd rather not listen to you. I'd rather just keep doing what I'm doing and sink into the quicksand and die. And personally, I'm indifferent as to whether we build the next generation of companies from scratch and replace the dinosaurs or whether the companies adopt these ideas and transform themselves into innovation factories. Either way, I firmly believe this is what work is going to look like 10 or 20 years from now. And for those of my colleagues who are in big companies, it's up to you. I'm just visiting. You're the one that lives there. So I think this is the transformation to make. I think it's important. But we are fighting against a very dominant, very entrenched legacy paradigm.

In the back over there. All the way in the back.

Thank you.

Can you give me an example of.

The methodology working in a service industry like consulting or insurance or something like that? I know it applies to digital technology.

But how it works.

Yeah, thank you. Thank you for that question.

Look, my background is in products and high tech software products, so I always use that language and I can't get away from it. It's just that's how I talk. That's the kind of business I am.

In the book, I tell a story of something called the Village Laundry Service, which is a company in India that does people's laundry. It was actually incubated by Innisite Ventures and they did the exact kind of hypothesis testing that we're talking about, trying to figure out how could they convince people who couldn't afford a washing machine to hand over their clothes to vls to have them wash them for them. And in the book, we talk about the different experiments and pivots that they went through. And it's, you know, it's just classic experimental design, just like we've been talking about. And that really isn't an important distinction, product versus service. Because when I use the word product, I just mean whatever the customer experience is. And so every service, the product is actually the playbook by which you train the customer service agents or the service delivery agents that tells them what to do in all these different situations. And just like in a product, the more elaborate and complicated and legacy, the playbook, the harder it is to change and adapt, the more fine tuned it is, the easier it is to run the kind of experiments we're talking about. So it's actually not a very meaningful tool distinction. I hope that's at least somewhat helpful.

Yeah.

In the gray. Gray shirt in the back or blue? Yeah. You turning around?

Yeah.

Someone actually just said to me yesterday that he. He wants to coin something that would be the equivalent of Godwin's law. For those who know Godwin's Law, it's like if you're in an argument online, whoever brings up Hitler loses the argument. It's over. At that point, he Was like, we need something like that for Steve Jobs because for startup people like we don't know what to make of Steve Jobs. We never know. Everyone wants to be like if Steve does it, I do it. I know a lot of people who do what I call putting on the black turtleneck. They're putting on a show to make themselves look as much like Steve Jobs as possible so that you'll want, want to invest in them. So here's the true answer to that question, which is that I don't know how product development really works at Apple and neither does any person I've ever spoken to actually know what happens behind the wall. It suits Apple's purposes. Great to have us believe that Steve just thinks up the idea in the bathroom one day and then the next day they launch it and everything turns out great every time. But I don't believe it. Now I think the more instructive example from Apple Computers to go look at what Apple Computer was doing not when they were multi billion billion dollar company but when they were literally three guys in a garage with the scruffy hair and the sandals. And in those days they couldn't afford to be super secret lawns. I mean they were doing the kind of techniques that we're talking about now. Are they still doing those techniques but behind closed doors? Probably. But I think the honest answer is we don't know. So one day someone will write the definitive explanation of how it worked and then we'll all know and listen if there's an alternate method and it's better than this one, it's more scientific and more rigorous, like do that.

So I want to ask you a.

Quick question because you brought up the discussion about businesses that have basically two sided networks.

So Facebook for example.

Obviously I think it would have been more difficult for Facebook to launch just a shell front page saying sign up here if you're interested or looking for more details.

Because really the beauty of value of Facebook.

I'll take a stab. But I'm going to invite Tom to take a stab if he wants to because we've got a world's expert on two sided market sitting right next to me. You got to remember, people forget what the first version of Facebook looked like. Facebook version one had exactly one feature called poke. That was it. So minimum viable product is not about like just doing a smoke test landing page. It can be about doing a very simple experiment. I mean that, you know the diary of him in the movie the Social Network where he's talking about his different experiments as he goes through and figures out version one. Like the dialogue that's being spoken there is from his real live journal diary that's public. Like, we know exactly how long it took to build version one of Facebook. We know exactly what he was thinking at the time because he was live blogging it. So it was like a classic mvp, but it had enough functionality in order to unlock that behavior. And the pattern of Facebook, I think is really, really the paradigm for doing this kind of testing in a two sided market, which is you try to find an MVP that's small enough that you can launch it in some kind of limited scope. That's a place where the connections are really, really dense. So like only at Harvard I meet so many entrepreneurs who are currently working on the next Facebook that don't have the courage to launch only at Harvard or the equivalent small thing with only poke because they see Facebook the way it is today day and they want to copy that. So do professor Couldn't do.

I couldn't do better and won't try.

Thank you.

Thank you. One more, I guess.

Yeah, right here.

Thank you.

But it has been difficult for me to get a good understanding of which.

So I'd like to ask you to.

What extent has it been adopted by.

Entrepreneurs both in on the west coast and also around the world? Thank you for that question. I think it's a really hard thing to answer. I know that the movement has grown. We have groups like the Lean Startup Circle in Boston, in well, north of 100 cities now that get together regularly and are really pushing these ideas. But like I said, I told you before that winter is coming. Remember that now is a great time to be an entrepreneur. Everything easy and summery and money's flowing and everywhere and it's exciting and cool and Lean startup, especially because it's so new and it hasn't had any. We've had no serious challenges really. You know, it's just our own ideas and we're just trying to put them to practice. So there's a lot of people doing what I call success theater, which is the work that you're running into, which they do everything to make it seem like it's working. And in fact, in Silicon Valley, it's even worse. A lot of VCs, this has stopped happening recently because I started telling people, please, please don't do this. But for the first year of Lean Startup, every VC I knew would come to me and be like, could you please make it stop? I'm like, what? They're like, every entrepreneur is now coming in to our office with the same old crappy pitch, but now it's lean. It's got MVP and pivots and quotes from Eric Reese and Steve Blank all over it. And it's the exact same pitch it was before. They're just lean Washington. I was like, oh, God, no, don't do that. Don't use the jargon. Get the results. So to me, the number one indicator of someone who's following the methodology is that they're able to say at one meeting what they're going to have accomplished by the next meeting. Results wise for customers. And then at the next meeting, they're able to tell you either that they accomplished that or that they didn't. And here's what they learned. Most entrepreneurs cannot do this simple exercise. If any of you have ever invested in a startup, you'll know exactly what I'm talking about in one board meeting. Lifetime value is the most important thing in the world. And we're going to drive up lifetime value. And in the next meeting, we got this great new feature. Check it out. You're like, what happened to lifetime value? It doesn't matter. Eric's law of Google Analytics. At all times, no matter how badly you are failing, there is at least one graph in Google Analytics that is up and to the right. I have used this to my advantage as an entrepreneur many times. It's like, it's midnight, all your graphs are down. You're like, oh, my God, we got to show some progress to our investors or they're going to be so mad. Like, they're going to go find something good or like, I got a razzle dazzle feature or whatever thing. Desperation move. So when you see someone doing that, you know, they believe in vanity metrics, you know, they're not being scientific, they're just following the ship. They're obviously not doing this. So I would actually rather if I was embedded in a meetup group in a lean startup community, I would rather have a smaller community of people who are really doing it so we can have serious conversations and hold each other accountable than a much more successful speaker series. And so I would say try to identify the few people in your network who really, really are willing to put their ideas to the test, who can admit when they failed, who aren't afraid to pivot, but, you know, are not necessarily so proud of the fact pivot is not necessarily like an enjoyable experience. People are like, yay, pivot. It's like, really? Do you understand what that word means? I don't think it means what you think it means. So, anyway, I hope that's helpful. I just want to say a huge thank you to all of you for coming, but especially to the people from the community, from the startup community who've made this possible. I feel like every day I wind up taking credit for your work. You are the people that are really advancing the state of the art. You are the ones who are helping us discover what entrepreneurship is really about. So I just. This is a thank you for making all this possible.

And before I thank Eric, thanks as well to everybody from outside the Harvard community that came, the Harvard folks, the MBAs in particular. I hope we'll all move over to Kresge. You got a bunch of practitioners here who are doing it. You can learn a lot from them for sure. In Silicon Valley, the ethos is pay it forward. That these folks in the startup community in Kendall Square and beyond got a lot of benefit from folks who coached them and advised them. They're here, they're available to you. You. You're all learning to approach them with new humility and gratitude. So let's keep that going. Eric, thanks to you for everything you do for us and everybody. Let's move it over to Kresge. Thanks.

Thank you.