How To Build A Product People Love with Phil Libin

I'm the CEO of the Founder Institute. We'll introduce Phil in a moment. In case you don't know what the Founder Institute is, it's a squirrel that's lost in the woods of Kyoto. No, just kidding. It's a program that helps you launch your business. I've been referring to it lately as a startup operating system, right? When you get going, you need to figure all these different things out. We lay all the stuff that you need to figure out. We bring in great minds from the local community like Phil, and they all help you to get your business going, to start your company. Um, and we have chapters in over 120 cities around the world. We're running in approximately 75 cities right now. So if you'd like to learn more, go to fi.co You can join any one of the programs. It's really awesome. And if you want to lead a program because you're more seasoned, uh, there's also opportunities to do that by going to fi.co/lead. I want to thank two quick, uh, partners, uh, that have helped make this possible. So All Top Startups, it's a great blog covering startups and resources for entrepreneurs. The link will be provided. And Foundsy, a weekly newsletter covering what founders are reading today. So if you're a founder, those are two great resources. But more importantly, we have a— and the purpose of the webinar today, we have a great resource sitting right next to me in the forest of Kyoto, Phil Libin. For those of you who don't know, Phil is a serial entrepreneur. He came out of doing a really hardcore tech company in security and then went on to take a CEO spot and co-founder position in Evernote, which through his leadership grew to be one of the most significant companies today. I use it every day religiously. It's one of my favorite products out there today, so he has successfully built a product that we love. He's recently moved on. Phil, why don't you tell us what you're up to and add anything you want on your background.

Well, I am now an executive chairman of Evernote, so I hang around and make strategic decisions from time to time, but I am not handling the day-to-day operations of the company. We brought in a great CEO, Chris O'Neill, to do that. I think the company needs to go through a period of growing up. And I am a partner at General Catalyst, so I have joined the world of venture capital to start a bunch more cookies. Great.

It sounds like your earpiece might be rubbing against the scruff on your face, maybe. Is that possible? So let me ask, so at General Catalyst you're doing more than just investing. Are you allowed to talk about what— or maybe it's the scruff on your shirt. Are you allowed to talk about—

It's just my scruff, man. All right, how's this?

Much better, much better. Are you allowed to talk at all about some of the things you're going to be doing at General Catalyst, or is it still a little bit under the radar?

Yeah, I can talk all I want. I'm going to be— so the great thing about General Catalyst, the reason I got really excited about it, is it was kind of a Goldilocks opportunity for me. When I sat around thinking, you know, what do I want to do next, I thought, well, I I want to launch more companies. I want to start more companies. I probably don't want to run any more companies. I think I've been CEO of something or other for about 19 years, and I think I've gotten running companies out of my system, but I very much want to make companies. So I was going to start an incubator. I was basically going to, you know, sit around with some friends. Yeah. I sit around with some friends of mine and figure out how to launch some companies and get, 3 or 4 things going and figure out how I wanted to spend my time. But then I started looking at what VCs actually do for a living and became really interested in that. And General Catalyst kind of combines the best of both worlds. So it's a large early-stage venture fund, but we also hatch companies. So some of the best known General Catalyst companies like Kayak are actually created inside the four walls of General Catalyst by the partners and then bring on co-founders. Build a team and kind of hatch it. So I'll be focusing on that. I'll be doing early-stage investments and hatches.

Wow, so that, that a lot of times that's called company building, and it's a great segue into the topic today because when you're building a company, like many of the people who are joining us today, the first thing you need to do is build a product that people love. Now I know a lot of people have just come on recently, so I'm going to point this out again, we're going to be covering a number of topics and if you have questions you can pop them in the chat and we'll queue them up as we go through them. So you know, one of the things about building a great product is, you know, what's a characteristic that is or makes a great product and you know, you famously say I have a 2-step revenue plan. Make a great product and then charge money for it.

It's sort of like the underwear gnomes, like collect underwear, make lots of money.

So, but what constitutes an amazingly great product in your mind?

You know, the number of people who aren't going to get that South Park reference are just going to be completely confused by what you just said. You just kind of glossed over the underwear gnomes, but okay. Is that, I guess it's just well known.

So yeah, you can look it up.

That was a 3-step plan. Mine's only a 2-step plan, so it's actually a simple pitch.

So what's an amazingly great product in your mind, Phil? Like, how do you— so do you know when you see it? How do you define it? And you're going to be doing it now. You're going to be making more of them. So how do you get started?

And hoping to help great founders make more great products. That's the real scalable— part of this. You know, I think, I think it's hard to identify specifically, but I think there's a couple of things that the products that aspire to greatness have in common. One is great products have a point of view, but they're not neutral. When you're using them, you can tell that the people who made them like had a particular point of view about how the world was supposed to work, how you're supposed to use the product, what makes sense, what doesn't make sense. It's not a you know, oh, do whatever you want, everyone is like special and unique. There is a particular point of view of how the world is supposed to work through this product.

Interesting. So, you know, maybe if I bring up an example from my own experience. So I've been using RelateIQ.

Yeah, fine product.

Which is now called Salesforce IQ, and with a lot of CRM systems It's like data entry, right?

You're like, uh-huh.

And with them, you just type the name of the company and it sucks all the data from the internet. It sucks all the data from your email, and it's like all the contacts, the name of the company, all the company information are there right away. And it's almost like their point of view is, you know, in a CRM product, you don't, you don't need to be doing data entry. It's pretty clear that that's what their view on the world was. Is that an example of what you mean?

Yeah, I think so. And I think that's probably an example of a couple of other good characteristics. One is it isn't the user's job to do stuff. It's the product's job. It's the computer's job. So I think really good products understand that they shouldn't rely on, on, on to get data, enter, to catch mistakes. As much as possible, that should be up to the product. So things like filling out forms, as a class of stuff that's about to get disrupted and go away, it's everything that feels like filling out forms. And Salesforce IQ is definitely a good step in that direction. But there's going to be lots more, there's going to be like another 100 successful companies are going to be created over the next few years that are basically all about getting rid of the behavior of filling out a form.

Well, I mean, you did that with Evernote. You can take a picture of a business card, you can take a picture of a whiteboard, you can take a picture of a wine label, and it sucks all that data in and figures out what it is without you typing. Typing it.

Yeah, we definitely took some good steps in that direction and there's going to be lots more stuff coming.

So what do you think, so that's one thing you think is a trend, what do you think is lacking today in design and user experience? So you got a product, you try it, you're like this is bad, what are some of those characteristics that you define right away as problematic and need innovation? Filling out forms?

Yeah, I mean, filling out forms is actually like— we can talk for years just about filling out forms and about the de-formification of the world and how that's gonna go. Like, every industry is going to go through this phase of just getting all the forms out of the system. You know, you can look at that as like— I mean, that even happened with like a like online dating, right? It's like online dating a few years ago was basically filling out forms, right? You would go on, I don't know, whatever, eHarmony or something, and you would like fill out your tax form, and then you would get back results. And it was like you were kind of like dating a spreadsheet. Like you would just compare forms to forms and see what was up. I never actually did any of this, so this is in research that I've done since then. And now, like, there's no more forms, right? Now you're just kind of swiping. So like everything is going away from the act of forms. Like the act of forms is very— the act of filling out a form is not cognitively ergonomic. Like it isn't how your brain wants to work. And we were forced into it in every industry because of the limits of the technology. And now those limits are coming off. Every industry is going to go through this process of getting rid of forms everywhere. That's a major, major step.

Let's take that a little bit. Amazing products, where is the first time you encounter a form? It's usually when you—

right away, right?

Like, log in, right? You get an app, you get a website, it's like, log in, fill out the form.

You're like, what?

What do you think— let's tangent away from negative and go to positive. What are some of the best onboarding experiences in your mind that you can see for mobile or for web in the modern world that gets a user really engaged versus like, hey, try this, fill out a form, right?

Yeah, I mean, that's tough. We spent a long time at Evernote and are still spending a lot of resources really trying to just optimize those first few steps. You know, when do you ask for, password? Do you ask for it twice? Do you make someone type their email address twice? Can you automatically get the email address? Like in Evernote on Android, we can actually pre-fill in almost everything. On iPhone it's a little bit harder to do that. So those experiences are definitely something that's worth iterating over. But the sign-up and sign-in experiences at least only happen once for most users. Once you're kind of in the product, the rest of using it should be really pleasant, should be really beautiful. I think a lot of products, when you use them, you very clearly understand that no one involved in the creation of this product was in love with it. No one involved in creating it and making it and using it and testing it was like, "Yeah, this is amazing. This is beautiful.

I love this thing." Okay, let's dive into that for a second. By the way, we just crossed 1,000 viewers, so you are a very popular man. Woohoo! Party! So loving, you know, that's hard to do, right? Like, got an organization, someone drives into work, they get to work, they've got problems at home. How do you make your team love the product, right? I think at some point they love the product, but They've got, again, family, commute. How do you make them love the product?

That's why work is like an oasis from all these things, man. It's like you come into work and you leave your problems at home behind and you just focus on a loving relationship with the product that you're building and it loves you back. It's the cure to all of life's problems. It's not made worse by it.

So, all right, well, that's an interesting way to look at it because most people are like, work is work, play is play. So what you're trying to say The best way to get someone to love the product is make sure that they love their job.

Yeah, I mean, you really can't beat more work out of unhappy engineers and designers. The happiness at work ultimately is the most important thing. That determines not just your current product, that determines your next 100 products. And companies are getting better at that now. Happiness at work comes in many different forms, and obviously there's lots of difficult times at work as well. And it's not that you're necessarily happy with every single step or every day or everything that you're doing, but a sense of fulfillment, really understanding what you're working on and why you're working on it. Like understanding that the work that I'm doing right now is having this very direct effect on the world that I want.

And I think—

Why?

Understanding the why.

Right, like everyone in the company should understand why they're doing something. And if you— once you go through a long period of lots of people in the company not understanding why they're working on something, then I think you start to— then you stop being a startup, you become something worse. And most companies are perpetually at that stage, and all companies are at that stage from time to time, but I think it's really worth fighting to try to prevent that from happening.

So right now we probably have 1,000 and change people on this call, most of who of whom are going to be early-stage entrepreneurs, right? Maybe they have themselves alone working on the product, maybe they have a small team of up to 5 or 10 people, but it's going to be primarily that group of people. And how do they keep the— you know, you've been there multiple times in your career and you're going back there now. How do you keep such a small team passionate about the product? So you let them know why they're working on it, right? And you remind yourself why you're working on it. What are some other tips that you might give these team members to, or these small teams to keep passionate?

Well, so at Evernote we kind of cheated on this, which we didn't do in my previous two companies, but now I'm convinced, like now that I figured this out, I'm like, oh yeah, I don't know why I would ever do anything else. Is we really did make the product for us. People talk about that a lot, but it's kind of an important thing. Like my first company, we were making e-commerce software for big retailers. And fundamentally, like at the end of the day, we don't really— like I only care, you know, so much about what big retailers want. It's like I'm not a big retailer. And so like, yeah, I want to do the right thing for the customer, blah, blah, blah. But ultimately, it's like you wake up and day in and day out, you're trying to say like, what does the customer want? What does the customer want? It gets pretty demotivating. And then our second company, you know, we made security software, like you said, for governments and banks. And like, that's not— like, that is important work, but it isn't for us. And I, you know, I became an expert after like 6 years of this. I became an expert on like Department of Defense procurement procedures, like that kind of stuff. I didn't want to know any of that. At Evernote, we really did make the product for us, and that, it honestly feels like cheating, because if you're making it for you, you can at least be— yourself, which is hard. But if you're honest with yourself and you're making the product for you, you can know when it's getting better. Like, you can know when it's worthy of, like, really being in love with this part or that part. You can know, like, when stuff's gone backwards, because, like, you are the target audience, and the iteration cycles are much more efficient. If you're not making the product for yourself, you're constantly in product-market fit mode. You're constantly trying to iterate to find the right product-market fit. You're making it for yourself, you at least know when it's getting better.

Right, and you're like, boom, this is perfect, right?

Yeah, and so I think like since the tech world is becoming more and more of a meritocracy, having a really great product is by far your best chance of succeeding at this point. Now, that doesn't guarantee success. But it's by far the best shot you've got. All of the other stuff, your channels, your marketing, distribution, logistics, like all of that stuff, you get massive leverage if you have a genuinely great product. And if you don't, like, it almost doesn't matter what you do in those other things. That's only been true for the past few years. Like, that wasn't always a true statement. That's only been true since the advent of like social media and app stores and things like that. Um, the way the world is wired up right now, making the making the best product is the most important thing you can do. And making it for yourself, especially if it's your first time around, is by far the easiest way of having a shot at making a great product. So I would just say, like, if anyone's actually doing this for the first time, like, pick an idea that's for you. And you're probably like 3 times more likely to succeed than if you're just doing something abstractly for other people.

You did a perfect segue into the next topic, which is how do you— pick an idea, how do you get going? So pick an idea that's for you is the easiest hack to get started. And then, right, so you're a first-time founder, you have a small team, make sure that the product resonates with you.

Of course, that's not—

doesn't, you know, I don't want to— so now like 500 of the 1,000 viewers are like, crap, I'm dumping my enterprise software idea. But you've done an enterprise company, so for half the audience that isn't necessarily building a product for them, is there a way or a hack that you can give them so that they can find that love? Like, can you love enterprise software for governments? Like, I don't know, you did it.

I've never done that.

It's hard for me to say.

Yeah, I mean, look, everyone is like, you know, everyone's a weirdo about something, right? Everyone is passionate about, about, about something. And there's obviously people who are super passionate about the, uh, about enterprise software, uh, as they should be, right? Because making good software for people to use at work is like very high impact for the world. Um, in fact, like, this is, this is kind of a good test for, uh, for founders, for me, for, uh, as an investor. So, you know, I invested in, I was an angel investor in ZenPayroll, they're just called Gusto now, they used to be called ZenPayroll. And, you know, this is a company that makes basically payroll management software for like SMBs.

Whoa, I love payroll.

Well, so funnily enough, right? So I heard, I thought, you know, I was gonna get a pitch from this guy Josh and he makes like payroll software and I'm like, yeah, this is gonna be good. And this guy comes in and he like pitches me about payroll.

Loves payroll.

Like, he is passionate. So much so that after like 20 minutes with him, I'm like, I'm really excited about payroll. Like, this guy has like convinced me that actually like one of the highest impact things you can do for the world is make it easier for like people to start companies and to deal with like— take all of this horrible crap about payroll and benefits and simplify it and like make the relationship with your employees better. And he is passionate about it. He knows everything about it. And he like— he basically took me from going like, I would rather chew my arm off than know about payroll software, to being genuinely excited to like be involved with this guy about payroll software. And so it was like an obvious— it was an obvious investment.

So, so you can be passionate about more mundane things. You just got to make sure— so your basic lesson here is if you're not passionate about it, you're not going to build a good product. So maybe you should refactor what you're doing. How So if you are passionate about it, you can of course test it with yourself, but what are some other things that you might recommend founders do to test that they've got it right? So you're passionate about payroll, you build it, you're like, this is amazing, I have a point of view on the product, but now I've got to show it to someone else, right? What are some recommendations on testing?

Well, and I think for Gusto, right, like for them it kind of became self-fulfilling, right? They were building like software to make it easier to run small companies, and like they were a small company at the time, so like that loop got closed pretty fast. And I think, I think that the enthusiasm was authentic, right? Like if there's, if there's 500 people on this call who are making something that they don't really care about but they think it's a good market fit and now they're trying to like psych themselves up for it, like that's probably the wrong chain of events. Like if you, if you genuinely really, really care about a particular subject area and you think that you're like the world's expert at it, then yeah, then it's a good thing to do.

So can I just go follow that? So if that statement's true, then you probably know— so if you're like in love with enterprise security, then you probably know enterprise security experts. So first you test it with yourself because you love the market, then you can go to your buddies in the field and test it there. Do you have any other advice on sort of moving outside of that domain? Michael asks like, what are some other specific metrics that you would track like cohort analysis, but I'll turn it over to you.

Yeah, so you gotta— so I think there's a few stages, right? Really, really early on, like before there's like, before there's any metrics, when it's just an idea, it's important to get the idea like out of your head. As, as fast, like, as soon as possible, right? Because you go through this phase where you're kind of getting excited about it, you're not really sure whether, like, it's only in the artificial world inside of your head where you've— that you've constructed. And you can kind of talk yourself into anything. And so you start talking to a couple of people who are potentially co-founders, and that isn't necessarily any better. Like, you can get a small group of people people just like huffing the same fumes and, and, and, and convincing themselves that like their idea makes a lot of sense, like in their heads, because they're really evaluating this as not so much a product, but like, what is this company going to be like to build? You know, what's it going to be like to work at? Oh, it would be so exciting to do something. So you gotta pretty quickly just like test the idea outside of your head on real people and just see if there's like a real level of enthusiasm. And that's as simple as, like, early on, it's just as simple as, like, talk to people who you aren't trying to recruit as co-founders and just see, like, someone with no stake in it, do they think that it's actually genuinely great?

Um, and David's asking, like, but even then, you know, how do you remain objective when you start getting different kinds of feedback about your vision? Because a lot of times people to your point about the payroll founder, they're so passionate they create a bit of a reality distortion field. And so they're like, payroll is amazing, payroll is amazing. And you're like, payroll is amazing, man. And then you're like, wait a minute, payroll is not amazing. So like, how do you remain objective?

Yeah, but you don't— I don't think objective is what's important, right? Like, you're not— this is not a rational decision. Anyone making a rational decision about whether or not they should like quit their job and start a company should clearly not quit their job and start a company.

I agree with that.

Yeah, I mean, this is like the reality distortion field makes reality. Like the reality now is that payroll is amazing. Like repeat after me, it's all sorts of actually genuinely awesome. Like any company that I start, now, or that I'm involved in starting, like, of course I'm going to use this stuff. And every company I've started in the past, like, the payroll stuff has sucked and the benefits stuff have sucked. So like, it's actually a huge— like, it's, it's a huge benefit to do this. And like, the reality distortion field is what makes reality. So I'd say like, I wouldn't worry about being, um, objective. I would just say like, talk to a bunch of people and then just see like, are you still genuinely in love with this idea? Are you still infatuated with it? Or are you kind of playing defense? Are you now looking for reasons to really like it because you feel like you've got some kind of commitment, you've already put work in, and you're just looking to justify it? Or like every day, are you actually like more in love with this concept? And at least for the first few months in the life of the idea of the company, you ought to be getting like more and more excited about it every day. So even if you talk to people and they hate it, You talk to normal people and they're like, yeah, this is stupid, it doesn't make any sense, like blah, blah, blah. And you get that feedback and you're like, oh, now I'm even more excited because now I've just understood that like these people don't get it. Like, that's fine. You're not necessarily trying to like average out what all the feedback is. You're just really trying to like test the idea outside of your head, like bang it around and then see if after a few weeks or a few months, like you still want to spend in effect the rest of your life doing this.

Really, let's quantify that a minute because I think you're really sage on what you're saying, but it's hard to quantify, right? Let me try and take a stab at it and see if you want to add anything. Essentially, and I know this from personal experience, you have an idea. You think it's great. You go talk to your buddies. They like you, or you talk to people in the industry. They like the industry, so they're all going to be like, "Yeah, it's awesome." So that's not the best feedback. But as you start to talk to more and more people outside of your direct circles, the gauge is really you. Not— so you almost want to take like, how passionate are you? 1 to 5. If it's like going from like a 4 to a 5, great. If it's going from a 5 to a 4, that's bad, right? That's a sign that, you know, if it goes from a 5 to a 4 after 30 days, it's going to go to a 1 after 60 days.

A year, right? That type of thing. I think that's right. And yeah, it's gonna go to like a negative 10 as soon as you've got like employees and investors and like your first problem. So yeah, like you're not gonna get more excited about it as, you know, as you actually start building. But the feedback that you're going for is not like— you're not necessarily describing the product in great detail, right? Like you're like— and you're certainly not asking people what they want. That people have no idea what they want. You're really trying to get a sense of like, is this an invented problem? Like, did I just make up this problem and like convince myself that a problem exists where really it doesn't? Uh, I've actually like a couple of companies that I've been involved with, I've been guilty of exactly that. We like, we like fell in love with some piece of technology, and because we fell in love with the technology, we like invented a problem to go with it and convince ourselves that this was a real problem. But really it kind of wasn't. It took a while to realize that actually like this just wasn't that big of a deal. So you're really trying to quantify.

Yeah, this is a global conference, and so let me throw something out. What you just said in Silicon Valley—

and sorry for the police car—

there's no crime. Oh wait, in Silicon Valley that's called a hammer and a nail problem. So a nail is the actual problem. Let's say it could be doctors need more efficient scheduling, and the hammer is a, a solution, like a scheduling software. So a lot of times people build a hammer scheduling software, but they don't have the nail, which is doctors, for example, need more efficient scheduling. Or a better example, a lot of times people in invent technologies like, oh, I can route packets faster, and it's like everyone needs faster packet routing. It's like, well, maybe not.

So the hammer and the nail problem is a very common dilemma in Silicon Valley.

But sorry, I didn't mean to interrupt you there. So a lot of people, at times people invent the hammer, but they don't have the nail. Yeah.

And then I think the way to like way to cognitively protect against that when you first think about it is like, don't think of what the product is. Like, think of what, what's the actual problem. Like, identify something that sucks in the world that people will acknowledge sucking and that people are willing to like go through a significant trouble or expense to fix. Like, that's probably 80%, right? 80% of this challenge is like, what's an actual problem that, that people in your target market, meaning yourself, people like you, if you're building for yourself, would actually be willing to like adopt the new product and pay for it to fix. And then once you've got that, you've got the relatively simpler task of actually designing a fix for it.

Okay, so now, now, so let's go on to how to scale because Mike's asking a question. So you love the product, you talk to a bunch of people, they validate the idea, You maybe build the first version of it. I'd like to talk about a little bit on the first version, but you put it out there. And how do you know that? How do you measure love? Right? How do you know that people love it?

Right? So, I mean, if you want to talk about specific metrics, I'd say probably the most important metric. So there's like a little basket of metrics that are really important. Probably the most important one is always retention. That's kind of what gets the closest to love, is like, do people keep using it? Like, you can say all you want, if your retention sucks, then, you know, pretty much nothing else is going to go right. And people are probably not in love with your product. So the first, like, real honesty check is retention rate. And that's more important than growth and MAUs than, like, conversion, it's just retention. Like, if you build something that people are genuinely, like, still using, that's a really good sign. And how do you measure retention? Well, you know, there's a bunch of different ways, but I would say it really kind of falls into two buckets. There's the short term, which is what percentage of people who, like, try your product actually wind up, you know, still using it after a few hours or a few days or a few weeks or whatever, whatever that initial time interval makes sense. And then there's the long-term retention, which is of the people who made it through that first time window of like getting to know it, how many are still using it much further? Of those two, the second is the most important.

Right, but you won't know the second for a long time, right? Are there any cues that you— so let me ask Evernote, you know the data, don't reveal anything you can't, but for example, if someone used it 5 times in one day, did you know right away these are long-time users? Were there cues in the early usage that led to it, you know, to say that person's going to use it a lot over time?

So, you know, we didn't actually see a whole lot of stuff like that. Like a lot of companies, you know, Facebook and others talk about like some magic, you know, if you do this 3 times, then you spin around and you do this thing twice and you throw salt over your shoulder, then you're going to convert it like X rate. And we never, like, we never got to that level of specificity. We did have very good numbers for short-term and long-term retention. And for us, long-term was like, it was basically starting in a month. So it doesn't take that long. Like for a general product, I would say, you know, count, look at the retention from day 0 to day 30 as like short-term. So how many people are still around, you know, a month later after they first started? And then long-term is like of the people who are around a month later, how many of them are still around 2 months later, 3 months later? Whatever. So it doesn't take that long. A few months and you'll have it. I mean, yeah, if you're looking for this in 10 minutes, then no.

Carol's asking, you know, that's great for consumer-facing products. In the enterprise world, retention doesn't always equal love since users might be forced into a product. Any thoughts there?

All sorts of inappropriate thoughts, but I'm gonna self-filter. So you can always get meaningful retention numbers, right? If people— like, I'm not saying like measure shelfware, I'm not saying measure like how many like paid seats you have, because yeah, that may be a total nonsense number that has to do with how much is being bought. But you could certainly measure usage. You can certainly see like, okay, are actual human beings like interacting with it? If they are, that's already a good sign because even if they're interacting with it but they don't like it very much, if they keep interacting with it, that's not bad. Usually people won't, like literally won't do what they don't like, especially if you're talking about knowledge workers, really can't force them to do very much in the long term. So you can still look at usage, although not necessarily at things like active, you know, like paid seats or anything. And you can look at things, you can look for other signs that people are using it like more than their absolute minimum. You know, are there, what are the different times of day are they using it? Are they concentrating all the usage in like one 10-minute block because that's like when they're like, okay, I gotta get this out of the way. Or are they like spreading it out throughout the day? Are they using it at all on weekends or something? Like you can try to say, how do I actually know that people like this thing? Thing and look at the retention. Now there's lots of other metrics like NPS— That's a great point.

You know, can I just— sorry if I interrupt you on occasion— when you say something like that, that's super important because I don't think there is a silver bullet. It's really if you build a product, your product and its love is something that you need to think about what to track, right? Because to the point that brought up about the enterprise product by Kryll, If, you know, if someone uses it throughout the day, if it's like a certain type of product that could be used throughout the day, but no one is using it throughout the day, it's probably a sign they don't love it. Right. So it's really product dependent. And you need to say to yourself, let me paraphrase that. What are the metrics for this particular product in this particular industry that would show they love it?

Right?

Is that accurate?

I think that's right. I think like, the thing about silver bullets, right, is like every modern werewolf movie ever made, the silver bullet doesn't work. It's like, the silver bullet is the thing that doesn't work. It's the thing that you sort of try to use and the werewolf's like, "Ahaha, that doesn't really work." That's kind of what the phrase means. So yeah, none of this stuff is like the single thing that's going to solve anything. You're really looking at a basket of metrics to try to give you a good idea. And at some point you run out of the marginal utility of any particular metric, and then it doesn't make sense to like keep drilling down further into it because you're not really trying to like fool anyone. So I would say like, you know, construct a plausible long-term retention metric that like sounds right to you, like honestly, that sounds right to you. That isn't like where you're in your brain not aware that you're tricking something. A lot of times, like, if you're a founder and you're making metrics, you know when you're being dishonest, right? Like, you know, and it's not even being dishonest, like, you know when you're trying to pick something to make it look better. And let me— like, yeah, don't do it, or at least don't do it for yourself. Like, have, have a set of metrics that when you've made them for yourself, you don't get that little tingle of like, oh yeah, if I pick it this way, it's going to come out looking a little bit better. Like make it look worse and then still have it be good.

Now you've jumped over to the investor side though, and you know that there's like the standard, like, DAU, MAU, like, LTV, blah blah blah. So DAU is daily active users, MAU is monthly active users, LTV is lifetime value, CAC is cost of customer. So there's these like investors like, what's your CAC, what's your LTV? You know, these are the standard metrics of venture investors, and you're saying some of them are okay. What's your view on them? They're certainly not useless, right?

No, of course not. So in general, if you're now— you're sort of jumping around a little bit. So if you're talking about invest, like how to deal with investors—

well, I mean, we're talking about metrics of love, and a lot of the metrics of love in the investor's mind are like CAC, LTV. If those look good, that's love.

It's good to know what those are for you. You know, it's like, go ahead and read the Wikipedia pages and, you know, and compute them just so you know. And they're important for good reasons. Like, investors think they're important because there's actually, like, lots of pattern matching where they are important. So it's useful to understand that. In general, like, what investors are looking for at early stage, look, I guess let me just speak from the first person. What I'm looking for as an investor in the early stage is 90% infatuation and enthusiasm and mastery by the founders. No amount of metrics, especially standard metrics, is going to overcome if I actually don't get goosebumps when I'm looking at— when I'm talking to the founder and saying, yeah, this is the person. That I want to be involved with. Like, this is the right person to solve this problem. So as a founder, anything you can do to really get yourself psyched up and to project and to articulate how much you're in love with this and why, and that you've really understood it, that's going to go much further in early-stage investment, at least with me and most investors that I know, than anything else. But of course, you should also have, you know, you should know what the other metrics are. But if you don't, it's okay. Like, if you, if you go for an A-round pitch and you don't— that somebody asks you what your CAC is you don't know what that means, like, that's not a disaster. Just like, say you don't know what it is. People will explain it to you. You'll calculate it. Like, no one's going to penalize you for not knowing.

Well, assuming that you're kicking ass in other things. JP, one other metrics question, then I want to go to the first version of the product and some scaling questions. JP was asking, what do you think about a metric like Net Promoter Score?

Yeah, NPS is good. I like it. It's, for people who don't know, it's you basically ask people to rate from a scale of 0 to 10 how likely they are to recommend the product to a friend. And you know this is super scientific because it's an 11-point scale, okay? 0 to 10. So anything with an 11-point scale, you're like, yeah, there's some real science involved in this. But despite the fact that it's complete like marketing flop, it's actually, pretty good and pretty useful. So I would start measuring NPS and doing it rigorously pretty early on and doing it for all sorts of stuff, doing it for the product, doing it like internally for the company. You can measure NPS of your employees. You could do NPS in a lot of ways. I have a concern that it's going to start getting less useful because when you think about it, NPS kind of only works when you don't— when like the respondent doesn't know that what's being done to them, right? Because there's like, there's a trick to it, right? The whole point is like you're asking this 0 to 10 question, but only 9s and 10s count as a good, and everything else is neutral or bad. And so like the whole reason why it works well is that most respondents don't know the psychology. If I ask you like on a scale of 0 to 10, how likely would you be to recommend, um, you know, to recommend, uh, to your friends that they you know, that they get this product, and somebody says, ah, you know, like 7, like actually they're never gonna recommend you, 'cause like 7 is not, they're not enthusiastic enough. If they say 9 or 10, then they're gonna recommend you. If they're like 7 or 6, then they're like, at best they're gonna be neutral. And if they say, you know, like a 4, then they're not just not recommending you, they're like actively on the internets, you know, venting their displeasure. So I actually like NPS a lot. I'm afraid that it's gonna get less useful now that everyone knows what it is. And like you see it all the time. Like I landed in like Munich or something the other day and I got a text from like some, you know, when my phone went into range of some, whatever the operator is over there, and I got a text saying like, oh, please rate your experience on a scale of 0 to 10. How likely are you to tell your friends to use whatever? So it may become less useful, but for now I would still actually use it quite a bit.

Let me ask, we got a few more minutes till we're going to do some live pitches. Let me ask a question about your first product, knowing you got it right and scaling it out. So, you know, MVP, MVP, MVP, minimum viable product. A lot of people take that to mean like put the shittiest version of the product you can together quickly and show it to people and see if they like it.

I think, yeah.

So talk to me like your first product, get it out there. and then when do you know— let's spend this next 5 minutes on what that product should be and then how do you know when to start scaling your team and everything else.

So I think of it as the MAP, the minimum awesome product.

I agree with you, by the way. Amen, brother. Everyone take that advice to heart. Not the MVP, the MAP. Minimum awesome product. Sorry.

I think like— because if If it— I mean, like, I'm not trying to be cute about this. If it isn't great, you're not testing for anything. It's only going to succeed if it's sharply great, if it's actually awesome in some way. If it isn't awesome, like, if you were the only product in the world, then you can have the minimum viable product and it'll be okay. But you're not the only product in the world. So unless you're testing whether or not you're sufficiently awesome to succeed, you're not testing anything. So the MVP, the traditional MVP framing I think I wouldn't apply. I wouldn't apply in an area where there's a lot of competition, a lot of choice. So I would say if you're building— the way that we always thought about it at Evernote, the way that I'm thinking about it now, is especially since we build the products for ourselves, the MVP, the minimum viable product, is the minimum product that we internally can start using to make the MAP, the minimum awesome product. So it's like we want to get the product internally to the state where it's like, useful enough to us before we never like release it. It's just like, it's basically like an internal alpha to actually start using for real in a way to take it to the next level, to build it so that it's actually sharply good at some things. And then launch it when we've gotten to a state where it's minimally awesome, which means it only has to be awesome in like a small number of ways.

Well, like Tinder was awesome in one way.

Yes.

No, yes, no, and then it could build everything else out after that, right?

Yeah, I think that's right. So I think like a core idea or two, but probably not two, like should be sharply great. And so that probably means one platform only at first. Like it's probably, you know, if you're gonna do mobile, you know, it's probably iOS or Android, but not both right away just because Again, it's minimal awesome product. It's not like— it's not lowest common denominator. It probably means building native. You know, it's very unlikely that you get to awesome by building using any lowest common denominator tools. Like, any of this cross-platform stuff is for the most part crap. If you're building enterprise software that feels like filling out forms, then maybe you can use one of these things. But if you're building that, you're probably doing something wrong anyway. So, you know, in general, I would say it's building native, it's one platform at a time, and it's get to something that's like sharply, like painfully good.

Right, so your MVP is internal, your minimum viable product, it's saying you use internally, then you iterate upon that till you love it, then you have your MAP, which is what you put out to the world, your minimum awesome product, and then when you start, you build metrics that actually measure love, that say how many times a day does an enterprise user use it, or whatever you think is right. When you start to see those scale, that's when you scale the business. Let's do this, Phil. This has been awesome. Let's bring some people on right now, use the last few minutes to hear their pitches, and I know that you are not shy with telling people your views. So let's bring the first one on, hear what they have to say. Whoa, my screen just got a lot bigger. There we go. Hey there, Guara, welcome. Uh-oh, did you say something? Bill, can you hear me?

I can hear you.

Guara, can we hear you?

Say something.

Sorry, I was muted talking to myself. Okay, you've got— all right, we're ready. 30 seconds, let's hear your idea, and then we're going to go and give you some feedback.

Sure.

The idea's name is Fitoku, which is a combination of two words, fit and oku. Oku in Japanese means helping or favoring. So as you can guess, it's all about helping and favoring your own fitness. Built using Salesforce and Heroku, which again rhymes well with Fitoku, it's a social fitness platform for employees in an organization where people are using all sorts of devices like Fitbits, Jawbones, Apple Watches, Pebbles, and they are having all sorts of statistics like step steps, calories burned, sleep quality, heart rate, and all those statistics.

The problem though is that none of these are compatible with each other, and that's a fairly conscious decision by all these companies which made these bands and devices.

So Fitoku normalizes the stats across all these devices, which people in an organization can share with each other. They can compete against each other on challenges.

And it's offered as native iOS and Android apps.

The normalization happens using a Heroku backend, which basically takes the APIs for each of these bands and runs them. Okay, okay, you've already run your— you're a little bit over. We want to try and keep it to about 30 seconds or a minute. I can go first, you can go first. Who wants to go first, Phil? You want me to go first? Okay. You pointed to the forest. Look, you know, we just talked about passion, right? And I did not feel the passion in your idea. Now that doesn't mean that you're not passionate about it, but you know, just the way that you entered into the whole pitch, right? You're talking about the name and it's like Fitbit and Roku and Salesforce, this and that. Especially after what Phil just said, I would open more with passion. I am a passionate health nut. I work out n number of times a week, and I try and run around the office, and I put signs up everywhere that says, "Work out, work out, work out." No one does, so I've built this app that collects all this data and nags people in the office so I don't have to put signs up to get them to work out, and it links in with Fitbits and watches and Salesforce, and it's super awesome for people to work out. Something like that would have— especially after what you just heard from Bill— instead it was more like, remember we talked about the hammer and the nail? That sounded very much like a hammer pitch, right? I work with Heroku, I work with Salesforce, it collects data, stores the data in a database. Phil, do you want to add anything?

Yeah, I think that's right. I think if you were to enumerate words that don't signal passion or excitement, they would be things like Salesforce. Collect data. Not that there's anything wrong with those, and they're very important in other contexts, but I definitely wouldn't lead with that. My fundamental skepticism about this idea is more basic. It's I don't want to work out with people at the office. I find those kinds of attempts of trying to get people at work to sort of do joint fitness things together. I just kind of find those attempts to be sort of off-putting. Now that probably is more to do with just me. There's definitely been all sorts of popular yoga classes and boot camp and whatever at Evernote. I'm constantly seeing people do it. It's just like, it's something that I've never actually felt particularly good about. And I've never felt particularly good about the idea of like, gathering this data and comparing it to data of office workers, like how fit they are and competing. It just seems, I don't know, it seems, um, it seems kind of creepy to me.

Like, it seems like I don't really think there's actually regulatory issues as well now that you brought that up. I mean, how much data are you able to collect comfortably about people's working out without violating some privacy issues in the workplace. Yeah, so let's try the next one. Let's get a few of these in. Do we have the next one that we can bring on? Try and be a little more— show me the love. I'm gonna hold a little timer up.

I'm gonna show you the love for the time. Do we have another one?

I thought— oh, there we go. Are you ready? We're gonna give you— we're gonna try and give you like under a minute. So try and tell us in under a minute. All right, you ready?

Okay, yeah.

So what recently happened is my aunt and I have another friend of mine, so both are new parents, and one of them shifted from Bangalore to Pune, and they are constantly facing a problem where they cannot find the right school for the child. So the problem is that there are hell number of schools in Pune, and parents are super confused about what— which school their child should go to, right? So I was thinking of creating a site that will aggregate all this information, and parents can actually— this will be like a portal where parents can actually review, rate, and, you know, find out which school they really you know, which school suits their child the best. Now the problem with the existing websites that are there on the internet are that they are basically just a Yellow Pages for your school. So this is just a school name, address, and there's some like this says this school is great, this school is not great. So, you know, that kind of information usually tends out not to be useful. So I wanted to create a community where peer reviews and comments can actually help people make better decisions.

10 me. All right, Phil, I went—

great job—

I went first last time, so now it's your— you're gonna go first this time.

Um, yeah, I think that was really good. Thank you. Uh, I think, uh, it's a good, good statement of the problem, good statement of why, like, what your connection is to it, why you're the person working on it. Uh, I think all that stuff is good. Um, this is something that I don't have a good sense about what else is out there and actually how bad or good it is. You know, it seems like the sort of thing that ought to already exist. If it isn't, if it doesn't, then someone should make it, and that someone could certainly be you. I have all sorts of questions about business model. Like, there's nothing that you said made it obvious about how you would actually make money. So as a—

I had another 30 seconds, so I kept that side out.

No, no, we don't need to do that right now.

Yeah, I knew that.

Let me add some— let's see if we have one more. Can I add some comments on that, Bill? Are you done? Yeah. So the problem I heard, which you didn't really— so this goes back to the reality distortion field issue. But you have a lifetime value issue and a cost of customer acquisition issue, right? So how many schools does the kid go to in their lifetime? Usually like, you know, under kindergarten, kindergarten to X, maybe a middle school, maybe a high school, maybe 4, right?

Then college, 5.

So call it 3 to 5 schools. So, you know, and it's at those decision points of picking the school that your monetization is most likely to occur. Ergo, you have a high cost of customer acquisition but potentially a low lifetime value because you can only monetize them at some key points. So I think in the problem statement itself And then that begs the question, who's your customer, right? Is it the parent that's paying you to— or paying the school and you're getting a commission? Or is it the school that's paying you to get in front of the parent at the right time? So you have classically what I would call customer confusion, which could very easily be fixed in the problem statement, right? So you say, look, Parents have these tough decisions like my aunt and uncle or my cousin, and they spend this much on schooling, so this is a very big problem for them. We believe that at these critical moments where they're making these big expensive decisions, we can come in and help them make a decision and charge this much in this way. Right? I don't know if you would want anything to add on top of that, Phil.

Yeah, I think, I think that's right. I think the way that I would think about this is to figure out What is your average price, what is your average revenue per sort of child enrolling in a school have to be to make this worthwhile? And it feels like that needs to be in the probably few hundred dollar range because of exactly what Hideo said. So then that becomes the hardest thing. It's like what is a product that you're going to be able to charge a couple hundred dollars for to do this? That's what you should be describing. And because if this is a product where you're making per child, you know, 50 cents, then yeah, you're just not going to get the numbers to make it worthwhile. So I know you didn't have time to go into it, but that's how I would think about that.

We got one more pitch, so let's quickly do it. Thank you very much, Tammy. We'll do one last, we'll be a little faster on the feedback. No, this was lovely, 1,000 people. All right, Damian, short and sweet, under a minute, we're going to give you feedback. Get going, my friend. Well—

Can I start? Go.

OK, my company Room is developing Classroom, a mobile and web application to help parents and teachers from pre-K to 5th grade communicate safely, privately, and in real time about the children and their activities.

This application allows teachers to share text, media, and files about the class to the parents, and parents to inform teachers about lateness, absence, ask questions, make appointments. So our first market is France. We have just been approved like an hour ago by the French government to be used in public schools, so it's great. So now we can enter any public schools in France and use the application.

And right now we need about $100,000 to test this application in 30 schools that have already signed up with us, and about $1 million to reach our European goals because we have the same contact in many European countries.

Perfect.

Should I go first? I'll keep it short.

Yeah, sure.

So I thought that actually was really great. Look, you're a little further along than the other two guys, clearly. You got approval from the French government. That didn't take a day, I'm sure. Took a long time. So you presented this more like a business, less like a passion, though I think you could have used a little more passion.

Um, and, and I'm—

it's only— I, I love, I love it actually.

I think it—

you, you're solving a real problem, but I don't see like what that problem is. I get there's a communication issue, right? So who benefits Who benefits? No, you could have answered when you told me a minute ago. So, and I'd love to know later, but I'm just giving you feedback. Who's the real person that benefits from it? Because you say you're selling it to the schools, so maybe the schools benefit from it, but, you know, parents would also seem to benefit from it. So I always love to explain the businesses from the perspective of the customer, always. And that's where the passion can come in, and that's where, you know, the business starts to make sense. Phil, do you want to add anything to that?

I think it was good, you know, for lots of questions, but for 60 seconds, you know, it fulfilled its primary mandate, which is I'm interested to hear more. So you really can't do better than that in the allotted timeframe. Gracias, Joe.

I thought it was great. Phil, listen, thank you, Damien. Well done. We didn't give rating numbers. Maybe next time, you know what we should do? I think this could be fun. You and I are very good at feedback. Maybe we do a session like this and just instead of— we do it like less talk, more feedback. This was great. 1,000 people, we're still up in the 800s. I want everyone to give a giant, you know, virtual round of applause for Bill Libben. Fantastic gentleman, amazing leader.

And, you know, I'm eager to see the next companies that come out of your next effort. Thanks, Hideo. It was great to see you again. Let's catch up soon, and thank you so much for the opportunity to talk to all these people.

Thanks, everyone. Have a lovely morning, afternoon, and evening. Bye, everyone.

Take care.