Josh Elman, Greylock Partners "The Only Metric That Matters"

It is, yeah.

Okay, great. Hi, I'm Josh. Thanks for the intro, Brianna. Sorry for the technical difficulties. I think these things happen. I meet a lot of startups and they always ask me to think about metrics and how to think about how to get your company— you know, how do you even just know if something's working? And so I kind of put together this talk to kind of help frame that. Hopefully you can follow along the video. You know, at the end of the day, like, we're all here to build products that matter. And we all start somewhere. Like, this is the very first Whole Foods in Austin, Texas. This is a picture of LinkedIn in 2003, picture of Google when they very first launched in 1999, Facebook when it launched in 2004. If you look at all these things, they are, they are very much these very original things that look nothing like the companies today. But it took this one thing that started working that gave them the opportunity to keep expanding. and to keep on growing. I'm sorry, it's backwards scrolling.

So, scrolling super slow. Hmm.

That's good, 'cause it's streaming. So, look, if you think about it, like, why do we even start companies or go build products? 'Cause we really wanna make an impact in our users' lives. And so every product, you have to remember, every product starts with a launch, even brand new products, even from big companies. And they have zero users. And so hopefully with the launch, with all the marketing things you do, with product reviews, with the way that you actually grow your product, hopefully users come or you bring them. And what I like to remind people is like, once you launch, that's when the real work starts. Everybody thinks you do all this work to get your product ready for launch, but it gets even harder. But it's also cool because you start learning what's going on. You start hearing stories. these anecdotes. You start hearing from people who used your product and had a good experience, who tried your experience and they told a friend about it. And you get all these anecdotes and they start to go, okay, maybe there's something here, maybe you're having a good time. What I like to remind people is like, we often talk about data. I like to joke that the plural of anecdote is data because you really— all that your data is, 30% conversion rates or 50%, you know, for 50% growth is a whole bunch of anecdotes of individual people having experience with your product coming one by one to actually do it. But as you start working, as your product starts growing, you start getting lots and lots of data. If you're building on the web or building a mobile app, you're probably using Amplitude or Looker or Mixpanel or something, and you start getting all these dashboards and you start hearing conversion rates and numbers, and it can be pretty overwhelming. And there's so many data points that you hear from other companies and you start to Compare yourself against those benchmarks. We just talked 200,000 signups, you have a million downloads or 250 million page views or a D7 retention or D30 retention or a Dow-Mao ratio. And like, it can be pretty mind-boggling to figure out which of these you should actually process. So whenever I'm meeting, um, and you sit there like, so where do you actually start? So whenever I'm meeting a company, I really just focus on working with the company. Um, I just focus on one thing, like, are people actually using your product? I'm like, that sounds easy, but it's actually much harder than that sounds. It's like, how can you actually tell if they're using your product? And the first thing is to really understand these 4 questions. Why do they use your product? What is it that your product is serving in their lives that they are coming to your product in order to address it? In the case of Whole Foods, you want to go buy— it started with natural and more organic groceries, vegetables, etc. So they would come to your store. your product whenever they wanted to acquire those types of products. Then you go with the core action. What is the key behavior? So how do I actually— what is it that they're doing when they're using my product? In the case of Whole Foods, again, it was shopping at Whole Foods or something. Then the cycle. Do they use it at the frequency you expect? How often do you think somebody should be there? You know, a product that solves boredom like Facebook, you want people there multiple times a day because you're bored. You should figure out what's going on with your friends. You know, go back to my Whole Foods example. I don't know, once, twice a week people need to buy fresh groceries. You aren't looking for a daily user of Whole Foods necessarily, but you're looking Or somebody who comes back on some cycle, and then you really think about that retention. Are they actually coming on that cycle to do it? And so if you put this all together, the key metric that you'd like to ask if you're trying to figure out if somebody's really using your product, how many times are they performing this action on the expected cycle? And if they're performing the action on the expected cycle over and over and over, you start to know that they're using your product. And if not, they're trying it, they're testing it. They might be a very light user, but they aren't really sticky. So I'll kind of go through just a couple of examples that I've created here that start to help you think about how to do it for different products. So this is LinkedIn circa 2005. This was what the website actually looked like, which is kind of funny looking, funny colors, 2 columns. This is Web 2.0 at its finest. There's a little button in the upper corner called Add Connections that the more red we made it, the more invites that got sent, and so we would keep keep playing with that. There's a— there was a horrible time where it was a very red button. But if you think of LinkedIn back then, it was a professional network that we wanted you to come in and find great people from your network that would help you address business problems or be found by people who were looking for you. And if you think about it, it's really important that it was both find and be found. For finders, we expect you to come and search LinkedIn for other members 2 to 3 times a week. And so we could see this very small subset of people who were active on LinkedIn searching that often. They were recruiters or business development or sales executives or whatever. But for most people who signed up for LinkedIn, you know, I mean, LinkedIn has over 300 million members now. And for most people signed up, their main purpose there was to be found, which is something that you expected them to respond once or twice a year, but it wasn't a daily usage or even a weekly usage. It was just a matter of being there, being present in the database and having your information up to date. And that this cycle of a very small group of finders and everybody else there to be found has caused LinkedIn to be a giant company that's, you know, probably pretty relevant in all of your lives. Certainly is in mine. Then think about a product like Yelp. And again, Yelp is not a daily habit app. The goal of Yelp is you find like local businesses and services around you. Maybe sometimes you review them. You think about that, like how often do you need to find a local business? Certainly isn't every day, but maybe a couple times a week. If you're a reviewer, how often do you really feel like writing a review? Is it every day? Probably not. It's, you know, a couple times a month. But if you're actually searching Yelp a couple times a week to find new businesses, or you're a reviewer and you're writing and contributing a reviewer rating, you know, every few weeks or a month, you're actually a really good habitual user of Yelp. And Yelp makes a lot of money because those searches for businesses are actually pretty high-value searches. So then I think of Facebook, and I think Facebook has sort of, you know, defined metrics in a way that's almost unhealthy for most other companies. I mean, the goal of Facebook is to help you stay connected with your friends. And you might be bored and you might say, ah, what's going on in the world? And you just kind of open up Facebook. And the goal of Facebook is they make a great business if they get people doing that multiple times a day. Like if you're not checking Facebook almost every day, then you're not that valuable to Facebook and they're not actually serving this value of keeping you closer to your friends because they think that's truly a daily habit. But not every product has to work like Facebook. And so don't get caught up in comparing your DAU/MAU ratio to them necessarily. Because a great counterexample is Expedia. And Expedia is, you know, a great system to plan a great trip, make your trip better, find the most affordable options. And for most people, that's something you do a couple times a year. You know, business travelers maybe a little bit more, and Expedia drives a lot of business from them. But for most people, it's a couple times a year. So Expedia has to do a lot of work to stay top of mind so that in those couple times a year, you'll actually choose Expedia as your primary place to book your trip, and if they're successful doing that. So again, the metrics for Expedia are very different than Facebook. They're both good businesses. So, you may be, you know, so in all of these cases, you may have people kind of using your product once or twice, but that still doesn't necessarily define what you really want, which is, are they really, really, really using your product. And so I like to think about who the users are that are these people who kind of are really, really using your product. I call them core users, which is a great way to think about them. And there's a couple of things that are, that are different about core users than most. One is top of mind. Do they come directly to your product versus being referred by a link or a special offer or a coupon or something? So if I go to Expedia because I type in expedia.com in the browser bar, And I go right there and I book my trip, or I open the Expedia app on my phone, that's pretty directed. If I go to Google first, because Google is my default place to go, and then it lands me on an Expedia link, that's not quite as good. It's the same thing as if I go to open Facebook and it's like, oh, there's a fun game to play, and I play the game. That's very different than seeking out that game and playing it. So, you want to think of a direct navigation or direct open of your app on a phone is much different. That user is much more committed. You're top of mind to that user. And so you can actually track this in your metrics. Look at the percentage of users who have come directly versus linked, have come via a link or some other thing. And I actually start to think that your email newsletter actually works for this as well. The second piece is recurring. So these core users are the ones who are really recurring. They come back again and again. And you should actually very much understand your metrics, which of your users on a given day are users who who have been consistent users for a long time, which of your users are brand new users, and then which of your users are maybe in the middle, like they're not quite brand new, but they're not quite sticky yet either. And if you start to break down your analysis of your users on a daily basis, you'll know how many of your users are recurring, you'll be in a very different place. And then the last thing we talk about a lot is what we call referring users. Do they share it with others? And this is where you start to track your virality, your K-factor, and you start to figure out what percentage of your users really are referring other people, and that's what creates your viral loop. And your core users are the ones who are very sticky, come directly, refer lots of users. And if you start to understand that and separate that out in your product, you'll be in great shape. So there's an exercise that we did, um, when I joined Twitter that, um, this was like 2009, so it now feels like ancient history, almost a decade ago. But, uh, we were trying to really understand Twitter and they're trying to say, how do we get people stickier on Twitter? We knew that there was a small group of people who were using Twitter every day, many times a day. They had set up great timelines, and there was a bunch of people who were trying it out and weren't getting stuck. And so we started to work through a process to figure out who our core users were. We did this first analysis, which is we basically just took users over a 2-month period and sorted them by how many days they visited in month 1, and then the percent of the people who visited that many days in month 1 who came back in day 2. And then we charted that. And what we found is there was a group of people who visited 1 or fewer days in the first month, and they were 10% or less likely to come back in the second month. But a group who visited the product 8 or more days in the first month, they were over 90% likely to be coming back that much in the second month. And by charting it and understanding this, we said, oh my gosh, once we get you to like 8 days or more, or once you find your own pattern to get to 8 days or more, You're quite sticky and you're likely to keep coming back, and that means you get the product. Now, we wanted to unpack the reasons that actually got you there, but we could tell now that those were the people who really had done it. And so we actually sorted this into— I call this kind of a kite chart— where we actually were able to translate, you know, of the people who are curious or just signing up, they would sign up for the product and they would be considered new in their first sort of 30 days. And then we would look to say, okay, in their second month, did they start to now look like these core users who became, you know, 8 times a month, or do they look like these casual users who came less than 8 times in the month, or cold or comatose users who didn't come back at all? And so this was a really deep understanding because we were able to chart the progression from one group to the other. What we identified at that time was only 20% of our new users were ending up in core. So every time we were getting a whole bunch of new users, we were basically 4 out of 5 of them were sort of not getting sticky with the product and were falling away. And so we then dug deep to analyze what was special about that group. What were they able to work through and understand? What was their motivation before they came to the product? What was their motivation as they engaged with the product that got them sticky so that we could build on from there? So then we started asking questions. The first thing we did is went to our data and we said, what did those people do differently? The first thing we figured out is those people, by hook or by crook, found their way to following more than 30 accounts. And once they started to get to 30 accounts, the likelihood they were going to be in that group was much higher. The second thing we found is they had a really interesting follower ratio. We called it sort of 1/3 people that they followed followed them back, 2/3 didn't. But that 1/3 who followed them back was actually enough to be friendly and sticky. And if you think of something like a, a, you know, Facebook, 100% of the people you friend friend you back. You know, on Twitter, it was unique to have a slightly different graph. where you follow people you were interested in, news, information, celebrities, etc., but then you still had 1/3 of the people you were following following you back. So you had enough friends or enough stickiness in there to be great. Those were the users who really got it. But data wasn't enough. The other thing we did is we went out and talked to them. And we went out and asked a whole bunch of questions of our users and really tried to understand what, what had meaningfully caused them to be motivated enough to to do all that work? 'Cause what we realized is following 30 people and getting that follow ratio was non-trivial. And so some users did. So we found a bunch of users who had, I called them bounce-back users. They had signed up, left the product, come back, and sort of then got active on the product. And so we were able to ask them, like, basically to walk us through that whole journey, why they signed up in the first place, what caused them to fall out, What caused them to try it again? And what then did they get sticky for? And you can ask your users like those 4 questions no matter what product you have. For Twitter specifically, the answers came back pretty consistently. Well, I heard about Twitter, thought I should sign up, thought I needed to have something to say. I didn't have anything to say, so I didn't use it. Then later I heard that my pastor or my favorite food truck or a celebrity I like or something was tweeting funny things out all the time. And I wanted to stay more in touch. And so they would come back and actually find that person on Twitter and follow them, and then start to decide that the timeline and other things mattered. So by asking those questions, we kind of uncovered the mispositioning of Twitter about being about broadcasting and about shouting. We discovered that actually teaching people how to follow and helping them get to the things they might have heard about and wanted to know about was actually really, really important. And I can't emphasize enough, when we talk about data, people look at data, data, data. Like getting out and doing 10, 20 interviews will give you these anecdotes that summed up might be even more powerful than the data, data, data that you can do. So these core users are special. I call them your superheroes, and they're the ones that you really want to stay focused on. So let's say you launch something. Let's say you get a few core users, it's starting to work, and of course you want more because you really want to have a big business and a big network someday. So I'm going to go through kind of 3 little, just sort of small, these problems that can start to happen and just a couple of ideas that, you know, if you've already done all this work to get your metrics and understand them and understand your core users, you're so far ahead of most companies that this is the next set of problems you'll work on. And these can lead to a much greater discussion as well. The first problem is, let's just say not enough people show up. You just don't have enough people at your front door. Well, when you do already have these good core users, you're actually in pretty good shape because they are using your product. They are sticky. So how do you help them be more viral? And there's a couple ways you can do it. One is what I call word-of-mouth virality. How do you help your core users describe your product? How do you help people— remember I told you Twitter had that wrong positioning? How do we help people who like Twitter talk about it in the right way? How do we feed them the language so that when they go to tell their friend the value they get out of Twitter, they're communicating in a way that will motivate their friend even better? Say it yourself to the press, like talk to the press all the time, and then you'll actually incept the idea of like, this product solves this problem and everybody will talk about it in the way you want them to. And all of a sudden people will be more motivated to sign up and get it right. The second thing I like to talk about a lot, and I have a separate talk on virality, that is online as well, is to have your core users share content and more from your product. You want them to demonstrate their deep usage. If they're coming to your product all the time, there's probably things that they're doing that are valuable, they should be passing on to their friends and family. And so, you know, I would highly encourage you to find those moments in your product and help your users amplify them. You know, and the third one is what I call infectious virality, which is how do you get people to bring their friends into the product? please come and join me on this product, et cetera. Sometimes we use incentives for that. You know, if you have a paid product that, um, you know, hey, I like this thing so much, here's a free coupon for $5 and I'll get $5 off too, to kind of motivate both sides. That's only if you can afford to do that. Um, if you are a free product, I would not encourage you to pay users to do it. Um, um, you know, and you're, you're seeing this a lot though, is like, how do you use these sort of viral tools? to engage it. The second problem is, let's say you get a lot of people signing up, but nobody's sticking around. And this is where, you know, we all see a lot of problems in products today. I mean, if you can't get enough people to do anything, that's one thing. But man, you're lucky, then you got like a feature in the App Store, a lot of people signed up, and then what? Like, I think onboarding is incredibly important for a company, and I think it's incredibly underrated. What I'd say is you have to onboard your users all the way to core. The big change in thinking is you're not done onboarding a user until they're on a habitual cycle. And so there's 3 basic things we can start with onboarding, but this is again a much longer conversation. The first is what I call the learn flow. When you think about onboarding in your product, most people think about a new user experience as like, how do I get them to sign up, create an account, enter the password, maybe like add their contacts, allow notifications like on a mobile app, and then quickly just jump into the product. And like maybe if it's a financial product, you have to add your bank account. I mean, there's all kinds of things, but it's like get them to do all these tasks and then bring them to my product. I think that's entirely wrong. A new user experience is not something that should be rushed through. It's actually this first great moment that you can bring your user all the way into your product. You can teach them your product, you can welcome them there, you can actually get them set up for success. train them a little bit on a product. Games do this masterfully, for what it's worth. If you've ever played a game, it just pulls you in, and then when you go to try a product like this, it just like, you know, like drops you on there and, and, you know, you're kind of screwed. So, so I'm really big on rethinking onboarding, and some products, you know, just do this masterfully. The second thing I like to think about is like your onboarding experience is not just your first-time experience. I call it welcome week. Like, if you don't get users to do everything on the first day, nudge them to come back, use email, use push notifications, use whatever you can to come back and keep going, maintain where they are in the flow, sort of give them other things they can do in those first couple of days to get habituated. If you can't get somebody onboarded within a week or two, then they're going to have to get new motivation to come back and really give your product a shot again. And so you're gonna have to re-onboard them, but don't think that it all has to be done in the very first session either. Think about actually extending this time as somebody gets to know the product and hey, they've now done a couple things on their third day. Now show them something new. And then the third thing is how do you use social pressure effectively in your product? You know, everybody has has friends that might already be on the system or in the network. If you can connect them up at all, show them what their friends are doing. Use those friends to pull other people in the product. There's all kinds of great features you can build, you know, around using social pressure. in order to do it. So, you know, I can talk about this stuff forever, but I want to make sure that we have time for Q&A as well, assuming that the Zoom video all works and everything else. You know, and so really, you know, as you think about figuring out what metrics matter, you know, don't ask yourself what metric, what target I should go for, what benchmarks I should go for. Look intrinsically into your product and into what you expect people to be doing. your product and business and be able to say how many people are really using your product.

Well, thank you.

Thank you. Thanks, Josh. That was awesome. Um, what I'm gonna do now is I'm gonna read off questions from the discussion section and then people are gonna post them into Zoom as well.

Got it.

So first question we have is from Robin Carswell from GetJobs. He said, what's the metrics that matter— what's the metric that matters if you're pre-product? Yeah.

So when you're pre-product, you obviously have no metrics. What you really should be designing is what you believe your metrics actually will be. So if you're designing a product for the first time, how often do you think somebody should use it? How— in what cadence? And what things should they actually do? How much time should they be spending on your product? And you should be able to basically articulate that as a good hypothesis. And then when you launch your product, you start to see if people match at all to that hypothesis.

Awesome, thank you. Next is from Fernando Treba. What is the most important metric for early-stage marketplaces?

You know, marketplaces are a great question. I'd be curious of exactly what type of marketplace you have, but in the very early days, you have this, this really unique confluence. You have to figure out how to get a bunch of supply, you have to figure out how to get a bunch of demand that matches with the supply so the supply feels used enough that it's actually valuable and that the demand kind of keeps coming back because they, because they want to do it. Now, this is— there's a bunch of like marketplaces are very broad things. So I'm thinking more like the eBay, Airbnb type of marketplaces where you have people who are doing listing, who are doing something. In cases like Uber or other kind of more managed marketplaces, things are actually a little bit different because you still have to get supply, but then you're sort of taking a bunch of the responsibility for utilization of the supply, and then you need to get enough demand. But in both cases, you want to figure out, is supply scalable and repeatable, and is demand scalable and repeatable? And usually, not always, usually it's actually demand that is the harder thing to do. So go and get enough supply, then go get demand and see if you can even connect it and fill with that supply. And if you can, you can start to get to liquidity where the supply feels like it's actually being utilized in an effective way, you know, where they don't need to feel the need to go list it in a whole bunch of other marketplaces or something. That's when you actually start to know that you have something working. Airbnb is a great example where you started with these people who were like, oh, I've got an extra room, and they would get a couple listings a month, and that was incredible. They're like, oh, I'm getting enough, I don't need to go somewhere else, and then it would sort of spin up from there.

Awesome. For context, Fernando's startup Lovett has fashion designers on one side and then consumers on the other.

Got it. Well, and so in that case, you have to make sure you have enough designers but keep them happy enough that your consumers coming back over and over are actually very happy about finding those products. Otherwise, the designers will need to find other channels.

Cool. All right. Next question is from Janine Yancey from M-Train. This is kind of a long one, so bear with me. We have an existing SaaS B2B subscription. In March, we're launching a freemium B2C subscription that provides content and legal advice to employees on issues such as harassment, bias, and ethics. What metrics should we be looking at? Time spent on site? Number of shared invites from current subscriber, penetration within each enterprise?

That's a great question. You know what, look, this is the hard question and this is kind of the point of my talk is for your site that provides content legal advice, I don't know what the right usage patterns are. This might be something that I actually only— that I need to subscribe, but it's kind of like insurance and it's kind of like, oh shit, I need to go and get this harassment advice. Thank God we signed up for this thing so that we can actually go, you know, search it all right now. Or is it like monthly training and there's like new articles that are coming out that people should be reading every month? It's a really good question. And so I would ask you, as you're designing it, how much do you expect the people who are purchasing this subscription to use it? And are they then actually doing that? You know, I'd be worried if when people sign up, they don't go and spend a bunch of time and read and kind of familiarize themselves with it and kind of that onboarding we talked about. But after that, I don't know if it's something that they use on a recurring basis or only when there's times of trouble. For things like your shared invites, those are great metrics to understand your virality specifically. And for penetration within each enterprise, again, it just depends on how often you expect, you know, the worker to use it. As I said, at LinkedIn, we didn't expect most people who are like an engineer at a company to even touch LinkedIn unless they got an inbound request for something, but we expected that recruiter to be finding people and making those requests all day long.

So, thank you so much.

Diet Coke commercial.

All right, next question from Rama, founder of Avocado Culture. Rama says, hey Josh, I like following you on Twitter.

Thank you.

Avocado Culture helps companies build their company culture. We're a SaaS product in live beta. At this stage, I'm focused on making sure the companies derive the most value from the product through metrics like usage and qualitative feedback from companies. So is there an NPS-type metric you'd recommend for this type of situation?

Yeah, I mean, I think, I think NPS is actually a great metric to use. There aren't many that have been invented that are better than just like Do you like this so much you'd recommend it to another company? So I would definitely, you know, make sure you are asking that and are getting that from your interviews and what other things you're doing. I would, you know, but I would do a lot of anecdotes, anecdotal questions too, whereas as you ask questions, you are trying to understand what people are saying about your product and how they're getting value. So that you actually start to piece that together and you kind of connect, like, is your theory of them using it actually connected with what they're actually doing? Um, and if so, then the NPS can be very helpful. Cool.

All right. Thanks for that. Um, next question from Nikita. Reminder guys that post questions to the Zoom channel, just go to chat and then post your question. I'll read it off for you. From Nikita, what top 3 analytics tools would you recommend for a new app or website?

That's a great question. You know, I don't stay quite as close to the actual tools anymore. This is what I miss, you know, going from sort of more of a product role to sort of an investor role. You know, the companies I work with that I've heard a lot of good things happening are Looker, if you can set up enough of a system to collect your data. I don't know if I was just starting out if I would start there. I like Amplitude a lot. That does a really good job if you kind of instrument your events properly. I've seen people use Branch Metrics and Adjust, which have really helped them do sort of viral and growth tracking. So those are kind of where I would probably start.

Cool, thank you. I have a question for just Founders Network in general. You mentioned frequency, that users should be using the product. How would you measure what should be the frequency? Where do you think we should start?

Look, I think that starts at the theory level of what the product is actually for. Because I think, now, the one interesting intersection is there's an intersection between frequency, stickiness, rememberability, and business model, right? So if you're like a product that people only use once a year or like once in their lifetime, like funeral services or something, just trying to, that was a horrible example. The frequency isn't very high. You know, it's family and referral and other stuff around that. But like the frequency isn't very high, but you want the value to be very high. Like an electronic store, like I don't expect you to go to Best Buy as often as you go to the grocery store, but I expect you to remember Best Buy and go and buy your more expensive electronics when you wanna do it. Whereas the grocery store, you should be going like almost weekly to refresh your food supply or, you know, it's Instacart, but same thing, right? And so you really, the really, it depends on the business and it depends on the product. You know, we talk a lot about frequency in media products that are advertising-based because the whole point of media products is they don't actually sell you anything. They are what you consume and then alongside that you're consuming advertising that then gets sold. So those need to be higher frequency.

Okay, that makes sense. Yeah.

Yeah, I posted on the forum. So we have like different products. We have events, we have a forum.

In the forum we have 1,000 conversations a year and 2 to 3 replies per post.

So I posted last night about this balance between heads down versus heads up. So what is the right balance and does it change as the lifecycle of the firm kind of progresses and your company changes? And it comes for us, for our events, we don't want to ask our founders to go to too many events.

Yeah.

Of course, you could be heads up all the time and get nothing done. And we don't want to ask too infrequently because then they're missing out on you know, conversations about the best analytics tools to use, or like, hey, I just met Josh Elman, would you like to— so those can be game-changing. But if you're just heads down, you're gonna miss those opportunities. So I guess I want to kind of tie it to that forum thread here because it seems to be right, and you have a lot of experience working with founders at different stages.

And there's no great answer on this one. I think the first answer is There are plenty of people who've made way more mistakes being too heads up than too heads down. So, so it is— if, if you ever feel like you're being a little bit too heads up, you are. And like, you should go more heads down. That said, like, look, the point of like a board meeting at a company is not to say, oh my God, do all your work and dress up your company once a quarter so that like everybody feels it. It's to cause you to basically do that step back and say, okay, we're doing all this work. Are we really aiming right? Because if you think of any company or any project, it's all about the aim and then all the work you do on the aim. And man, like, you're shooting for the moon, you know, whatever. SpaceX just launched their rocket last night, right? And like, they're shooting for a very precise spot in the world. And if they're like off by a little bit, and then the faster you go, you end up off by a lot. And so, so the key of being heads up is to use those opportunities to pull back and make sure that all that heads-down work you're doing towards the aim is going in the right place. But if you do it too often, you kind of like keep resetting the aim, you never get any traction towards that thing anyway. So there's no real answer to that question, but hopefully I gave you a couple frameworks.

Yeah, yeah, well, I'm hoping to create a framework based on stage or some dimension so that our members can level set against each other. Kind of be in a normal range.

But I think, I think it's quarterly. Yeah, I think, I think on a quarterly basis you need to pull your head out and just make sure your aim is proper. I wouldn't, you know, maybe, and then maybe there's a smaller version of that monthly. And so if that means you go to some events or you do other things that cause you to break up, like, you do that. Also, we can only all work so much. So if you've been working for a 10-hour day and you can go to an evening thing that is partially fun and partially work, and that's what energizes you to do that. If you're somebody who needs to go retreat and like sit and play your Nintendo Switch, like you should do that. Like you just gotta recharge.

Yeah, cool. I like that. I like the, uh, the quote of the plural of anecdote is data, 'cause that also is super relevant to that thread for, especially for our Bootstrap members.

All right, we got a couple more questions in here. Another one from Nikita here. Do you recommend A/B testing and how to best integrate it into an app?

I mean, A/B testing is great. It's how you kind of can try a couple different theories. Does this word work better? Does this work better? Does this work better? And then kind of learn and pick the ones that work most. When you're at the very early stages of your business, you can A/B test yourself, but you only end up with local maximums. You don't end up making the real thing. So you're better off making bigger bets anyway. But sometimes, again, you can learn. You're like, hey, do people talk about my products? Like, I'll use Airbnb as an example. Talk about it as like, rent a room in somebody else's house, or like find a cheap, local, affordable place to live in your, you know, to stay in the city, right? Like, those are 2 very different ways of saying basically the same thing. Which one appeals to people? And like a little bit of A/B testing. It's not about this got more clicks, this got more clicks. You actually want to back into the meaning of like, whoa, this messaging actually appeals to people in different ways and has different implications. Which should I do? So love a little bit of A/B testing, but don't overdo it. And definitely don't just like, well, this one tested better, this one tested better. You'll end up down this funny path.

You mentioned your red button.

Yeah.

Which has almost become like a metaphor.

Yeah, make the button red, it'll get clicks. And all the buttons turn red and then nobody can find anything. So then somebody makes it blue.

Next question from Gail. As a B2B2C B2B consolidator. Yeah, it's spelled incorrectly. Okay, consolidator and marketplace. We are focused first on building our community, being launched the full marketplace, but we are getting pushback from potential investors on holding off on chasing revenue. Are there key metrics in this type of model that can help us build our case?

It's a great question. I don't quite know what a B2B consolidator and marketplace is and what the community is— look, again, the key metrics come back to what is it that you expect?

Oh, okay.

Yeah. Oh, our community board.

Oh, dog owners and the businesses that love dogs.

Oh, that's great. I have a cute little puppy.

Galen. That's Galen Udell.

Hi, Galen. I have a little puppy at home.

That's your whole name, Galen. Sorry.

Look, I think it all comes back to how frequently do you expect them to interact with your product and what are the valuable transactions you expect them to make? And so, if you're building out your community because you're creating those engaging actions, that's a really powerful place to be, and that's great for you. If you haven't figured that out or gotten that working, like, Like, then I would, I would focus on getting the metrics again. Like, what is it that makes your product meaningful? And what is it that you expect people to do on both sides, for both dog owners and businesses that love dogs, you know, to use it? And it sounds like you need to get enough businesses on there, like I'm just guessing, that you would then have dog owners want to come and search and find businesses. You need to figure out what that recurring rate actually is to make it work. And so what investors are really looking for is not to tell you what metrics you need, but for you to tell the investors, here are the metrics I'm using for my business. Now I have more confidence that it's working.

Thank you. All right, and then next is from Tom. What are average good and really, really good conversion rates on ecommerce sites that you see?

You know, again, this totally depends on the commerce site, 'cause it's hard to know what conversion looks like. Just for example, in the physical retail world, a Gap store has about a 30% conversion rate. Of the people who walk in, about 1 in 3 walk out buying something.

That's not bad.

A supermarket has about a 97% conversion rate, because by the time you drive and, you know, you're at a mall, and Gap is often in a mall, you don't drive just to Gap.

gap for that.

So you're often there just to browse and sometimes you buy. Whereas like a grocery store, you're there to very much shop. A Tesla car dealer doesn't have the highest conversion rate because you're there to do a lot of consideration before you actually make the purchase. So you might visit it, or a car online site, you might visit it many, many times before you actually choose to buy it. So I always have trouble with these average really good conversion rates. That said, if you're getting really good intent from Google, you should be able to— and it's a fairly direct purchase people should make. You want to be in the high single digits, tens, I think, range for Google traffic because Google people are browsing and they're looking and you expect them to go back and forth all the time. Most conversion process though really is a funnel and there's products like Kissmetrics that really help you understand all those multi-touchpoints until you get there. And then once you get to a steady state, then it's how do you get it? Like really, how do you get it better is more important to know than benchmarking against other people. All right.

Makes sense. Okay.

Awesome.

I don't see any other questions coming in, but I'll leave everyone a couple more minutes. Do you have any closing questions here, Kevin?

No, I think I, I think I got my question there. I just want to thank everybody for their patience.

Yeah.

Um, with Twilio, and I think we're gonna officially be moving over to Zoom after this call.

Okay, great. Zoom seemed to work.

It seemed to work, yeah.

In a pinch, too.

Yeah, so I'm glad that all worked out. And thanks again, Josh, for taking the time to do this.

Thank you guys for having me.

Thanks, Josh. Any closing thoughts from you?

Focus on if people are really using your product, and don't worry about other metrics. Worry about ones that really define your unique product directly.

Cool.

Great, thanks, guys.

Appreciate it.

Thanks, everyone. Thanks for being so flexible.

See here, exit full screen and stop share.

Oh shoot.

End meeting.