Gibson Biddle on his DHM product strategy framework, GEM roadmap prioritization framework, 5 Netflix strategy mini case studies, building a personal board of directors, and much more
20 Jun 2022 · 60 min
Gibson Biddle, Netflix's former VP of Product, walks through his DHM framework for product strategy and GEM for prioritization. He shares five Netflix case studies showing how the company delighted customers while building competitive advantages and improving margins.
Gibson Biddle, whose name is really fun to say, by the way, has directly taught and worked with more PMs than only a handful of other people in the world. Before going full-time on teaching, he was VP of product at Netflix and at Chegg, and has written what is almost surely the most popular Medium post on product strategy. In our chat, we learned about Gibson's very popular product strategy and prioritization framework, and we go through a bunch of real-life case studies from his time at Netflix to see how they apply in the real world. I love how Gibson comes up with such clear and memorable frameworks which help demystify the vagueness that surrounds strategy and prioritization. We're really lucky to have Gibson share.
These with us and I hope you enjoy.
This episode is brought to you by Flatfile. Think the last time you imported a spreadsheet. Did it work the first time? Chances are it did not. You probably got some weird error and had to try a bunch of things like removing the blank title rows above your column headers, or Googling how to save with UTF-8 encoding. What even is UTF? Who cares? You're just trying to get your file where it needs to go so you can do your actual job. Your customers run into the same issues when it matters most, right after signing up for your product. Enter Flat File. Flat File is the data onboarding platform built to take the acute pain out of importing customer data into your product so they can see the magic that you promised them. Flat File is SOC Type 1 and 2 certified, GDPR compliant, and even HIPAA compliant, ensuring your customers, no matter where in the world they're located, are sharing their data securely and in compliance every step of the way. No more emailing files back and forth, no more help articles that just don't land, just clean data on day one, when it matters most. Get started importing millions of rows of customer data in minutes at flatfile.com/lenny. This episode is brought to you by Coda. Coda is an all-in-one doc that combines the best of documents, spreadsheets, and apps in one place. I actually use Coda every single day. It's my home base for organizing my newsletter writing, it's where I plan my content calendar, capture my research, and write the first drafts of each and every post. It's also where I curate my private knowledge repository for paid newsletter subscribers, and it's also how I manage the workflow for this very podcast. Over the years, I've seen Coda evolve from being a tool that makes teams more productive to one that also helps bring the best practices across the tech industry to life. with an incredibly rich collection of templates and guides in the Coda Doc Gallery, including resources for many guests on this podcast, including Shreyas, Gokul, and Shashir, the CEO of Coda. Some of the best teams out there, like Pinterest, Spotify, Square, and Uber, use Coda to run effectively and have published their templates for anyone to use. If you're ping-ponging between lots of documents and spreadsheets, make your life better and start using Coda. You can take advantage of a special limited time offer just for startups. Head over to coda.io/startups. Lenny to sign up and get a thousand dollar credit on your first statement. That's coda.io slash Lenny to sign up and get a thousand dollars in credit on your account. Yes, sir. I am so excited to be doing this. We've collaborated on some writing. We've done a couple of our side chats together, but we've never really dug deep. into anything. And so I'm really excited for this opportunity to chat. So welcome.
Great, Lenny. Thanks for having me. It's fun to virtually be in the same place at the same time.
Someday we'll do that in person. Oh man, I can't wait. For folks that aren't that familiar with you, could you just share a very brief overview of your journey through Product?
It's hard because I'm older than dirt to be very brief, but I'll give my best shot. The intro was I discovered I love teaching. Early I ran a sailing school, took a year off from college. That was my first startup. And then the three chapters of my life, the first one was in marketing. So I actually started in the mailroom at an ad agency and then, you know, went into account services and create a service that helped name new companies and products. Like if you're in the Bay Area, the Versatil or the Versateller, it's a Bank of America name.
I've always been so curious about that industry, but that's a discussion for another time.
Oh yeah, well we can come around to that, but at the end of the day, it's all about packaging and positioning and branding. And so I, I was curious about that. I got good at it. And then I lived in Silicon Valley and I wanted to go into tech. And so I decided to go back to to a business school and I went back to Tuck, which is at Dartmouth. It's the only skier, it's the only business school that has its own ski area, which is why I went there.
It's in Hanover, New Hampshire.
Okay, that feels like product market fit for Gibson. Exactly. And then after B-school, this is the tech journey. I joined Electronic Arts, Punk Kid in the right place at the same time, at the right time. I joined in marketing. And then I switched over to product. It was a great place to learn product. They had, they called it Producer College. I learned a ton. And my first startup was actually a joint venture between Electronic Arts and Disney. We created, it was called EA Kids at the beginning, and then came Creative Wonders. And we sold that company to Mr. Wonderful from Shark Tank. He was the CEO of the learning company. Oh, wow. I know that. Rita Rabbit software and Oregon Trails. That dates me because most of the listeners have probably played Oregon Trails as a kid. Dying of dysentery. That's what I remember. Yeah, yeah. And so the product roles, I just, I grew up into, I call it a muckety-muck, but a VP of product or chief product officer. Joined Netflix in 2005. I left out some failed startups. That's important. It's important to let folks know I had some failed startups. But you can gloss over it in the brief version. Netflix 2005 and then 2010, I went back to my heart, which is teaching. Chegg was a textbook rental and homework help company. It continues to exist. And we took that public sort of 2014. And then the last five years, I stopped direct deposit. And I'm really back to the coda in my career. I'm back to teaching. Which I really love to do. So I do talks, workshops all over the world. I do more writing, which really saves my voice and then experiment with different ways to really try to be helpful to product leaders around the world.
Awesome. We're gonna be spending time digging into some of that stuff. Did you describe this point in your career as stopping direct deposit?
I did.
That's amazing.
Wow. At least we've never heard of that.
I totally get that. Amazing. So question to ask folks in your line of work especially is how many how many PMs would you say you've helped train through workshops and advising and online courses and anything else you've done?
I thought it was going to be a how many PMs does it take to, you know, screw in a light bulb? We thought it was going to be a joke. That one too. Okay, my swag. Swag is a stupid wild ass guess. Important idea. So my guess is in real life, I've worked with product leaders where I was in the building with them. It would be like 500 to a thousand. When I would manage an organization, you know, there might be two or 300 people in it. So that's the in real life. The next one it would be via talks, workshops, newsletter. That's probably, I kind of can swag my way into that. Probably 50 to 100,000 a year. The reason I sort of know, you know, I'm a feedback freak. So there's a survey monkey at the end of everything I do. So for instance, every morning I wake up, it's always feedback for the same talk. I did a branding for builders talk for Product School and they'll be like 10 surveys. I've got like 6,000 surveys for that. They're all watching and recording. You know, my guess is that means like 60,000 people have watched that one video, for instance. Anyways, and then lifetime, it's probably getting into 500,000 to a million. So that's my swag. I could be 2x wrong on either side.
Okay, a million potentially.
Well, your number's bigger. Like you could just go off a, what, 115,000 subscribers to your newsletter? Yeah. You're getting to a billion, Lanny. You're going to be the next level. 500 million to a billion.
I feel like the level that you've helped PMs is at a deeper most of the folks, especially the courses you've done and things like that. So, so it's a little hard to compare, but anyway, that's really impressive. And I feel like when I ask people this question, I feel like this is gonna be a high bar to meet. So I'm excited to dig into a lot of the stuff that you've learned along this journey. Maybe the area you spend most of your time on is helping PMs build their product strategy muscle through your workshops and you have these legendary medium posts about product strategy. So I'd love to spend the majority of our time talking through just helping people build this muscle. And maybe just start. You have this really simple model. for describing what product strategy should be for companies to delight customers in hard to copy margin enhancing ways. So there's kind of these three parts, like customers hard to copy margin enhancing. I love to just kind of hear your take on each of these three parts, how you came up with this, and then maybe a story or two of how you relied on this and.
Use this at Netflix. Sure. So you got the model right. And I mean, the short answer is I definitely learned it from Reed Hastings when I started at Netflix. Actually, he did a reference check to a friend of mine checking up on me. And the only question he asked, my pal, his name is John Dawes, was, Hey, John, is Gib ready? Can Gib delight customers? That was the only question he asked in the reference check. So luckily, you know, I built really delightful kid software. You know, back then, people were talking about satisfying customers, listening carefully, understanding them, et cetera. And the delight, it just said, no, no, no, the job is to delight customers. Like Peter Thiel, his book, From zero to one, the job of an entrepreneur at the beginning is just to find out something that's 10x better. Delight is trying to work in that magnitude. The hard to copy part, definitely I learned that. I had a product leader who worked for me. His name was H.B. Moch. He was focused on the non-member page. That's where people sign up for, they put in their email, their credit card. They would get a free trial. It looked like crap, you know, like everything, startups sucks at the beginning. But he put a happy family on the couch on the screen. And that's where you put in your email. And that got a lot more people engaged. and so it delighted customers but it was helping to build the business because more folks were setting up for a free trial. I'm like, that's great, HP. But, you know, I'll bet my paycheck that within a week Blockbuster is going to put a happy family on their couch too. And this is the problem of doing things that are easy to copy. So in the long term, if you can delight customers and create hard to copy advantage, things are just a lot better. And so the last phrase, margin enhancement, that's just a fancy phrase for making money. So, you know, margin technically would mean profit. So, you know, I just gave you example with that non-member page with the happy family on the couch delights folks. It helps build a better business. But in that case, Blockbuster was able to copy it. So that's the model. So the hardest part in that model is how do you balance delight versus margin? So if Netflix said, Hey, everybody, you're paying 20 bucks a day, but you're going to get exactly the same service tomorrow for five bucks a month. You'd be just like, this is fricking awesome. You'd be really delighted. but Netflix would not, the business wouldn't work. So one of the first places we sort of experiment on how to evaluate trade-offs between delight and margin. In the old days, Netflix was a DVD by mail company. And yeah, if you asked customers what they wanted then, they all said the same thing. I want my new release DVDs faster. Back then, most customers would have to wait a week or two or four weeks. Back then, the movie came out in the theater. Two months later, it came out on DVD. We just couldn't afford to buy for that initial demand. So the way it worked was some people got it the next day in the mail and some people had to wait two months. And so the A/B test we set up, we said, okay, everybody says they want this. They, you know, in focus groups and qualitative and surveys. Well, let's A/B test and see what we really learned. So the A/B test was called the perfect new release test. So imagine, you know, we're at about a million customers circa 2005, 10,000 peeps are in a test cell, they get their new release DVD the next day in the mail. Awesome, right? And the control gets it whenever, let's say average maybe two weeks later. And so the way we measure delight in this A/B test is will you improve retention for the folks that are getting their DVD the next day in the mail? That's how we measured it was retention. You know, when Netflix started it was like 10% cancel every month. 2005 it was about four and a half percent cancel every month today it's about two percent cancel every month. So I'll just put it back to you. Do you think when we looked at the AB test results, by the way, you can't get this wrong, Lenny. So just relax. Do you think the perfect new release experienced improved retention?
My guess is because you're telling the story, it's going to be a surprise. And I would guess that it did not.
Okay, so your guess is we won't see an improvement in retention. And I'm guessing people, you know, listening to us right now, there's going to be some folks that, oh, yes, that's what everybody's asking for. Of course, you we should deliver it to them, right? Well, in this case, both right, right? So we saw a very small change in retention. That was the surprise. And so it went from something like 4.5% canceled in the control, it was 4.45, very small change. So we can measure it. And so now you get in the math of delight versus margin. If you push this out to all million customers, you would essentially save 5,000 customers. And so what's the value in saving 5,000 customers? The way we valued it, we said, well, we've got 5,000 customers. the lifetime value of a customer then was 100 bucks. And then we multiplied it by two, which is our word of mouth factor. And so our theory was if you loved it, Lenny, you'd tell your friends about it, you'd rave about it, and you'd bring in one other customer for free into Netflix. And so we used this 2x word of mouth factor. And so if you do that math, it was worth about a million bucks to us as a company. So now you get into, okay, what's the cost of that additional inventory? And the answer was five million bucks. So on one hand, you're bringing in a million bucks of value by retaining more customers, but you're spending five million more. So it doesn't seem to make sense. Okay, so easy question. Don't overthink this one, Lonnie. Would you roll this out to all customers? I would not. You would not? Okay. Can you help build an argument for.
Somebody who would choose to roll it out?
Oh, no. Give me the hint. I'll take it. The hint was there was an assumption in my formula. You know, I said 5,000 customers times 100 bucks lifetime value times a 2X word of mouth. So the argument for why somebody might do it, they disagree with the 2X word of mouth and what the argument is going.
It is. Basically, the experience is so much better that word of mouth increases.
It's so delightful. Yeah. So Amazon would use a 10X, right? And so sort of a wash there, you'd probably lean forward and do it. We actually worked hard to try to isolate what that word of mouth factor was. It was really frustrating. And then the reality was Barry McCarthy was the CFO. He was apoplectic about the idea of using a 2X because it means you'll invest more in building a better product. If you let the product team use 10x, like very is like, we don't have the money. You know, we don't have the money to spend 5 million on DVDs. Anyways, this is where we sort of refined our thinking about the model of delighting customers in hard to copy margin enhancing ways. Pop quiz. I know you know the answer, Lenny, but I want everybody else thinking about it.
What makes Netflix hard to copy?
The licensing deals and content that they're creating.
Original content. Yep. Keep going. Their brand that they've built over time.
You trust them with their credit card.
Every month for 20 bucks. Oh, absolutely.
I think I've been a customer for five years.
Yeah, I don't even know. What else makes them hard to copy?
Imagine their tech that used to be the ranking algorithm recommendation engine. Yeah, so there's unique technology. Personalization is a great example. You just brought that up. There's only like one other idea that.
I think is, well, there's probably two others for Netflix. So keep going.
Dig deep. Maybe, I don't know if this is an answer you're thinking, but just the talent they've built up over time is probably. That's really interesting. People go to culture and then I sort of specify. What makes the Netflix product hard to copy?
Right?
So I won't allow that one. Oh man.
Okay, keep going. What makes Netflix hard to copy?
I'll give you one clue. Like, what made Facebook wicked hard to copy?
Their network effects and growth loops.
Yeah, yeah, yeah. So we actually, you know, we tried to experiment with friends and social getting movie ideas from your friends. The idea is if your friends were on the network, you wouldn't want to leave. And you'd also get great movie ideas.
By the way, that was a failed hypothesis. Right, I was just gonna say, I remember that. Did not work out.
I don't see it anywhere. Did not work out. Yeah, I guess we could make an argument today every screen in the world is magically pre-wired so you can watch Netflix anytime, anywhere. There's sort of a kind of a flywheel or network effect at work. But anyways, that's great. So that's just a model, but I.
Have found that model to be helpful in my thinking a bunch of times. There's a couple things I wanted to follow up on there, because there's a lot there. One is this idea of testing the ideal almost, and kind of working backwards from that, that's something I always find found really helpful. Is that something that you espouse and find to be really powerful?
You know, honestly, I think of these product strategies as these high level theories, these hypotheses about how you will delight customers in hard to copy, margin enhancing ways. So I just gave a failed hypothesis, the social one. You actually brought up a winning hypothesis. Personalization delights customers because it makes it easier to find movies you'll love. It's hard to copy. Netflix sort of knows that movie tastes of a billion people worldwide, that's 222 members times about five profiles per. And then the margin, this is an interesting one. When Netflix is making an investment in a TV show or movie, they kind of can guess how many people will watch it. So they guessed that 100 million people would watch Stranger Things, so they were willing to invest 500 million. They guessed that 20 million people would watch BoJack Horseman. I'm a freak. So they're willing to make a hundred million. I call that right sizing the investment. but that's how personalization helps Netflix to build margin or to build a better business. Anyways, back to your original question. I mainly was just trying to find these high level theories and hypotheses that I hope would delight in hard to copy margin enhancing ways. The reality was we'd have 10 ideas and probably six of them would fail, but a lot of value in the three or the four that worked. When we went to TV-based systems, we actually couldn't an A/B test at first. So we were going back to qual there, you know, looking over over people's shoulders and focus groups, etc. But eventually Netflix could A/B test what's a good experience on a TV-based system. They have server-based systems now where they can create different experiences for different folks. So I guess my short answer was so hard to find these product strategies that worked. There was no notion of building a perfect experience of all these things together. So we just took it one theory at a time.
Got it. And then on the concept of delight. I've always wanted to ask you this question. I've never had a chance. For B2B, Do you find this is also something companies should really prioritize and do because not a lot of B2B products are delightful and a lot of them are still really successful? How do you think about that kind of framework in B2B?
Okay, for first, I think that most or all of the frameworks I use work for both consumer and B2B. Personally, I spent my whole career on consumer. The thing about consumer is you're sure to trying to catch lightning in a bottle, right? Where enterprise, you can actually walk into it a little bit more thoughtfully. You can always find your first customers, etc. Anyways, I was thinking about what's different. So for instance, one of the hard to copy advantages of B2B is switching costs, right? So I didn't bring that up in the context of Netflix. It's wicked hard to cancel. It's wicked easy to cancel Netflix and restart Disney Plus, whatever you want to do. I was thinking about, I used to complain that the user interface for B2B software was horrible. And it's like, you got to do better than this. And of course, in Netflix, we proved that a simple, easy experience actually improves retention. So I would advocate like this is important. And they would explain to me all the basics. Hey, Gib. Actually, one of the basics surprised me. The people that are using our software are using it 10, 12, 14 hours a day. That's very different from a Netflix experience. And in fact, they want and need to engage in the complexity. Like there's stuff that they're trying to do that is really complex. You can just think about our use of spreadsheets to build a cash flow statement. I'm highlighting some differences, but the model, I think, is still there. If you're starting a punk B2B SaaS company, I think at the end of the day, you're trying to find something that's 10x better than what's currently out there today. And then over time, your hard-to-copy advantage, it will be different from Netflix. It'll be different for every startup. But if you can delight your customers in these hard-to-copy ways, that also builds a business. I mean, the reason you do that, you want to feel like you don't have to compete with Peeps, really. Like, you don't like it when you're always on guard, about what your competitors would do, you sort of want to get to this place where you don't really care and you're sort of true north is, okay, how can we best serve our customer? And that's what I really love about that particular model. Lenny, I would be very careful about anything I say about enterprise and B2B because I've spent zero hours in this. Okay? Now, as I said, half of my audience is in B2B and enterprise and you hear a lot about the consumerization of enterprise software. and I think that's good. I think that's helpful and important.
Awesome. That's really helpful caveat for folks listening that if you're building a B2B business, maybe don't follow all this advice to the letter.
But I imagine there's stuff that you can play. It's so idiosyncratic. It's going to be different for each company, each startup, or different stage. You know, and that's why the tools, the models, the frameworks I use, they.
Have to work generally, but then people have to apply their own good judgment. Sweet. Okay. So what I'd love to do is get a more concrete using this model and go through a few examples of how you may have thought through problems you were tackling at Netflix through this lens to kind of see how it actually works in practice. Does that sound good? That sounds like we're gonna do some.
Rapid fire mini cases. That's exactly right. And I'm gonna put you in the hot seat. Lenny, you're the proxy for the listeners today. All right, just make sure they're easy questions.
No matter what you say, I'll make you look smart. I like that.
I just want that applied to all.
Of okay, so the first mini case study is something that I've seen a lot of people build kind of hack together is this idea of watching Netflix together with friends. I think one app is called Netflix party. So the question there is should Netflix launch a Netflix party feature? Basically letting people watch Netflix together, but in different places.
Yeah, yeah. So that's kind of a real case. A bunch of engineers at Patreon, six or seven of them, they actually enabled, if you're using Netflix, they let you to connect with your friends on Netflix, watch the same TV show or movie at the same time, chat with each other while you're watching it, trash talk each other, use emojis, what have you. They called it Netflix Party. The Netflix lawyers noticed it and so they renamed it Teleparty. It exists. You can use it across multiple games, across Disney and Netflix, whatever you want. Okay, so the hypothesis is Netflix should launch this idea for real, not just to download the Silverlight. It's a download the Chrome extension. Chrome extension. There you go.
Yeah, yeah, yeah, yeah.
I think I tried it actually. Yeah, it's complicated, right? I mean, you and I are both freaks, so we can figure it out. So the idea is it will delight customers because this sounds like fun, especially during COVID We can have this connected experience when we're also disconnected. Hard to copy advantage. We're kind of building a network effect. You and I will connected and watching on Netflix. We don't want to quit because we don't want to leave each other behind. And then the margin for Netflix, it's largely about will it improve retention? So Netflix today, 2% are canceling. Netflix is just ticked away at it point by point. It's these kinds of things that create a little bit more value that improve retention. So the way I think about this is the way that Addition of this feature would actually improve retention is if some reasonable number of customers actually used it. And so what I look out for is I don't like two percenters. So if you find an idea that only works for 2% of your customers, now you're creating complexity. One more thing to choose. What happens when I hit this button? And frankly, some complexity that everybody else building the product forgets, right? Like, oh, like this happened to me a lot. We'd have a, in the old days, there was a profiles feature. When we launched streaming, we forgot about the profiles feature for DVD. Like, ah, crap. So as a rule, I never use rules of thumbs, but these two percenters, I would kill them. If I launched something and it was only 2%, we called it scraping the barnacles. Just get rid of it. So I think the key question here is what percent of Netflix members would use Netflix Party if you launched to all? So you want to guess at that, Len?
Let's say five percent.
Five percent. Okay. I think that's a good guess. So I'm gonna give you some historical context. We actually did Xbox Party back in 2008-2009. You know, my guess with the Xbox team was it would be a two percenter. I think it barely squeaked to five percent. And then because it was only a five percent, we killed it. So I think that's a great guess. If you would said 10 or 15%, then I would have been scratching my head. could that actually have a shot at improving retention? Anyways, Netflix hasn't launched this. They could of course test it, but I think the history, the failed hypothesis of social kind of leans against it. And I just gave you two examples, the Friends Network giving ideas of friends and Netflix and the second, the Xbox party. Those are two failed instances of social. Netflix will keep experimenting with this, but in this case they chose not to. to. So not enough delight despite the fact that it could build hard to copy advantage. And if you don't have enough delight, enough usage, there's no shot at improving margin. Okay, you were awesome, Lenny. I told you you sounded exceedingly smart.
Okay, what do you got? What's the next one? Nailed it. Just real quick, I really like the reminder of just focusing on reach and making like as good as the idea might be, just always coming back to like, what is the reach of this thing? Even if conversion increases like 50% if like 10 See, it doesn't really matter.
Yeah. Such a good reminder. Cool.
Okay, next one is something that you've shared is that during COVID Netflix auto canceled apparently something like half a percent of its members because they were inactive. And so the question there would be how, why do they do that?
And how would Netflix have thought about that? Yeah, probably seen flicks' stock history, but the beginning of COVID they thought they were going to have 8 million new members. Q1 2020, they picked up 16 million, right? So that was awesome. so it was about that time that my perceptionist, the product manager at Netflix, was looking at the data focused on non-member experience. I think his name is Eddie Wu. And he's looking at data and he notices half percent of the members weren't actually enjoying the service for a year. So they clearly had entered their credit card, their email a year ago and then just forgot that they had the service. And so when he thought about it, he said, you know, I feel like it's the right thing to actually auto cancel people who using the service. They're paying. Of course, he was doing it at a time where there was great growth. Okay, so let's use the delight and hard to copy margin enhancing model. Is this a delightful thing to do? Absolutely.
That's my question. I feel really good.
Yeah.
If a company's like, hey, take your money back.
You're not using this thing. Exactly. And what's the hard to copy advantage that you'd be building by engaging in this best practice?
I imagine there's kind of a brand halo.
Just you feel good about Netflix being really good to totally. So if I told you, and this is the case, Netflix chose to auto cancel these folks. Like, huh, that's a really cool thing to do, you know? And then the margin, the bad news, you're going to lose 100 million bucks. So there is delight, there is hard to copy advantage, and you're going to lose 100 million bucks. So does that delight and hard to copy advantage outweigh the negative of losing 100 million? So I'm just going to ask you two more questions. Is this A high stakes decision or a low stakes decision? Eddie's decision to auto-cancel these folks and lose $100 million. High stakes or low stakes? You're the product manager, Lenny.
It all depends on how much $100 million is worth to the business. I imagine Netflix is such a large scale where it's not a huge deal and it's a one-off that's not recurring.
And so you give it a shot. So you've skipped ahead. You're looking too smart. So on magnitude, it's $100 million against a company that's doing $30 billion in revenue. new. And then the second thing you brought up is it's reversible, which is he could do it now, but he doesn't have to continue this practice forever. And so my joke there, I got married 30 years ago. I was anxious about getting married. My friend said, if it doesn't work out, you can always get divorced, right? So they're saying it's reversible. Now I've been married 30 years, so it's worked out. But my point here is as product managers, we feel like every decision we make is high stakes. It's good occasionally to think about, Hey, what's the magnitude? And then is it reversible? Amazon calls those, this was a two-way door decision. It's reversible. The one-way door, those are the bigger deal. And I think frankly most people would argue that getting married is a one-way door, but hey, you know, very hard. Very hard to reverse. Reversal, but expensive.
What else you got? Okay, so you touched on Netflix's recent troubles with growth. And so something that's always come up is this idea of why don't they offer a really cheap plan or a free plan that's advertised supported? and so I imagine you've thought about that a lot and Netflix has thought about that a lot. How does that work with your framework?
Well, let's fast forward two years. Netflix earnings Q1 2022 were bad. So for the first time in 10 years, they actually lost customers. And they'd like to get the growth going again. Okay, that's a different context, right?
Today. This episode is brought to you by PostHog. PostHog offers a suite of product analysis tools. including funnels, heat maps, session recording, and experimentation, all in one easy-to-use platform. PostHog is open source, so you can host it on your own infrastructure, which means that you have control over who has access to your data and makes regulatory compliance a breeze because you don't need to send user info to third parties. PostHog's app system works seamlessly with your data warehouse, both for importing and exporting data, which enables you to bring your data into one place and easily understand user behavior. across a range of touch points. If you'd like to learn more, check them out at posthog.com/lenny.
So your question is, should Netflix have a lower priced advertising supported plan? Okay, great. Okay, let's do it. We'll DHM it. Would an ad supported plan that's lower price be delightful for customers, Lenny?
I don't know if delightful is the right word, but if I didn't have a lot of cash, it would feel really nice to be able to use Netflix.
Netflix. Okay, so the possible delight is it might only cost you five bucks or who knows, it might be free. I don't know. But for a set of customers who are ad tolerant, that might be a reasonable trade-off right? You don't have to pay 10 or 15 or 20 bucks. You could pay maybe five or two. I don't really know what the right price is. Okay, so the next one is would.
Netflix be building hard to copy advantage if they did this? I guess the hard to copy pieces, if there's a network effect, they just.
Accelerate that further and that gives them some advantage. I don't know. Maybe so if they create a, they pick up a new set of people, a new set of profiles, The technical term as you get big is you build economies of scale. So if they are able to grow Netflix spent, they'll spend about 18 billion on content this year, whereas poor Amazon will only spend, you know, 8 billion. I just love saying poor Amazon because Netflix can amortize it across 220 million members. So, you know, more growth, more customers, hopefully a different segment of customers that they haven't gotten yet. So it's not just cannibalistic. could build more hard copy advantage. My argument, I'm just going to skip ahead. If you cross fingers, you are picking up a new customer profile, new customer type. So you're going to have incremental revenues, which is really what Netflix is looking for. For me, I just want to tell you why Netflix hasn't done advertising yet. We actually did do it 5-6-7-8 2005 through 2008. We put big-ass ad banners on every page on the site, even for the members. We did a retention test and it actually did not hurt retention. A big surprise. And that was like the first year that we generated like an operating income of 20 million bucks. It was like the first profitable year. We also did this other business. We were selling used discs, previously viewed discs. That's how we first made money. And Reed Hastings, the CEO came to me in 2008 and said, Give, I need you to kill advertising and previously viewed because I think we can deliver a profit in our core business. and he wanted to keep things simple, create this simple experience that's part of subscription. You have had this, we have a lot of pride and ownership. He didn't involve me in the decision, all of these things, right? He's just sort of sticking a gun to my head. He only asked me two questions. He said, Gib, who's going to be the best in the world at advertising? And I said, Google. And then his second question is, who needs to be the best in the world at personalization? And I said, We do. And his argument was to kill previously viewed and advertising to create a simple experience for the customers and so that we could stay maniacally focused on on personalization. Now, fast forward. So, on the earnings call, for him, the debate is really between simplicity, which he's a huge fan of, but during the earnings call, he said, actually, there's something I believe in even more, and that's customer choice. And he was saying, you know what? I think for this new set of customers, giving them the choice to have a $5 a month plan with advertising, he did not say the five bucks. I'm just plugging that In this case, customer choice may be more important than delight. And in this case, rather, customer choice may be more important than the complexity of advertising or maybe the stinky experience. Phew. There's something important. Ah, what he said is today it's relatively straightforward to execute an advertising-based business doing this. There's a zillion partners who could do it. So we could continue to focus on the core of what we do, which is personalization. which I thought was a really thoughtful response. So my guess is they will do advertising, but he was very contrite. Stock at half of its value, he felt he had disappointed shareholders, so I think he was looking for ways to.
Earn back that trust. Yeah, the market sometimes forces you to do something you didn't expect. I really like the Socratic method that Reid uses in the story you shared, where he kind of asks you these questions that help you get to the.
Same conclusion he got to. Honestly, I reflected on that for like five years. Like, why didn't he engage me in the decision? And then I just realized, I mean, by the way, being a CEO is a really hard job, but I realized I had so much pride and ownership, he just knew that I wouldn't want to kill it, that it would be hard for me to be objective. And he felt strongly that he believed he knew the right answer and said, Hey, I just need you to do this. And he was right. So it took me five years to process that, but there's a lot of things you do to support the CEO. That's because that job is really hard.
We're gonna talk a bit about the role of a CPO and what you learn in that journey. But one final mini case study. Should Netflix charge for customers to share their accounts with other people outside of their home especially?
Yeah, this one's got a lot of history. Okay, so I sort of told you a bit of the story, which is Netflix's growth has been excellent until now. And the reason they had that challenging earnings call is it was just really hard to forecast. So they had the influx of customers at the beginning of COVID I call them fence sitters. They just happened to join en masse. And then are they gonna leave? when COVID lifts, are they going to leave when the theaters open up again? How do I... They shut down for 700,000 Russian members, they shut off the service. You can imagine that in Eastern and Central Europe churn was a little higher than expected, just nasty. One of the things that this cloud of COVID forecasting did obscured how the large extent to which sharing happens. And so what they said was in the US and Canada, there's 100 million customers total. They think there's about 30 million folks that are sharing their email password outside the home. Okay, so how did Netflix get here? Netflix, you know, generally they've got three prices, 10, 15 and 20 bucks, and they're always trying to give people reason for choosing the $20 one. And so if you looked at their, I call it the price and plan page, it looks like a gas pump. You know, do you want the left one, the middle one or the right hand one? One of the incentives that they provided for folks to choose the $20 a month was the number of multiple streams. So they had gone all the way to four streams. at the same time. So in my household, four people could be watching at the same time. And of course, people shared outside the house, right? And by the way, I don't think Netflix was clear that you couldn't. So now if you look at the price and plan page today, they're not sort of pitching for multiple streams as the reason you should go to 20 bucks. They're pitching other things. There's better resolution. That's the key one. Better sound quality. You're going to have HDR, sound and video. and then they did clarify in the fine print that you can use one, two, or four streams for members in your household. And that was a new clarification because it used to be kind of with a family. Kelsey and Brit are my daughters. They're in the house. Of course they can use it. They go off to college. They're still my daughters. They're part of the family. Of course they can use it. Kelsey gets married. Like, okay. So this is the first time they clarified it. So now I come all the way back to your there as a potential growth opportunity. They'd obviously like to clarify how this should work. And they started testing in three different countries. Peru's one, I think like in the Caribbean. These smaller countries, I'm sure they're focused with new members to figure this out first. And so they have two different things. You could let a person outside your household, you could actually pay a little extra for them as the primary, as the sponsor, if you will. And the other is a function where a person who's sharing your account in a different household can bump up to their own membership. Okay, so this is just complicated. Can I make the DHM model work with this? Well, it explains why they did it. So they were giving multiple streams because it was a really delightful experience. And Netflix isn't in rules. They were very permissive. I don't really think they worried about this. Was it building hard to copy advantage? Absolutely. More taste profiles, right? And you wouldn't leave because you didn't want to cancel the service your daughter's and, you know, using it on the East Coast. and was it building margin? Actually, it was driving people up to the $20 price point and it was improving chart. Like you're getting more value out of the service. Okay, so now what do they do? For me, this is just testing and math. Like, honestly, let's try it, Lenny. Well, think about those 30 million folks. If you get an email saying, hey, we've noticed you're enjoying the service on somebody else's Would you like to upgrade to your own plan? What percent of those folks are going to say yes? Right? I mean, do you have a guess on that? I don't have a guess and I.
Don'T have an answer, but this is the game, right? I find with like emails, open rates and all these things, it ends up always being really tiny.
So I guess again, like probably 5%, maybe 3%. I think that's a fine guess. I'll just give you a different perspective. This is kind of like a free trial, right? They've been using a free trial and the free trial conversion at the end of a month was 90% at Netflix, right? 90% enjoyed their first month so much that they continued with the service, right? So, I mean, I agree with you. It might be five or 10%. I was just giving you that 90% to get you to imagine, huh, maybe it could be 10%, right? Maybe it could be 20%. So this is just speculation. I think they're experimenting in these smaller countries. You know, as we do this, if it were 10%, They pick up an additional $3 million in a quarter. I think that's going to help them when their growth is at a standstill. So the other question would be, would the primary account quit when you tell them their daughters on the East Coast can no longer use the service? Right?
Right.
So I think where I'm going on this is, I don't know if we can give any real insight. The team at Netflix, this is what they live for. And the team at Netflix is testing and experimenting they've probably put in an assumption like yours, 5% will convert, and now they're seeing if they're right or wrong. And they're figuring it out on these smaller countries, and a year or two from now they will have worked it all out and we'll see the answer in North America, right? But I think it's gonna take them a year or two to figure this out because it's really hard. But they're keenly motivated because they're trying.
To get the growth going again. I was trying to watch Super Pumped on Showtime recently, and Showtime wasn't working and I had to call my dad because I'm leeching off their cable login account. And turned out they moved and they changed their cable plan. And I'm probably the reason they're keeping.
A lot of it. Well, so Lenny, I was thinking about it. You know, I'm open. I don't know the answer on this one. But just you and me comparing notes. What do you think? This is the kind of behavior that I love all product leaders to engage in. I call it building your personal board of directors. You're on mine, because I've been learning a lot about newsletters from you. Right? Super helpful. But when I would have questions like these, I would reach out to my pals, like, okay, what's the right level investment on a mobile app versus desktop? Like, that was a moving target for years. And I would just text my pals and they would just give me the data. Like, that's amazing insight just by having peers in the business that you talk to, which this is the behavior that I want everybody out there engaged in. I call it building your personal board of directors.
I'd love to hear a little bit more on that, actually. How do people do that? Is it like pick a few people on a topic? and have them as kind of your sounding board.
So I didn't even know I was doing this. So this concept began for me. I was probably 30-ish. 30? 30. Yep. I got promoted to VP and I asked my CEO, how do I learn how to do the job well? And he said, I don't know. Reach out and build your community of peers. Like, oh, that was good advice. It really was. So at any moment in time, you actually had one of my board members on a podcast, Melissa Perry. She's on my Right?
Amazing.
Yeah, yeah, you want a word. So short answer is it's really easy to build your community of peers. Just keep up your former colleagues. Keep up on LinkedIn. The harder part is building the mentors. Okay? And first rule of thumb is don't ask a person to be your mentor. That's really awkward. First, identify them. Say, you know, this is a person I think that could be helpful to me. And then find ways to be helpful. So everybody needs help. Everybody, everybody, everybody. so to help you understand the kind of help I need, my greatest fear is aging ungracefully, right? So I'm trying to like understand what's going on. Like, okay, you know, I asked you earlier, should I be on Discord? Right? Like, oh crap, do I have to be on Discord? But in many cases, somebody can help me to understand Discord. That's super helpful to me. And that's how those sort of mentor relationships eventually end up. And between the peers and the mentors, the mentors can help you to see around corners. actually, my favorite story of somebody who did approach me, they were in data. They wanted to be in product. They kept asking me, hey, do you have any startups I can work with on the weekends? That's a good idea. I didn't have any answers. I got sort of frustrated with them. I said, just build me a website, a baby website. He said, I can't build a website. So I gave him my credit card and said, get on Squarespace. So if you go to gibsonbiddle.com, that's my baby website that John Lou built for me. That was incredibly helpful to me someday I should get a real one, but I've decided it's not that important.
I also have a baby website on Squarespace.
Exactly. We're in good company. Anyways, that's the concept of a personal board of directors.
It's really important and helpful career advice in the long term. Awesome. I'm glad we chatted about that. That wasn't something we planned to chat about. So something else I wanted to dive into beyond strategy is prioritization. You have a really great model for prioritization called the jam model that I.
Share often with folks. I'd love to just kind of hear your overview of this model and when it makes sense and just roughly how to use it. use it? Yeah, well, it's a lot easier because it spells something right, as opposed to saying, dumb, dumb. Jim, I'll just give you a good case. I joined Chag 20 Town. You know, we were inventing a concept of renting textbooks to students instead of buying them, saving a lot of money. And my first week was challenging because on one end of the hall, the CEO, Dan, was saying, grow, baby, grow. The most important thing we can do as startup is grow. And at the other end of the hall, I had my CFO partner. Greg saying, slow, slow, slow. We actually don't know if we have a business model that works. And I could tell he was sort of, the company's got like 40 people sort of driving folks crazy. So as the product leader, what the heck should I do? And the answer was I got them in a room and I said, listen, I just need you to force rank these three factors for me. Growth, engagement, and monetization. And I need the two of you to agree. So you could say growth, this is most important. When we say growth, it's basically year over year customer growth. Do you want it to be 50% or 10%? Okay, whatever the answer is, engagement is how I think about product quality. A more engaging product is a better product. So if you think investing in building a better product is super important, then you go with engagement. And the proxy metric that I use for that, Netflix was retention. And then the third is monetization. You have these customers, you have this product, and how much effort do you put into turning that into a business. So Dan said, well, that's easy. Growth, engagement and monetization. And Greg said, yeah, that's easy. Monetization, engagement and growth. He did the flip, right? I'm like, I'm going to come back in two hours. Like, you guys got to fight this out, which is sort of what happened. So they agreed on growth first, engagement second, monetization third. And then a couple of months later, it started to happen again. Greg started to say, no, no, no. And so it was that point that Greg actually left the company, okay? Which was, these are the kinds of fundamental misalignments that can wreck startups. I'm not saying that one or the other was right or wrong, but as leaders in the organization, we've got to get some of the basic stories straight, including how do we fundamentally prioritize growth versus engagement versus monetization. This is really the number one source of misalignment that I discover among startups. And startups, you're always flipping back and forth between growth and engagement. Grow faster, build a better product, grow back forth, and then later you get a little later to monetization. But this model, I find it super helpful to get leaders across an organization fundamentally more aligned.
So if you're a PM or maybe a founder thinking about using this model with the first step get your leaders above you to kind of align with you on here's of these three things, here's how we stack rank it.
Personally, I'd start with a swag. I would take a shot, stupid wild ass guess of what I think is right. So at least I started with a point of view before I shared it with my team. And frankly, the next most important thing that happens is if you're trying to create a metrics focused organization that appreciates data and learning through numbers, deciding, I mean, the hardest one is what is your engagement by How do you measure product quality? So at Netflix, it's monthly retention. But that's a hard conversation. The growth story is pretty like it's usually some percentage year over year customer growth. I mean, but it's always different. But the only variation on what you said was have a point of view yourself and then get everybody's feedback. That's frankly like when I would join a new company, I would give myself two weeks to develop the product strategy for that company, which is a little outrageous, but I would just do it fast. I would develop the swag, but then I'd go to one person. Hey, this is my best thinking. What do you think? And of course, they had been there for four years. They were much smarter than I, and they would refine my thinking. And I do that one by one, and then maybe six weeks later, I could share a product strategy across the company. But this is the value of a swag, which is a stupid, wild-ass guest.
And don't be afraid to start at that level. I really like that strategy versus going in a little hole, spending months just.
Thinking about the perfect answer and only then presenting. I find this works so well. Or hiring a consulting company to do it. Go shoot me. Okay.
I like that you're hiding your mouth.
As you say that. Only Lenny can see. Give us a podcast, dammit. Nobody can see you. Yep.
I will reveal. Okay. So on that topic, actually, I'd love to transition and talk about our last topic, which is around career or product manager. So you're the CPO at two very successful companies. Or maybe you're called the VPR product.
But roughly head of product.
You get these two companies. Makarima. That's right. Chief Makarima. And in theory, this is kind of the end state of a PM career if you stay along the path? A lot of PMs go in different paths, but in theory, this is kind of where your career is heading. What would you say it takes to become a CPO someday? What kind of skills do you think you need to build that are maybe not obvious to someone that's not in that role yet?
Yeah, let's take a little step by step. So individual contributor and then to manager. So when you're just starting, you're just trying to learn the job. And I'd say at a high level, you're just trying to optimize for learning in your entire but for individuals, building products hard and there's a number of technical skills that are required. So yes, some technical, but a lot of creative skills. There's some consumer science, that's the A/B testing. There's the management. How do you get people working together to actually build stuff? These are technical design skills, right? These are the technical skills. So early in your career, you're just trying to learn the job and by the way, the job is hard. And second, it's different from every company to company and different different stage. So you, Lenny and I, we're trying to be helpful, you know, in a leveraged way. But the key thing is the job's a little different in everything. So just try to become expert in your area, expert, whatever your one thing is that you're building. And then the next step to become a manager is there's a lot of communication sort of the heart. And that's why I try to product strategy is a great way to communicate, you know, what's important, what's not. I try to demystify it. If you're early in your career, ask if you could be on interview teams, you know, even if you're not hiring, because then you're starting to practice something that's really important as you want to grow your career later, which is it's really hiring and recruiting people. So when I was in the thick of that, I spend one to two days a week hiring and recruiting. It's really the most important thing. And then some amazing opportunity would come along at the company and they'd say, you know what? Maybe somebody on Gibbs team should do it, right? Like, because they knew I could replace people. There's always somebody new to pop in. So I really was expert in hiring and recruiting later in your career. So if you want to be a muckety-muck, my theory is that the skills of a leader are the same whether you're head of product or the chief financial officer or head of data. So when I'm interviewing any of those kinds of candidates, I am looking for leadership skills. So can they do inspired communication of a vision? I do look for product strategy skills. That helps you to frame what that vision is. I do look for management skills. Have they built and managed teams? I look for people who are proactive results oriented. For me, leaders lead. You can't be a follower. So I look for really proactive results oriented folks. This is pretty nuanced, but later in career I look for folks who understand how important culture is. because culture helps people to understand the skills and behaviors that are wanted of everyone within the building, and they let you provide leadership in a highly leveraged way instead of using evil processes and meetings and rules. I look for people who appreciate culture as a tool to help lead organizations. And then of course they had to grow up as a builder. They had to learn all those damn technical skills for product management. If I abstract outside of all that, honestly, the thing that I probably did best, I think maybe I did three things best. So to know me, I was an English major, I'm non-technical and I didn't care. First I was in marketing and then I went into product, which I just loved. I think I did a good job of sort of optimizing for learning. So actually one of my hacks was I started doing talks. I used to call it Topic de Samedi on Friday mornings about something I had just learned. That was the easiest way for me to learn something. So optimize for learning. we talked about building a personal board of directors, and that has been incredibly helpful for me. And that probably gets to the third thing. I was a pretty good picker, and the reason I was a good picker of companies is I would lean on my personal born at director. Some of them were CFOs. I'm like, hey, Barry, I'm thinking about joining the startup. Chegg. He's a CFO. He's a VC. Would you invest in it? Right. It's the same question I have to ask. Should I invest my so I had people like that who helped me to isolate, you know, was Electronic Arts a good company to join in 1991? Yes. You know, was Netflix a good company to join in 2005? Yes. Was Chag a good company to join in 2010? Yes. So I was fundamentally a good picker. And I would really give a lot of credit to that personal board of directors that I had for that.
What about just like day to day as a PM, what have you found to be really good habits? and kind of like a routine that PMs follow that you've seen to kind of contribute to success of an ICPM especially.
My short answer is begin your day with intent. Okay? What are the three to five things I'm hoping to do today? Second, minimize meetings. Okay? Minimize meetings. That sucks the life out of everybody including you. Spend a lot of time with your customers. It could be focus groups. It could be usability it could be looking at survey data, it could be digging in the dirt to understand their behavior to the data, and it could be designing and executing A/B test results. Those are all ways of becoming the voice of your customer, and that's a big part of your job as a product leader. Find a balance between doing and thinking. So most of us do, do, do occasionally stop and ask yourself, okay, what's important here? You know, what should I really be doing right now as opposed to the things that I enjoy doing? Like self-managing yourself. The only reason people meet managers is that we need someone to force us to do the things that are important that we don't enjoy doing. Okay? So I'm always self-managing myself. Give. You have to do an invoice. Crap. I gotta bill people. That's no fun, right? But inside voice. Yeah. That inside voice. Yeah. The invoice. Give. You should write a damn book. Ignoring that inside voice.
Yeah.
You and me both. And then, I don't like for me, exercise is awesome for me. It keeps me happy. And don't watch too much TV. And I watch too much TV. So there you go. And don't do what I do, do what I say. Right. Yeah. I also probably watch too much TV.
Funny coming from someone that helped build Netflix, but I totally get it. It's too good. It's too good to resist.
Well, my wife's trying to cure cancer. She's like, oh, congratulations, give you, you helped invent binge-watching, you idiot. What good are you doing for humanity?
Brings us to light, and that's valuable.
Yeah, thank you very much.
Final question. You've had this illustrious career. What's one piece of advice that you give a PM in the beginning stages of their career? And then maybe as a bonus, so on, maybe mid-stage, what comes to mind?
Yeah, I think I did the early stuff. Pick the right company. At some point, understand that you're responsible for your career, not your workplace. and that's one of the reasons I encourage people to start building that personal board of directors so you can compare notes. You know, the other thing I say is don't listen to your parents because they're a generation ahead. They really don't know what the future looks like. I mean, I know I would give bad advice to my daughter and said, don't listen to your parents. And then later, so ever since I stopped direct deposit, I think of myself as just purely career hacking. So in your career, It's just a lot like building a product. You have theories and hypotheses. You find ways to experiment with them and then you are successful or you failed. So one of my hypothesis, whatever it was six years ago, was I would enjoy teaching. I actually tried it. I taught at Stanford. Entrepreneurship for graduate level engineers. And I really liked it, but I didn't love it largely because I was required to be in Palo Alto every fall. I couldn't travel as much as I liked. to, right? Okay, so on one hand, okay, don't teach in the classroom and my next vector was what if I teach outside the classroom? That's what I really love to do. I do talks, workshops all around the world in the last two years virtually. I just love doing that and I have a ton of flexibility. So that's just a little example of me treating my career like what's my hypothesis? Finding where to experiment with and then based on the results, do the next thing. And I've really been doing that my entire career. So if you embrace this idea of experimentation, you're like a product, you need to have hypothesis, you need to try, see what works, not. The next thing you need to do in building a product or building your career is be bold, okay? So don't wimp out. Like, you know, at some point when I'm working with different companies, they all kind of get attracted to the small incremental wins, and they sort of forgot what made them a successful startup in the beginning, which is taking on fundamental risk. And so I just nicely encourage people to be bold, to go a little bit out of your comfort zone. because that's where you learn more. And that comes back to this idea of optimizing for learning. Anyways, how do you encourage people to be bold? How do you encourage them to try new things? And the simple thing is just start. Try it tomorrow. Like for me, the baby step was when I'm on talks, I would just ask my friends if I could drop by and do a talk. I just did it. I didn't overthink it. I didn't spend 97 years setting up my display and getting the right click I just gave a really bad talk, okay? But that's how I started and then I sort of optimized from there.
What a perfect way to end our chat. Really inspiring and helpful advice that I'm gonna try to re-listen to and use myself. Just to close, where can folks find you, reach out to you, and then also just how can listeners be useful to you?
That's a great question. Thank you. The first, I would say that Lenny has been a mentor for me in AskGiv product newsletter. so I reached out to him via Twitter, asked him lots of stupid early questions, so he was incredibly helpful. So probably the most important thing is to know that I write a newsletter every two weeks. That's the cadence I'm on now. Lenny knew me at the beginning when I was doing dailies. He's like, Dude, don't burn out. So it's called Ask Gibb. I answered like 63 questions so far. I really have enjoyed it. Second thing would be my baby website at gibbsomville.com. .com I have the advantage of being the only Gibson Biddle on the internet. I'm sure you're the only Lenny but nobody can spell your last name, right? Yeah and then the third, how can people be useful? So you'll, I'm a feedback freak so you'll notice at the end of every talk I do every essay whether you find me on Medium or Ask Gib, wherever, there's a link to give me feedback and it's always the same question on a scale of 0 to 10 where 0 sucks, 10 is excellent, how likely would you be to recommend this to a friend? That has been incredibly helpful to me and that's how I've learned to slowly get better at everything I do. The insight I get each day from folks is amazing. And I'll ask one follow-up, what would you like and what could be better? And it's just been incredibly helpful. So don't ignore that link at the end. Don't treat it like the United Airlines asking, how was your flight? This is important, damn it. And your feedback back. It's incredibly helpful to me.
So thank you. I'm gonna go answer some of your surveys now. Thank you so much for doing this.
I learned a ton and I really appreciate your time. That was great fun, Lenny, and thanks.
A ton for your help with my.
Ask Gib newsletter as well.
Forever and ever. Okay, cool. I'll hold you to that. That was awesome. Thank you for listening. If you enjoyed the chat, don't forget to subscribe to the podcast. You can also learn more at lennyspodcast.com I'll see you in the next episode.
More from Lenny


How Meta designed Muse: an AI agent people actually want to talk to | Alex Cornell (Design Director)


Why every company now needs to think and operate like a lab team | Josh Woodward (Google Labs, Gemini, AI Studio)


Jev: 8 real use cases for the fastest, cheapest model I’ve ever used | John Lindquist


The grief, loneliness, and burnout sweeping through the tech industry right now | Molly Graham






