Our New Podcast Format and The Ideal Hook Length (Experiments Dec 2023 Week 3)

All right, so we're going to talk about experiments for this round. And we got— we're going to start off with this new format that we have going on over here, which we haven't talked about too much, this new pod format. So we are basically doing 15-minute episodes for 5 times a week versus what we had going on before, which was 5 episodes— 7 episodes a week for 5 minutes. And we believe it's going to add a lot more in terms of retention. And the first episodes actually start coming out next week. I will say this so far. So far with this experiment, Neil and I are putting a lot more work into ideation, a lot more into preparation, which we kind of didn't before. And, we believe it's going to be a lot better. And plus, we're thinking about maybe every once in a while we'll bring in some guests for a 15-minute episode and we'll just kind of shoot the shit with them. But, I think the episodes are a lot more fun as well. And, we haven't reinvented on a format for maybe 7 years or so. And maybe that's why it's been fun.

Yeah.

But it feels like it's gotten a little stale. And now I think it's starting to become even more fun now.

So Eric has these stats, I don't have them, but there's the number one reason we want to end up doing this. So Eric, what percentage of our listeners are repeat? I know you have that stat somewhere. You want to share it?

What was the rough percentage? No, I have a retention graph I look at. And so let's say like 100% of people come in month 1. What ends up happening is like 13 to 15% come back in month 2. And then 7% or so come back in month 3. So that's not a very good retention graph.

It's not a good retention graph. So the podcast gets over a million listens a month. Now think about it this way. We're losing 87% of the people who listen in month 1 from on to month 2. So instead of focusing on cranking out 30 episodes or 31 episodes or 29, depending on how many days in the month or whatever it may be, We want to switch our focus to creating better episodes longer. And the big experiment here is we don't think that the podcast is going to generate that many more views per episode, but we're hoping the long-term benefit is because the quality is higher, the retention will be better. So instead of 13% on month 2, if we could get it where it's 25%, 30% on month 2, And, or let's say if it's 25% on month 2, 20% on month 3, et cetera, the numbers are game-changing. And that's what we're hoping for. So yes, less episodes, but if we can get the same listeners to listen to more and more episodes, we would be much better off. And a podcast that does a great job of this is the All-In Podcast. If you look at it, a lot of the people that listen to All-In Podcast have been listening to the All-In Podcast for a long time. The retention is great. Longer form episodes, more curated topics, not using ChatGPT to just come up with your topics, not being lazy about it. That's the overall goal with this experiment. We'll report back to you in maybe a month or so what's actually happening with early data, or maybe it'll take us a few months, but that's the goal. And when we've done longer form episodes and from talking with people and all the surveying we've done, we do think it'll solve it.

You know what's interesting, Neil? We're going to talk about this in a couple episodes, but the All-In podcast, when it first started, I was listening immediately. So this was in 2020 during the middle of the pandemic. And it started with Jason Calacanis and Chamath. It was just those 2 first. And then Friedberg, they have come on as a guest every now and then. And mostly what they talked about was their investments, tech and investing and all that. Now, along the way, they added David Sacks and eventually became the besties, the 4 of them. And it evolved into more around politics. It evolved into more around current events and it became more entertaining because Sacks and J Cal would go after each other. They bring on guests and all that. The key thing I'm trying to call out here is that in the beginning, it was kind of boring, right? Like for tech nerds like us, like Neil and I, like we like listening to this stuff, right? But when they started going into politics and all that, sometimes Neil and I would call each other. It's like, man, they're talking about politics now and all that. But then eventually it's like, well, it's kind of entertaining. And so it's really important to reevaluate how your format looks and how it feels to you and iterate, right? And that's what we're doing with this podcast.

Yes. And we'll end up finding out what happens. But early data, because we've tried longer form episodes, shows us that when we produce longer form episodes, it helps a little bit with retention, but we've never tried just doing all long episodes. So directionally, our first experiments that got us to where we are today, Have showed us that it could help.

And the final thing I'll say is, and then we'll move over to your topic, which is your X experiments. But you got to think about your— if you're a content creator, you got to think about almost like you have a software product. And software products, what matters at the end of the day is how well you retain people. And so you can look at your retention graphs, your cohort retention graphs. You can use something like Chartable to look at and you can see how people are retaining over time and how you've improved. And I guarantee you our cohort charts are going to get a lot better, Maybe I can screenshot the before and after. But if we're able to, to Neil's point, we can retain 3 times more, maybe 21% by month 3 or so, we're going to have a compounding effect. Because what happens is these people come back and these people also continue to share as well. And then, that's how the downloads will stack because it means the product is a really, really, really good product. And that's what we're aiming for at the end of the day.

Yep. Another trend that we're seeing or more experiment, not even a trend, is I'm posting on Twitter daily. And we talked about this on one of our episodes last week. And I'm taking the most popular stuff and then I'm posting it on Instagram, LinkedIn, et cetera. See, with Twitter's algorithm, in case you guys missed the episode, if you post 5 times in a day, 6, 7 times in a day, it doesn't hurt your previous tweets or post or whatever you want to call them now because it's X. But on the flip side, if you post on LinkedIn 5 times in a day, it hurts the reach for some of your earlier posts. At least from all the experiments we ran, and that goes with most social networks. So the concept was you take what's working on Twitter, you know, it's just like word vomit. You're just putting out a lot of ideas. You take the best stuff and then you put it out on all the other social networks and you record it into video and long form. You take the same exact text used on Twitter combined with the video and you see what happens. On the flip side, what we're seeing with Twitter and the algorithm throughout these experiments, and we've been seeing this more over the past week and a half, and I've been telling Eric about this, getting likes on Twitter, like hearts or whatever you wind up calling it, it's not the same in which if you get a lot of likes, and I have some tweets that have a lot more likes, and I'll give a prime example of this, okay? So I had something that I put 10 hours ago, 22 reposts, 15 comments, 89 hearts, 12,000 views. I posted something a day ago with 13 comments, 410 reposts, right? You're talking about 20 times more reposts, 554 hearts. So you're talking about roughly 6x more hearts and it only has 18,000 views. And keep in mind, the one that's 12,000 views only been 10 hours. So what's the big difference? Because the one that's, you know, at 10 hours won't get as many views or won't get as many hearts and reposts, but yet it will get more views. Well, the big difference is, is on the one that was 18,000 out of 1,225 engagements— I'll actually share my screen here. This could actually help. So share screen. Okay. Look at this, right? So you see the post analytics here, 13 versus 15, 22 versus 410, 554, 89, 12,000 versus 18,000, right? So if you look at the analytics here, 1,225 engagements, 186 detail expansions, that's people clicking see more. If you look here percentage-wise, 343 engagements, 188 expansions. It has a higher ratio for people not only expanding the content, but also visiting my profile. This one is already 41 and 2 followers while the other one has 30 profile views and 2 followers. And what LinkedIn, I mean, what Twitter really cares for is how many people click show more. You know, I don't care what they say. I've done a lot of tests with this on multiple accounts. It looks like that has a lot of weight in their algorithm. Maybe they say otherwise, but we've continually seen this and I have countless examples of this. You know, here's an example of a movie one, tons of engagement, but again, not a ton of people clicking see more and stuff like that because there wasn't anything there. And then, you know, as you can see on my screen, some have more likes, some have less likes, but it really comes down to how many people are clicking see more. Like what is marketing? 29 comments, 227 reposts, 412 likes, 28,000 views. On the flip side, this is how effective Google Ads really are. 29 comments, so same as before, 59 reposts, roughly 1/4, 313 likes, which is roughly 75%, but it has 79,000 views. Check this out, okay? 79,000 impressions, 2,000, Engagements of 2,026, 1,171 detailed expansions. On the flip side, the 28,000 views, 1,081 engagements, 309 detail expansions. As you can see, that only has 1/3 versus the other one has 50%. People clicking show more has a massive impact. And what we're finding when we create content from the other networks, if you just base it off of Twitter views, it's really off. And I know directly Twitter views, more likes, hearts, et cetera, more you're going to get. But what we found is if you actually base it on not Twitter views, but what's getting the most reposts and hearts, and then post it on the other networks, we're finding that to be a better leading indicator of what's going to do well on the other social networks.

Cool. My next one will be, oh, actually I might have a 2-part here, but the one experiment that we've been running on the YouTube side is we've been experimenting with different lengths for our hooks, right? And what I mean by that is when a video starts, we'll have an intro. Could be a 15-second intro, could be a 25-second intro, could be a 35-second intro. And what we found is that the average viewership at, let's say, after the initial hook on 15 seconds or so, is maybe about 50% or so. Okay? So average people viewing at 30 seconds, I should say, after the hook, right? So 50%. Now, for 20, 25-second hooks or so, I should say 20 seconds actually. So 20-second hooks, it's about 54% or so. And then for 25-second-plus hooks, maybe about 30 seconds or so, it's about 59%, right? So when you look at the numbers on that, basically, we are talking about an 18% increase just by having a longer hook, right? And these are hooks that are— we're pulling from some of the key points of like when I'm interviewing someone, like hot points that will get people excited about the video that they're going to watch. And so this is a test that we ran over a series of maybe call it 20, 30 videos or so. So the sample size is really small, but it also seems to make intuitive sense as well.

No, totally. Question for you is what sample size will you end up leveraging?

It's a small sample size. So I'm talking 20 to 30 videos and we're still testing this a lot more right now. So I'd have to come back to it, but this is an early test right now just saying, hey, let's try longer hooks and see where it goes. The other thing to call out here, we should have called this out in the trend section, but Neil, I threw up something on—

And then have you tried hooks that are like really long? Like, I don't know what's really long, but like a paragraph hook instead of like, you know, a quick sentence or 2 sentences, but like actual like whole paragraph that could be a hook lengthwise.

No, I haven't tried that yet.

Oh, okay.

No. But when I say hook here, I'm talking about like a well-produced introduction. It's almost like It's almost like a sizzle reel for the first 30 seconds, if that makes sense.

Yeah, no, that's why I asked if you're using something that's like a paragraph or so, because I get it.

You're—

it's not just like a sizzle reel. You're using like storytelling to like really draw them in and appeal, uh, evoke emotion. But the question is, is if you made it even longer, what will happen, right?

Like, yeah, imagine watching— It's worth trying.

Yeah, imagine watching a 2-hour movie. The hook isn't usually, you know, 10 seconds. The hook could be 5, 10 minutes long, right?

Dude, that's like the intros that are like 2 to 5 minutes long. You know what I mean? And that is very much a hook in itself.

Yeah.

The other thing I want to call out before we get out of here is there's a rise, and this should have been the trends episode, but there's a rise in creator founders. And so we talked about this over the weekend, but someone wrote a post about the rise of writer operators. And the way I look at this is I think there's creators like MrBeast, Emma Chamberlain, Logan Paul. But there's also, there's going to be a rise of creator operators coming. And what I mean by that is actually like people like Neil and myself, we probably spend 80% to 90% of our time on the business and maybe 10% to 20% of our time on content. For creators, it's more reversed. And I think there's a lot of value in being an operator because the level of experience that we can bring, the stories that we can bring, it's like even the questions I ask in a podcast are going to be very different than what a creator asks. And I think there's also, there's a content void right now when it comes to people that are creator operators or business creators, whatever you want to call them. And I think that's going to continue to rise because entrepreneurship is just going to grow in popularity. And I think there's going to be, we're going to see more and more solo entrepreneurs. We're going to see more and more entrepreneurs out there. And so, I think people, those of you that are listening right now that this is a business podcast basically, right? I think you're in a really good place right now to jump into this blue ocean. to become a creator-operator. And that operating experience, the wisdom that you get, that is what experience is at the end of the day. Like wisdom comes from experience, right? And that operating experience is super undervalued, I believe. Please don't forget to rate, review, and subscribe. Please let us know what you think about this format, what you like and what you don't like, what you'd like to see change. Just go ahead and tweet at us for it. And that being said, don't forget to rate, review, and subscribe, and we'll see you tomorrow. And also check out this video over here. And yeah, goodbye.