
Mike Lyons separates launch metrics (readiness, reliability, performance) from engagement metrics (actual usage and value). He once measured only launch readiness and missed user adoption entirely. Paul Hearn's North Star framework splits engagement into attention, transactions, and productivity. Hold a metrics workshop early with stakeholders to align on what success looks like at launch and three months in.
Chapters
Let me tell you about a mistake I made. I started a lot of these like that. Well, let me tell you about another mistake I made. Early in my product owner career, I was so focused on getting this cool new feature out the door that I forgot to ask a basic question. How will we know if anyone actually uses this thing? We see we had all these great metrics about the launch. How fast does the page load? Are there any bugs? Will it work on all browsers? But three months later, when my boss asked me how the feature was doing, I had no idea if anyone was actually using it. You're in big trouble now, mister. Well, that taught me something important. We actually need two types of metrics, Launch metrics and usage metrics. Now I like to use the word engagement metrics instead of usage metrics. But the idea is how are we going to measure engagement? Think of it like opening a restaurant. Launch metrics tell you if the kitchen works. Usage metrics tell you if people are coming back for more. Okay, let's look at launch metrics first. These are your getting it out the door numbers. How many bugs did we catch? Does it work as fast as we promised? Is the code clean? Is the system stable under load? Are all the security checks passing? Can we roll back if something goes wrong? And is the monitoring set up right? These launch metrics, they work alongside your existing metrics to give a full picture of launch readiness. But launch metrics only tell half of the story. I've seen perfect launches of features nobody wanted. That's where engagement metrics come in. These are your Is it getting used and do they like it numbers. I was reading a great article by Paul Hearn about the North Star framework. He studied how over 11,000 companies measure success. And he breaks engagement metrics into three simple attention, transactions and productivity. Let me share eight solid engagement metrics pulled from this research. These are battle tested by companies like Netflix, Spotify and Airbnb. Weekly active users. This tells us if people are making our feature part of their daily routine. Like Netflix tracking how many subscribers actually watch shows through to the end each week. Time spent using the feature. It's like YouTube measuring watch time because time spent usually means that the user found value. Completion rate of core actions Are people finishing what they start. Like Airbnb tracking completed bookings, not just searches. Return frequency how often do users come back on Spotify? They watch this like a hawk. They're looking at daily listening patterns. Successful transactions for features where users need to accomplish something specific. Like Uber tracking completed rides, not just how many times a user opens the app. User productivity rate how efficiently are people using the feature? Think Microsoft tracking How quickly people complete tasks in Excel Engagement depth Are users exploring all parts of the feature? Facebook measures this with their interaction patterns and adoption rate across user segments. Are all types of users finding value? Well these metrics, they tell us the real story of whether our feature is actually making a difference in users lives, not just whether it works technically. Remember, it's not about tracking everything, about tracking what matters. Pick the metrics that make sense for your specific feature and your users. As I was thinking back to that mistake I told you about at the start where I had no idea if anyone was using our feature, it wasn't just about picking the wrong metrics, it was also about when we picked them. That's why now I host a metrics workshop early in product development. Get everyone in a room. Stakeholders, developers, designers and a couple of users if you can get them and ask three simple questions. What does success look like at launch? What does success look like three months later? And most importantly, can we actually measure these things? This early alignment does three powerful things. First, it forces everyone to get specific about their expectations. No more vague we want it to be successful statements. Second, it gives the technical team time to build in measurement capabilities. Because sometimes tracking the right metric means adding code or tools, and that's why it's easier to do at the start than after launch. Third, it creates a sort of shared scoreboard. Everyone knows exactly what we're aiming for and how we'll know if we hit it. These metrics become your North Star, sort of guiding everyone towards the same goal. Great products don't just happen. They're built on clear expectations and measured results. The time you spend getting metrics right at the beginning will save you months of confusion later.










