Metrics that Matter for System Optimization

So my family and I, we like to take road trips. And for me, since I do most of the driving, I'm really focused on, like, how fast can we get there? Or maybe I'm focused on where the next rest stop is so I can use the bathroom. I drink a lot of coffee on these road trips.

I know this isn't the best time, but when you get a chance to pull over.

So I'm always watching the speed limit signs, I'm watching my speedometer, and I'm making sure to reach the maximum allowable speed. Now if I were only focused on speed, I might be ignoring other things like how much gas we have left in the tank, do we have the correct tire pressure, are the kids getting hungry? Or what's that smoke coming out of the tailpipe? It's the agile equivalent of focusing on the wrong metric in agile delivery, it's sometimes easy to fall into the same trap. We obsess over metrics that don't actually tell us if we're delivering value or improving our system. We need metrics that provide actionable insights and and also support continuous improvement. I'll often see a focus on easy to obtain surface level metrics that fall short in giving us the information we need to make informed decisions and optimize our systems. Of these, velocity obsession is particularly common. Many teams will focus solely on how many story points they can deliver each sprint. Now, while velocity can be useful, obsessing over it often leads to local optimization at the expense of overall system performance and value delivery. The this is a prime example of a vanity metric. It's easy to measure and looks impressive, but it doesn't necessarily correlate with delivering value to customers or improving the system as a whole. So an important shift in managing agile projects is to start measuring outcomes. Now. We'll still need to measure outputs, but our primary focus is to optimize for outcomes. Charlie McGinnis proposed a model that presents a sort of tug of war, highlighting the tension between the different aspects of delivery. Consider the link between productivity and sustainability, or speed and quality. This is the most common one I see, and also between value and predictability. It's important to recognize that there's always a tension happening between metrics. When you optimize for one, you often have an impact on another. This is why it's important to establish a balanced dashboard of metrics that helps visualize the push and pull and gives us insight into how our entire system is performing. So consider these metrics as you create your scorecard for productivity. Consider the Sprint burndown chart. This is going to help teams visualize progress against our sprint goals. Now be on the lookout for flat lines indicating potential blockers or maybe lack of updates. And for sustainability, we could use Employee Net Promoter Score, gauging team engagement and satisfaction, which is critical for long term success, as well as the other side of the rope, which is productivity. Okay, for speed, we could measure cycle time. That is the time from when we start a piece of work until it's complete. This will help us identify bottlenecks in our process. And for quality, I like to look at escaped defects where we measure defects found after delivery. This helps us understand the effectiveness of our quality processes. Now consider how we measure value. This could be things like top line revenue growth or bottom line cost reduction. This helps us understand the true business impacts that our efforts are having on the organization's profit. And the other side of that is how we measure predictability. This is where I would suggest something like committed versus completed. Compare the number of work items committed to those that were actually completed. The this will help improve the forecasting and planning of our teams. Now, something to keep in mind here, this whole tug of war thing, it's just an analogy. The point is not that when you optimize, say for predictability, you're always going to do so at the expense of value. It's that every time you optimize for one thing, you're often going to have an impact on another thing. It's not like there's a direct unbreakable rod between sustainability and productivity, but they will influence each other. By balancing them, you can get a pretty holistic view and avoid local optimization by focusing on improving the entire system. I once worked with a team that was solely focused on increasing their velocity. Increasing velocity is a terrible idea. They celebrated when they hit the 50 points per sprint mark. Unfortunately, customer satisfaction was dropping and the number of sev2 tickets was increasing. So we introduced cycle time and escaped defects as additional metrics. Well, this revealed that while we were delivering more features, quality was suffering and valuable features were getting stuck in long cycles. By shifting focus to these more actionable metrics, the team was able to improve their processes, reduce defects, and ultimately deliver more value to the customers even though their velocity actually decreased. So this whole thing, it's a balance. And it's important to find just the right balance of these output related metrics and these outcome related metrics. Output metrics tell us about our delivery process, like are we being productive? Is our quality high? Are we predictable? And outcome metrics tell us about the impact our work has. Are users adopting our features? Are we processing cases faster? Are we meeting our compliance requirements? By monitoring both types of metrics, we can ensure we're not just building a feature factory, but actually delivering value and achieving desired outcomes. The goal of metrics in agile delivery is not to create pretty charts or hit arbitrary numbers.

This is a pie chart describing my favorite bars, and this is a bar graph describing my favorite pies.

It's to provide insights that drive continuous improvement and value delivery. So always ask yourself, does this metric help us make better decisions? Does it encourage behaviors that optimize the entire system instead of just local processes? Metrics are powerful tools, but only if they drive action and improvement. By choosing metrics that matter, you can transform your agile delivery from a numbers game into a value driven engine of continuous improvement.