In this presentation Milan and Brian from Jellyfish frame an engineering organization as a complex system that transforms many inputs—customer feedback, business strategy, budgets, internal roles—into the simple output of working software that satisfies customers. They argue that to improve delivery you must measure outcomes rather than just outputs. Outcomes are business‑specific goals such as speed, quality, alignment, productivity, collaboration, and clear processes. Measuring a single metric can cause the system to ‘cheat’ (Goodhart’s Law), as illustrated by the nail‑making analogy where focusing only on count leads to many tiny nails, or focusing only on weight leads to a few heavy nails—neither meets the true goal. Therefore, they advocate measuring each outcome in multiple ways. Milan outlines the six outcome areas Jellyfish tracks: alignment (working on the most impactful work), productivity (volume of work), speed (end‑to‑end lead time), quality (delivering reliable, bug‑free software), collaboration (effective teammate feedback and healthy conflict), and process (visibility for stakeholders). The talk sets the stage for using Jellyfish’s Life Cycle Explorer to surface data that supports these multi‑dimensional outcome measurements, helping engineering leaders identify bottlenecks and accelerate delivery.
Chapters
And I wanna start not by talking about Lifecycle Explorer, but by level setting a little bit on how we think of engineering as a system. We think an engineering organization is an extraordinarily complex system, but like any system, it takes inputs and it turns them into outputs. That output is super, super simple. It's working software that makes your customers happy. But all the inputs are pretty complicated, right? There's customer feedback, business strategy, all of the market conditions that you have to deal with, whatever budget the engineering org is given. And then it has, has a ton of moving parts inside of it. You have to deal with your own engineering strategy. You have to deal with management, product managers like me, designers, developers, like Brian, the mission of the team. And all of that engineering organization system is designed to take those inputs and turn that into the output of working software. But just like any system, like any system in the world, it can be measured. An engineering organization is a system that can be measured. And while this isn't easy, we think you can make it a lot easier. Now, when we think about a system that can be measured or thinking about the engineering organization as a system, we think there's 3 keys to measuring the engineering organization as a system. Number 1 is you don't want to just focus on the output of working software. You want to focus on the outcomes that you're going to deliver. And within the system, you want to focus focus on the outcomes that you want to happen within that system. Do you want to go faster, or do you want to go have higher quality? These are all outcomes that you can think about. When you start thinking about these outcomes, we think it doesn't just make sense to focus on those outcomes, but to focus on the outcomes that make sense for your business. If you're a company making software for pacemakers, your outcomes are likely pretty different than the outcomes that we go for as Jellyfish, where we're a SaaS company helping you understand what's going on within your product and engineering organization. Both of those are going to be pretty different, and you want to focus on the outcomes, you know, like quality for the pacemaker company versus, you know, speed and customer value for Jellyfish. That makes sense for our business. And then when you, when you know the outcomes that you want, when you know what you're going for, you want to try to measure these outcomes in multiple ways. When you start to measure these outcomes, it doesn't make sense to just say, you know what, we want to go fast, so we're only going to measure how long it takes something to get into production. And that's because of some really important points that Brian is actually going to talk about shortly. Brian, do you want to talk a little bit about, you know, why we want to focus on multiple things?
Yeah, so I actually love this slide because it's simple and it's like just astonishingly true, right? We just spoke a lot about, you know, outcomes versus outputs. And when we talk about measurement, it's really the same concept, right? Like we're trying to measure towards our way at a certain outcome. We're not measuring just for the sake of measuring. And we do that because regardless of what you measure, the system will cheat, right? If I am an eng— engineer and my team lead manager cares a lot about the number of issues I close, well, I'll just write 5 issues for a certain task instead of 1, and I'll close all of those things. And then I look really good and life is awesome in sort of my world, right? But that's not really what we're after. I think a lot of folks have probably seen the next slide here, which makes the nice sort of analogy out of the engineering world and, you know, moves it over to a really simple task, which is making nails, right? If we're just measuring for the sake of measuring, we might only care about the number of nails made, and that could result in a ton of really small nails. So then we might change our sort of approach and what we measure and say, well, I want to actually care about the weight of nails. And then we would end up with a few really big heavy nails because the system has cheated, right? Neither of these outcomes are good if what we're actually after is a bunch of the correct size nails so we can hang up a bunch of awards, family photos, what have you, right? So the whole point here is that when you have a measurement become a target just for the sake of hitting that target, and we're not actually looking at the outcomes of what those measurements are trying to achieve, those actually cease to be good measurements. So we've talked a lot about outcomes. Milan, do you actually want to chat about a few of those and what they might actually look like?
Yeah, I'm happy to. And I think, you know, these are a set of outcomes that we've seen a number of customers focus on. We're going to be focusing on and talking a lot about speed today. But I think depending on your business, you might care about any number of these, or you might care about some of these more than the others. Internally within Jellyfish, we think about these 6. We think about our alignment, which is how much are we working on the stuff that is the most important stuff to the business? Are we dealing with, you know, a bunch of minor bug fixes, keeping the lights on work, or are we really making progress and aligned around here's the thing that's really going to make an impact for us, for our customers, and really drive the business forward? On top of that, we also start to think about productivity. This is just the sheer volume of work, and are we doing enough volume? A lot of times if we get hung up and, you know, we end up stuck on something, it's because we're not putting out enough volume of work. We're struggling to make a decision, and that usually means that we're not producing enough. We should just put something out there, make a decision, and see if that works or not. Same thing with speed, right? That's what we're gonna focus on today. So I'm not gonna belabor that too much. But speed is all about how long does it take us to do something from end to end? Can we do things fairly speedily or fairly quick? Or do we take a long time to really get feedback and learning from the rest of the market? Quality is probably something we all think about and that we all care about, right? Are we actually delivering good things to our customers? Do we deliver things that work for them? Do we deliver things that are broken? Do they have bugs? Or, you know, God forbid, do we cause a bunch of incidents and outages for our customers that really make us, you know, struggle to actually even measure if we're doing something well or not? And then two more internal outcomes that we think about here are collaboration and process. Collaboration is all about, are we actually doing a good job of being good teammates with one another? I think we firmly believe, and Brian in particular has said this many times, like, we have to be good teammates to one another, not in terms of just being nice to each other, but actually giving each other good, meaningful feedback. That means having comments on PRs, asking questions about PRs, disagreeing with somebody's approach if you disagree with it, and having productive and healthy conflict. But that's also a fine line to walk. You don't want to have too much conflict because otherwise that devolves into you know, hoarding code or having difficulty actually getting things out and done. And then the final bit is process. Um, a big piece of what we think about with process is how well or how clearly are we giving our stakeholders visibility into what we're doing, what we care about, what they should care about with what we're doing, as well as giving them visibility to are we gonna be done on time or not? Are we going to actually be able to deliver the things that our stakeholders want, whether those stakeholders be, you know, the executive team, customers, or just some of the other engineering teams that we might be working with.
