Nicholas Arcolano explains how Jellyfish now measures DevOps performance by tracking deployment and incident data. The company connects DORA metrics like deployment frequency and incident response time to business outcomes, then shows how customers use these measurements to improve engineering health across their teams.
Chapters
So my name is Nicholas Arcolano. I am the head of data science here at Jellyfish, and my team, uh, one of the things that we do is, uh, we lead efforts to, um, research new data sources, uh, and, um, that we can bring into the Jellyfish platform and, um, A data source that we got really excited about, about a year ago, is what can be done with data coming from deployment and incident systems. And so since then, we've done a lot of work to, to learn about the entire DevOps space, these DORA metrics, what can be done with this type of data, the power that it has, the Jellyfish platform. And we've worked with dozens of customers to help them bring data like this into Jellyfish and to set up DevOps metrics and to understand what we could do to create new features in this area. And so in the rest of this talk, I'd like to get into a little more about why we think it's important to measure these things, talk about the specific type of metrics that we're talking about, share a little bit about the customer journeys that we've seen, where different customers are coming from, what they're trying to accomplish here in the DevOps space, and then provide a little more detail on what this new feature and new integration looks like for Jellyfish and provide a demo as to what you would see in the app. So why measure DevOps metrics? I expect some of you folks here listening are already sold on this stuff, which is why you're here. For those of you that aren't, maybe you need a little convincing. When it comes to metrics, I'm sure many of you have had this experience where there's no perfect metric. It's very difficult. You're in the situation where things that you care a lot about, like say revenue, are at one end of the spectrum where they're very relevant but far downstream and not very actionable. To all of the things upstream that contribute to them. Things that are closer to the work, closer to the ground, that you can measure day to day, that can change how your teams operate, it's hard to know which among those are important. And there's dozens of metrics out there, and we provide a lot of them in Jellyfish, and we think a lot of them are important. But, you know, at the end of the day, you have to pick some things to really care about. The kitchen sink approach only gets you so far. And you have to measure things, because if you can't measure it, you can't improve it. What's exciting about DevOps metrics is that this is a class of things that describe the health effectiveness of your engineering machine that are— there's a huge body of research that links these things to business outcomes. They have been proven to be good things to measure. If you improve them, they— you achieve better business outcomes. So there's, there's been a lot of research around this. You know, it was popularized originally in the 2008 book Accelerate. And so, you know, elite teams that perform at the highest levels across these metrics are correlated with better business outcomes.
