The Mysteries of Detection Engineering: Revealed!

Security.

Hi there. Welcome to the Cloud Security Podcast by Google. Thanks for joining us today. Your hosts here are myself, Tim Peacock, the product manager for threat detection here in Google Cloud, and Anton Chuvakin, a reformed analyst and esteemed member of the cloud security team. here at Google. You can find and subscribe to this podcast wherever podcasts are sold and distributed, as well as at our website, cloud.withgoogle.com/cloudsecurity/podcast. If you enjoy our content and want it delivered to you piping hot every Monday afternoon, please do hit that subscribe button on your podcasting app of choice. You can follow the show and argue with your hosts on Twitter as well, twitter.com/cloudsecpodcast. This is a fun episode today because we are talking about the discipline of detection engineering, which I think I can summarize and give away a joke from the show as putting a PM in your SOC. What do you think about that, Anton? Is that a fair summary?

Actually, yes, but I also think that there is a lot more to this. And frankly, the whole bringing engineering discipline to the detection space has been a lot harder than people think because we do see a lot of people who are crafting detections as opposed to engineering them. So they are talented craftsmen, But they're not engineers. Maybe their titles in the SOC needs to be security craftsmen because they're really good. They make detections, but can they scale? What about the notorious bus test? What if they're hit by a bus? Does your SOC go kaput? Like, what happens? So to me, the engineering part is the elusive, difficult part of building detections that work well, get improved, that scale, that catch the attackers you need to catch, don't waste time. To me, there's a lot more engineering needed, and I don't feel like many organizations actually have engineering in their security engineering.

I think that's a really reasonable point. One of the things I've gotten feedback on in the past is avoiding the appearance of artisanal rules. We don't want to be in a world where it requires 10,000 hours of expertise to craft a rule. You need to be able to churn out high-quality rules that consider the user at the end of the day.

Well, you sell rules for money, sort of.

Yeah.

You and me are product vendors. But if I am in my own SOC and I'm building my own custom rules, I think the same logic is somewhat applicable, but it's less critical.

I agree.

You can be successful, but then you'd have a lot of fragile elements because of your approach to detection. If you are a detection craftsman, you may be successful, but the whole thing, the whole system, Again, another tip from the podcast that you'd listen in a few minutes is that the whole system approach is critical.

I think that's absolutely right. So perhaps with that, let's turn it over to our guest today. I am delighted to welcome today's guest, Keith McCammon, a co-founder and problem janitor at Red Canary. I assume that means you clean up problems, not that you're a problematic janitor. I want to start off with, I hope, what's a softball question. Could you tell us What is detection engineering? Is it different from building rules? What's detection engineering?

Softball may be highly controversial.

Exactly.

I always try to steal from others when I'm trying to think through the definition for this. But in general, you know, Mauricio Velasco, who's with Splunk, has done a bunch of research on this and always appreciated like the simplicity of his definition, which was just the continuous process of building and operating analytics with the goal of finding threats, right, while applying engineering concepts, which is obviously really important. Like, that's the definition that I usually work from. In my head, there's a bunch of ways to unpack that. But I would say in terms of how that differs from building analytics, you know the way that we've always approached detection engineering at Red Canary as an example is just that it's one part of an entire security operation system. And so recognizing kind of like any piece of software you build, right? It's like you know it's made up of a whole bunch of parts, and it's important that each of them be done really well. But you can build high quality rules and analytics. Those are inputs into some other thing, right? A queue that someone has to burn down or investigative leads that someone has to run down. And so. There's a bunch of rules or principles you might apply if you're just building rules or analytics or things to kick over the fence, but those are going to be very different than detection engineering in the context of like an entire working operations system.

So would you say the magic is in applying engineering discipline? Because I feel like we're sort of talking about how is it different from just a bunch of people writing rules for a SIEM or for an EDR. Like, what is the magic? What makes a team of people who write rules to be a detection engineering team rather than a bunch of people writing rules? I feel like we can go double down on this one.

The first one is absolutely, I think, approaching it as an engineering discipline, thinking of it as software, right? And so, you know, writing rules for a SIEM in some domain-specific language, you could be doing detection engineering while you're doing that. But I would say, like, the real differentiator there is absolutely, like, keeping in mind what's on both sides of that analytic. And what I mean by that is really thoughtfully prioritizing what you're looking for in the first place. So applying a lot of rigor kind of in the same way that you do software engineering, right? Like applying a lot of rigor to like what you're building and why, making sure that you're building the right things, making sure that you're looking for the right things, making sure you have like the right inputs into that system. And then on the tail end, for our part, The analytics are really critically important. That's how we achieve coverage in particular to make sure you don't miss things. But maybe the most important part of our detection engineering system, at least as we've built it, is that the suppression engine that we use to deal with false positives and keep those under control in a way that's very specific to the target environment, that makes all the difference in the world, right? You could lose thousands of analytics on the world. I think we've proven that, like, thousands of analytics might give you thousands or millions of alerts. Definitely doesn't make you safer, right? And so, applying the same level of, like, engineering rigor that you do to building any piece of software, right? And just having good inputs, having good measurability, and things like that.

So, it's writing rules that are prioritized with an eye towards metrics.

No, I think it's a whole system also, right?

No, no, it kind of sounds like he invited a product manager to hang out in his sock, and he thought, wow, this is a good outcome.

Funny, I would say maybe the product manager that's missing from most SOCs is a really good intelligence team. I love the way that you like the simplicity of how you kind of stated that, and you're right, working on the right problem, right? And working towards the right thing is critically important, right? And like that product manager for us is our intelligence team, right? Like, yeah, what is the universe of threats we might look for? What's the subset of those that we can see that are important or high impact? And then like, you know, there's a bunch of other dimensions you can apply to that, but that's an awesome way to frame it and think about it is like, who is the product manager for this thing? And And on the tail end, once you build your product, make sure it's working, right? And that's really the job of the operations management, detection engineering team managers, making sure that there's kind of an overwatch on those rules and analytics and making sure that you can measure them in all the right ways to figure out which ones you need to iterate on.

But wouldn't you say that it's a little bit of a whole system approach? Because you mentioned what's on the out, what's to the right, what's to the left. To the left of the rule, they have intelligence, and to the right of a rule, we have somebody dealing with a signal, right? So it sounds like the engineering is not just applying excellence to the middle, but kind of a whole system excellence where you consume intel, consume knowledge about the attacker, which is kind of our second question. And then at the end, when you engineer your detections, you also think about people who would deal with them, right? That's another angle that I feel didn't get enough attention.

Yeah, 100%. Literally, systems thinking just dropped into a security operations context, right? Turns out that works really well, right? It's just, it's an easy thing to overlook. And having all those moving parts, like having a good Intel team and even having like the systems and the time to go do like thoughtful analysis of the investigative leads that a given analytic is kicking over the fence to your CERT, right? Or to your investigative team. Like even just like having the time and attention to do that is super rare, right? I don't know what percentage of teams are actually doing that, but it's definitely not most of them.

We both mentioned intelligence, but like for some teams I've encountered that the challenge is like they learn something about the attacker, and they want rules, but they don't really have a clear approach for how to turn intelligence they've gained, they've discovered into rules. I think, Tim, we asked your colleague from Google, right, Julian, right, who runs some of the detection engineering team here about this. And he gave us some view of the process that his team uses. But could you outline a bit more about how do you turn intelligence into usable logic?

Like, there's 3 components to it. So the first is like the intelligence team and operations in the first place, right? And so That's a whole bunch of intake, right? So things that we see, things that we learn from InfoSec Twitter, things that you read about in reports or white papers. There's threat research, right? Like that we have to take into account. Like there are theoretical threats that get published and/or proven. We don't know if we've seen them in the wild or we don't know if we ever will, but all of those are the inputs. And so there's a tremendous amount of just prioritization and kind of like thoughtful operations that just goes into taking all that stuff and racking and stacking it, right? Based on your intelligence requirements. So the next step, we've identified the requirements. And so like we refer to them as threats, right? And so threat can be a piece of software, it can be a person. It's very abstract, but we've got a set of threats that we care about. The function of threat research, which comes next, is to then take that thing and understand how it works, right? What are the behaviors? What are targeting criteria they use, right? What's the software they use? Where do we need to observe it? And so like, you know, a lot of that is breaking that down, you know, whether you use ATT&CK or some other model, but figuring, you know, answering some of those key questions. in the context of a threat that's important. Once we've done that, it becomes pretty easy. You could kind of see the linkage between that and the detection engineering team, right? Like now we know what does it do? What do we know with high confidence? What do we know with low confidence? Where are we going to see this thing? Like where in our telemetry or our data sources are these signals going to appear? Now the detection engineering team, it's definitely not an easy job, but now they've got a pretty concrete foundation to work from in the context of that threat, right? You can kind of look through your data sources and now you're just trying to make sure, hey, with Using the tools at our disposal, are we describing the software, the behaviors, the relationships that describe that threat? It's a system, right? So it's iterative. You know, we tend to like really optimize for things that are interesting, much more so than things that are accurate when we go build those rules and analytics, right?

Mm-hmm.

So we're okay with false positives. We've got really good tools to deal with those. And I think any detection engineering team that's like dealing with like high quality at scale, like having tools to deal with false positives really effectively is important, but really like not It's kind of a sweet spot between being really specific and being overly general. Look at all the PowerShell things and find the bad one is not a good strategy, but building 10,000 targeted analytics for every single implementation you've ever seen is also not a good strategy. That didn't work well for AV. And so, it's somewhere in between. And then going back to that kind of systems thinking vein is understanding the tools you have at your disposal and using those to figure out how to find that balance.

Right.

Really optimizing for not missing things, but in a way that you can afford.

So that balance and landing on the right balance point, how do you think about the problem of telling good detection from bad? And perhaps even more interestingly, how do you tell good detection from great detection?

Yeah, and funny enough, this is my obsession for the last few weeks, if not more. So I'm so looking forward to Keith's answers.

Anton, isn't this like the obsession of your career, as far as I can tell?

Okay, whatever, if you have to say it.

I think there's a couple ways to think about this. Good detection, we'll think of as like the simplest thing that you should be willing to pay for, is going to have good coverage for the majority of threats that you expect to see day in, day out. And so the places where like you'll see people optimize that I would say maybe aren't good detection is like you can definitely over-optimize for finding extraordinarily clever attacks. You can really try to optimize heavily by looking at past 0-day and trying to predict how you're going to find future 0-day, when in reality, like, whether you're dealing with a state adversary or whether you're dealing with a ransomware operator or something in between, like, most of the techniques that you're going to see them use are going to be really consistent. You're going to have a very small percentage of threats in practice are going to use some really novel technique that you've never seen before. Or going to require some really esoteric form of analysis. And so I would say good is really like, you can find that most of the time by just looking at who can explain concepts like coverage, prioritization, visibility really simply. That's usually good detection.

So you're trying to say that much of the ML stuff, and we are saving ML jokes for later, that sounds like much of the ML stuff would fall out of that category, right?

I would love to talk about ML. We should absolutely bookmark that for a few minutes from now. So that's good, right? It's optimized for the things that you need to do with it most of the time, right? And it does them well and it does them consistently. The difference between good and great is really like how well you can operationalize that thing, right? I always like to remind folks, it's no one's hurting for security technology to buy, right? If you want more alerts or you want more security tools, you are not going to have a hard time finding people to sell them to you. But going from tech technology or a tool you can buy to an operational capability that's really effective and that through testing and otherwise, like, works when it matters. I think that's the difference between good and great. And it has absolutely nothing to do, or very little to do with detection and everything to do with like operational excellence, right? Being able to take the same set of rules or analytics and be able to put those into practice and go from 1,000 investigative leads To the one thing that you need to stop what you're doing and go address, right? And that's what's missing for most folks. And I know, like, that's the only reason that Red Canary existed in the first place. And I know the only reason that, like, a lot of just MDR and the security services market exists is because it's exceptionally hard to go from even a really great product to an operational capability that's like finding, containing, responding to things in an amount of time that's meaningful and prevents damage.

So it sounds like you're not just talking about what's on the left, what's on the right, but you're talking about like a broader frame set in the people process side around the detection logic. So it's basically good detections and great detections aren't about the language, aren't about the type of regexes or type of code you use, but it's kind of about the frame around the detection logic. Like what happens with stuff? What kind of stuff comes in? What happens with stuff that comes out? Who looks at it? What do they do? How do they refine? How I'm interpreting it maybe in more actionable terms. Is that fair or not?

I think there's another piece I heard in there too, which is great detection is quiet. Good detection is catching things, but great detection is not catching the wrong things. And it's easy to go buy more findings. The challenge is buying analyst focus on the right ones.

Yeah, absolutely. I agree with both of you. And I think that, yeah, that's absolutely right. I mean, it's like the outcome you want, particularly if you're paying someone a lot of money to do this for you, is like not units in the of alerts. It's units in the form of confirmed threats that you have confidence in and can go respond to, right? That's kind of where rubber meets road here in all cases.

Perfect. We're going to sort of move a little bit out of that frame and kind of talk about not just telling good from bad, but also perhaps applying different approaches to test the detections either you built or that others built. So I know Red Canary has this whole framework for testing things, but how do you approach testing detections, whether your own or competitors' or community detections?

We're very operationally focused. So for us, it's actually, we have the detection engineering component, right? So I would say the engineering-driven component of testing, that is that detections are software, right? Each analytic is effectively like a little software program where you have things that you're selecting out of the data. So you have your domain-specific language, you have the things that you're looking for, and it's state and activity, right? It's a very living thing. And you've got unit tests for those, right? And in some cases, you go a step further and you have functional tests for those, but making sure, just like any piece of software, that you don't have a regression down the road, right? So that's like, I'd say, like, you know, in the context of detection engineering in particular, that's how we approach it. I mean, I can't remember, it was a couple of years ago now, but huge milestone for us when we realized that, like, we had written 10,000 unit tests just for our analytics, right? And that's not the rest of our platform. That's not our engine, like not our pipeline. That is literally just tests for our analytics, just to make sure those things behave the way that they should, that you can't come along later and tune it and introduce a regression that makes it not work at all. And so that's the engineering component. I would say in terms of, you know, the more practical component, you know, at the really high end, I think everyone's familiar with the really high end, right? Which is that like you either build or you hire like a really skilled adversary, right? Someone to come in, they're introducing like evasive behaviors. They may know some things about you, they may know nothing. But you're hiring someone to look and act and behave like a real adversary, right? And that's at the really high end. The thing that we'd love to see everyone do a lot more is just, you know, what we kind of affectionately refer to as taking your vitamins, right? Which is like, we use Atomic Red Team. That's an open source project. It's not really ours insofar as we've done, you know, a small percentage of the work to make that thing go, but it's really a community effort. But that kind of very atomic approach, which is, hey, like at a technique level, Identify a number of implementations of that technique, and in the simplest form, copy and paste a thing and see what you see, right? Like, did you have visibility into that technique in the first place? Forget about detection. Like, can you see it? Like, when I do the thing, does anything come out the other end in terms of logs? And then you can start to figure out, hey, do we detect it? Once you've figured that out, then you can go think about things like prevention, right? And so that really, really atomic approach You can build on that really simply by just saying, hey, like this adversary or this threat will do these 3 things in sequence. However imperfect, just doing those and again, always finding that sweet spot, right? Like alerting on a technique might be a terrible strategy operationally, but if you take 2 of them and look for them in order, that could be really high fidelity, right? And really actionable.

Alerting on an IP address is horrible but maybe useful, right? Here and there.

So this is a super cool debate. We've had this a whole bunch of times internally. 10 years ago, the market for threat intelligence was that, right? It was domains, it was IPs, it was hashes. And most of the reason for that is because, like, there were really no tools to, like, take to operationalize behavioral detection. Times have changed, thank goodness. Like, one way to think about that balance is, you know, we look at behavioral detection, even the best behavioral detection, right? Like, if you have a really good suppression system and a way for, like, dealing with false positives again in a very environment or, like, target-specific way, You can keep this under control, but say for the sake of argument, like, the very best behavioral detection you build is still going to be a relatively expensive way to find things, right? Anything I have to send to a human to look at is expensive. Now, if I find it the hard way, right, a whole bunch of behaviors, human figures it out, decides it's malicious, now you're absolutely going to have infrastructure and things like that that you can fingerprint. And like, the way that we think about using those is just like They might have a half-life of hours or maybe days at best, right? But you're absolutely defending the rest of the team from finding the exact same thing the hard way over and over again, right? So great to find it the hard way. And sometimes, like in often cases, like it's the only way to find it. But when you do fingerprint infrastructure, there's no such thing as cheating, right? Like, you know, if we've got your infrastructure pinned, we got your infrastructure pinned. That's not a good strategy to lead with those types of things in place of rules or analytics. But, you know, maybe if you're kind of thinking of this linearly, you build your rules, you build your analytics, you do your detection engineering, you do your investigation. On the end of that, maybe a great rule is that you should absolutely come out with some sort of fingerprinted infrastructure, some static indicators. There are very few attacks that, like, once you get to the end of figuring them out, you don't have those things. And if you're finding and collecting them in that way, like where you have a lot of confidence in them, huge time saver. Usually really actionable. They're not going to last forever, but that's fine, right? I'm like, that's okay for a period of time, in particular when you're dealing with an incident, right? And you're very rarely dealing with just one bad thing at a time. That means that you've got time to move on to the rest of the stuff in your queue, which even if that only saves you an hour, it can really be a lifesaver in a pinch.

I want to shift gears a little bit and make sure that we live up to the first word in our podcast name, since we are the Cloud Security Podcast by Google. How does this change when you're building cloud detections? Is it different in cloud?

I think the number one way that it changes is that we fundamentally, like, security doesn't like just apparate out of nowhere. Like, it only exists in the context of the technology that we use, right? Day in, day out. So technology's conceived of, gets built, gets adopted. Now we have to figure out how to keep it safe. It took, I don't know, 20 years For us to get really good at doing detection on Windows, Mac, Linux, like computers that you can stare at or hold in your hand. So I would say, like you know, maybe as a leading answer, like one way that this is different is that there are very few people that understand how cloud works, right? A lot of it is a black box, and that's on purpose. Like we pay a lot of money for black boxes so that we don't have to deal with the complexity of the software that goes behind it. And so just understanding the technology, which by the way is always being changed on the back end, right? Like by Google, by Amazon, by Microsoft. Microsoft, by whomever. We don't get to know how those changes are made or what the impact is. Like, I'd say that's like, as a going-in position, one way that it's changed is like, we just don't understand that technology well enough. And like, that pace of change for cloud-based technology is orders of magnitude faster than traditional computing platforms. So like, not even getting into like, you know, how you build rules or analytics or get visibility, it's just like, we don't understand the technology all that well.

Right.

It's a black box, like, to us as defenders by and large. Whether it's infrastructure as a service or software as a service, I think the same thing applies. And so that's one way it's different. You can't really effectively protect or defend what you don't understand at a really low level.

I agree with that.

For better or worse, understanding this stuff at a low level is extraordinarily hard. It is a moving target.

Interesting. I was expecting you to say something about identity and its role in adversarial behavior.

But I think the black box in nature is a really good observation. Because if people are used to taking apart packets and what happens in the Windows driver DLLs, then suddenly that part is kind of gone to an extent. Again, it's not gone from your life, Tim, but it's gone from the cloud consumer life who doesn't have access to the black plane and all the fun stuff. So the unicorn cavalry, we have to have some time.

Oh no.

To the question about is ML unicorn cavalry coming? So what do you think of a role of ruleless such as ML detections? Detections where we sort of don't have to write rules as such. So is this coming? Is this happening? Or are we already at peak ML? What's your take on this?

I love this topic. I am absolutely not qualified or intelligent enough to get into it in a ton of detail. But I will say, you know, we have this conversation internally a lot. Like we get questions from customers, prospects, our own employees, hey, like, how do we use ML and AI to do detection? The short answer is we don't.

I love this answer. This is a great answer.

To the point of me not being an expert on this, I will absolutely point to an expert on this. The talk that for years now I have pointed folks to when we get into this debate is actually former Googler Halvor Flake, who did an awesome, awesome talk in 2017 on just the complexities of using ML for threat detection. Machine learning systems are meant to take in training data, learn from that training data, and then meant to find like things with little or no human interaction. The thing they're not meant to do is deal with adversarial change, I think is the term that he used, right? That said, like, I think they're extraordinarily useful for detection engineering. And where I think they're probably like most applicable is that like we take in a petabyte of endpoint telemetry a day. We probably have an exceptional training dataset for activity that is good. That's really useful if you can point machine learning at that to defend your analysts from things that are statistically Not at all likely to be bad. You can see so much good stuff that you can select out a lot of that good stuff and that's a downstream improvement. Nobody sees enough of bad things to be able to apply machine learning. I'm not sure that it exists. The thing that always stuck with me about this talk and some of the other stuff that Hal Varflake did was that if Google doesn't have enough data to train machine learning to find new bad things, it's very unlikely that anyone else does. So I definitely think machine learning has a place, it has value. It's not the thing that I would pin the hopes and dreams of my organization on when it comes to detecting novel threats. You could absolutely use it for optimizing your overall detection engineering system. That would be my short answer, not so short answer to where and how machine learning fits. That's the last thing that I would say about machine learning is that it is not meant to deal with adversarial change. It doesn't deal well with people Trying to deliberately trick it. And for better or worse, like, that's our job all day, every day, right? Is effectively trying to find people who are very deliberately and very cleverly in a lot of cases trying to evade technology, trying to evade machine detection, trying to evade human analysts. And it's not well suited for that, right? It has a place and it fits into the system, but it's not a system unto itself for threat detection.

So this means basically it's not coming to save us any day soon. That to me is a useful conclusion, which I fully share.

Anton, I think you might be putting I think you're putting words into our guest's mouth there a little bit.

Okay.

But I agree with you, Keith. I think that the challenge of this problem of playing against people that are trying to win is what makes threat detection such a— I mean, it's the thing I do all day. It's the thing I love because of exactly that problem. Image classification, puppies look like puppies. It's a different world.

That's one of the best analogies that I've ever heard, right? Yeah, we're not trying to find cats that are trying to look like puppies, right?

You're right. It's a totally different ballgame. So I think I get the pleasure of asking our traditional closing questions today. For audiences and listeners who are engaged in detection engineering or rely on detection in their day-to-day lives, can you give them one concrete tip to improve their lives? And I'll follow that up with, do you have any recommended further reading for our audience on the topic of detection engineering?

In terms of tips, absolutely approach it from a systems perspective, right? So do not try to go do detection engineering without understanding operationally How are you going to work? Right? Like what are your intelligence requirements? Then you can figure out how you go detect the things that you've prioritized and and figure out how to effectively respond to them. That would absolutely be the one key takeaway is just like this stuff never exists in a vacuum. You can't just go sit down and build MITRE ATT&CK coverage in the form of analytics and expect to be successful. In terms of reading, Ray Canary blog shameless plug, right? But I do think we've tried to write a lot of really good educational material as it relates to very painful lessons we've learned. Some of the things we've done. That have been successful for us. Other things that I would recommend checking out, I know like the Splunk team has done some good work related to detection engineering in general. Palantir blog, alerting and detection strategies, and that approach has always aligned really closely with like how we've approached detection engineering. So like a very holistic approach to understanding threats, doing thoughtful threat research as almost the very last thing you do, going and trying to figure out, okay, like what's the analytic or analytics that'll find this and send it to the investigator? Those are a couple of the things that I'd go check out.

I really like those suggestions.

This was awesome. Thank you very much for being on the podcast and looking forward to, well, having more conversations about this.

Thank you both. This has been awesome. I wish we had another 2 or 3 hours.

I think there's so much more we could talk about here.

I don't know.

This has been a ton of fun for me.

And now we are at time. Thank you very much for listening and of course for subscribing. You can find this podcast at Google Podcasts, Apple Podcasts, Spotify, or wherever you get your podcasts. Also, you can find us at our website, and notice that we are no longer joking about having a website. We have a real website, cloud.withgoogle.com/cloudsecurity/podcast. Pretty easy to remember. Please subscribe so that you don't miss episodes that, as Tim mentioned, come every Monday afternoon in the US time zone. You can follow us on Twitter, twitter.com/cloudsecpodcasts, and also your hosts are on Twitter as well. Anton underscore Chuvakin and underscore Tim Peacock. Tweet at us, email us, argue with us. And if you like or hate what we hear, we can invite you to the next episode. See you on the next Cloud Security Podcast by Google episode. Security.