Our Adversaries Teach Us: Enhancing Security with Realistic Adversary Emulation

Informing your decisions with intelligence analysis and insight brought to you by the team@ooda.

Loop.Com hello, I'm Bob Gourley, the Chief Technology Officer of OODA LLC and today on the oodacast, Bryson Bort Bryson is the CEO and founder of Scythe. Scythe, you probably know, enables organizations to evaluate security investments in both technology and people. SCYTHE contributes to blue teams, red teams, purple teams. Blue teams can use SCYTHE to build and test detection capabilities on their own. Security operations centers can then reduce false positives using SCYTHE capabilities. Red teams can use SCYTHE as a common platform to build and operate tools for their approaches. Purple teams can use SCYTHE for the quick wins with adversarial emulation and tailoring what they do. Bryson is also the co founder of the ICS Village which we'll talk a little bit more about shortly. This is a nonprofit 501 for education and awareness of the industrial control system security. Bryson also founded grim, a consultancy which brought a lot of operational experience in cyber missions to support business risk management. So I want to talk a little bit about Grim also. Bryson has also spent time leading the cybersecurity business at mantech. He spent four years as an army officer following school at West Point. And so Bryson, welcome to the oodacast. I want to talk about all this stuff with you.

Thanks Bob. Don't I just look like a prototypical West Pointer?

Yeah, there's something. You're in a box, you're out of the box, you're out of control.

Not out of control. I'm never out of control.

So really there are a lot of things I want to ask you about, but I want to start at the early days like you went to West Point. And would you tell us about your decision? Why did you want to go to West Point to begin with?

Yeah, so I applied early admission to West Point my senior year, so I only applied to one school. I actually got a scholarship offer from MIT even though I didn't apply there. But I wanted to go to West Point for two reasons. So the joking one is if I'd gone to mit, I definitely would have beat up my nerd roommate. And at West Point I was one of the smaller guys, so there was no danger of that. The second is I wanted a well rounded education. So of course West Point is not just a focus on academics. You have to balance that with of course being physically fit to go into the army as well as an understanding of military tactics. And the reason I wanted to go to West Point is I Wanted to give something back. So I went out of a sense of patriotism. I wanted to give something back to the country. And I thought service through the army was the appropriate way to do that. Right.

And, you know, when I think of West Point and the many people I've met who've come from there, I think leadership. One thing, it must be a great place to study and reflect on leadership and decide what your personal leadership style would be. And so I wanted to ask a couple leadership questions. Mike.

Sure thing.

What is a leader in your mind? How do you define leadership?

A leader is inspiring folks on a common vision. In some ways, it is getting people to do things they don't want to do. But overall, it is a force multiplier to getting that sum of individual parts to be greater of a whole, to execute a specific idea. That vision. Right.

That makes a lot of sense. And what's a manager? What's the difference?

A manager there is not the vision. Right. A manager is the tactical execution of particular tasks.

Yeah. All right, that's clear. And these kind of lessons, what else did you learn at West Point that you think helped you in your career to this day, that you still reflect on anything?

What did I learn at West Point? I learned it was a better place to be from than at. That's our inside joke. I was probably gone more often than the average cadet for various reasons. I think that was the first time that I learned the balance of my personal limits. The structure of the academy is set where you cannot wring more time out of the day. You have to be in your room, you have to be lights out at certain time. You have to be up at a certain time, you have to be physically present at certain places. You have multiple responsibilities, and they set it up where it is impossible for anybody to accomplish all of those things. And you learn the Pareto Rule of 80, 20 of what can I do? What do I need to do? And what can I let fall? Because that's, you know, those are your limitations of what you can accomplish on the leadership side. Actually, the greatest lesson I learned was when I took over my platoon as a platoon leader, in a second lieutenant, as a commissioned army officer was. So I was a. I was a Signal Corps officer. Signal Corps, of course, is the telecommunications or my joking way, running around sandy boxes that you don't know where you're at and making sure everybody can shoot, move, and communicate. You're the communication part. And you are one of the younger folks. Second lieutenants, 22, 23 years old, platoon sergeant's typically going to be somewhere between 15 and 20 years in. They know everything, they've done every single job and they're your right hand person to help run that unit. And all of the soldiers, even down to the 18 year old privates, have had more technical training on their equipment than you have. You are the least technically knowledgeable person on average there. And what I learned which really worked into a leadership style is I didn't pretend, I didn't run around trying to push my weight. I came in, I had my rank, but I went around and I just asked lots of questions. And I think a lot of people are intimidated to ask questions. And particularly as we get older, we can be even more intimidated because folks think that we just still should know everything. And I learned the skill of questioning through leadership or questioning leadership by questioning, get it right, questioning by leadership. And I would just hey, specialists, like what are you doing there? Oh sir, blah blah, blah blah. And I'd be like, okay, thanks. And one, they appreciated that I took an interest. Two, it's not like I gave away that I didn't know what the hell I was doing. They gave me an impromptu education and I built that up over time to get a lot smarter. And as I was able to do that over experience and my soldiers teaching me, I was able to affect a lot more. But I didn't just naturally assume it.

Yeah, it sounds like this is a trait that a lot of leaders today could really use. Let's say C suite leaders. You know, a CEO, even a cto, a CIO shouldn't have to be the one going around implementing patches on all of your OSes. And a CEO especially, you don't even want them getting bogged down. But it'd be great if you ask questions from time to time. You know, are we well patched? How do we know? How do we measure that? I like this approach of asking questions.

On that note, I want to say two things. One, I think there is an over. There's too much attention applied to the patching question and I suppose we can dig into that later. The second is that particularly in the technical field, so many folks continue to hold up as the, the benchmark either where they were as individual contributors and great coders, great researchers and have a hard time letting that go and consider, almost consider that a mark of weakness. But when you start to look back at where management leadership comes in, those bigger visions and those managing of larger teams come from you putting your hands on keyboard less and helping make a lot of other people Better.

Yeah. Makes a lot of sense. And so this as a Signal Corps officer, is that how you got your start in the world of cyber security?

No. So I was the first full time brigade information systems officer at West Point. So first basically full year s, six for the core. So I. That technically was that. I mean, I was also a computer science major, but back then we didn't call anything cybersecurity. Security was not a taught concept other than back then in the 90s, you ran around with golden disks patching things when the latest virus came out because computer networks were still a new thing.

Right. Okay, cool. All right, so let's fast forward to today. I've heard you talk before about the state of attack and defense. We'd love your comments on that. Just the cybersecurity field overall, what is our situation?

Okay, that's where I get to rant, right? Yeah, Rant. Rant time. Okay, first up, on the solution side, too many nerd solutions to nerd problems with nerd results. I'm fucking tired of it. I'm sorry. There's a reason that there has been as much money. And by the way, I'm not saying I'm the smartest guy. There are way smarter people than me who have been a part of this solution. And the problem is that you're serving at the behest of an operational concern. If you work at a company, whatever that company does is the primary purpose for that company. Security, ain't it? Yeah. You got to speak in business terms. You have to talk English in lay people terms. And do not look down on users. Do not look down on the manager who doesn't get security. It's your fucking fault, not theirs. Sorry. Okay. I'm so, I'm so. I'm so frustrated by this stuff. And I'm frustrated on the technical side, where these technical solutions solve part of the problem. And everyone's like, look at the greatest piece of this. Your largest risk area is people. It's the users. The users that use computers are the surface area. It's not the computers by themselves. If you're looking at a computer by itself, you're missing that they're sitting right next to the. Which are an employee. Anytime you tell them no. The security culture of no creates its own version of an insider threat. Not the insider threat that we always talk about where somebody's coming in and stealing company data, but it's the person who's like, I really need to do this to get my job done. Because, by the way, that's how we make money and stay in business. And the IT and the security organization is not making it easy for me. So I'm going to figure out my own way. The second greatest hackers in the world, kids, right, seems to see the same thing. You put a parental control, you put a restriction. I don't care if it's technical or non technical. They will social engineer you, they will avoid, they will lie, they will whatever they need to do to get the thing that they want to get done. So when we're that culture of no and when we're only looking at a part of the problem, we're not understanding the problem realistically at all. This, the I don't know, second, third, fourth, I don't know which point of my rant I'm on.

Well right now this is a very good point. I don't want to interrupt your rant, but this is from the CISO viewpoint is also a critical issue because imagine you're a CISO and now there's 3,000 venture capital funded technology providers that want to get one minute of your time. Not going to happen. You know, you know you have problems and you have all these people just want to talk about their technology. And it's tough from both sides of the equation here because those poor technology vendors can't get any CISO to pay attention to them and they may have nothing to offer to the business that the CISO needs to help with.

So now that I'm not quite so. So a CISO's job is to be constantly looking at the vendor market. Okay. People, you know, pick on salespeople. Salespeople are just doing their job and all of the technology by itself is actually pretty individually great. I don't buy into the snake oil thing. I've been asking for years, give me an actual company you think does snake oil. I won't throw them under the bus. There's only one company that's been put to that name that I've heard, but most of the rest of them actually kind of fundamentally do what they say they're going to do. This is where I will fault the, the end user. And by the end user, I mean the, the security team that buys them is so one CISO should absolutely be out there looking at the latest and greatest, talking to vendors, because vendors are going to, even if they're not, you know, getting the sales pitch, vendors learn from that and start to understand the business requirements. So it is a collaborative ecosystem and the more of that exchange of information, the better. Now I'm not Saying hey, you know, do the $5 Starbucks bribe for a 30 minute sales pitch. That's just a one way sales pitch. Talk to real people in that organization, in the vendor organization to be able to have that exchange of information versus a one way pitch. But it is fundamental to the viability of this ecosystem that those conversations happen. So I do believe that and I think it's necessary back to blaming the vendors. I mean not the vendors, the security teams. So what is it? Something like 25% of products that are bought go on the shelf? 25% of products that are installed are installed with their default configuration. So you've now just increased your risk or from a security perspective have an assumption of what you think you're going to get, that you're not going to get another percentage above that are not properly tuned. So a large percentage of things that are bought pretty much either make things worse or do nothing. The, the part then that I wanted to tie in back on my rant to this is the perimeter is dead. The perimeter is dead. Now I'm sure everybody's watching this right now is going like this and then is going, well, I'm going to spend all my money on penetration testing and vulnerability assessments. That's your perimeter. Do you need to spend money on it? Yes. If that is your focus, you will lose. You will lose again and again and again. There are only five initial access categories. There's only five ways to get into a company. Those five ways can be used in an infinite number of permutations. They will get in shift to an assume breach model. It's not that you give up on your perimeter, but assume it will be breached and focus on what happens next. That you can control. You can control that. You can decide what applications run or not. You can decide what communications are allowed or not. You can, you can decide what is hard to get to, what's easy to get to. If you don't know where to start. Just figure out the most important things from a business side. Circle your wagons around that, let everything else burn. At least you've got control of something. Because if you have control of nothing, you, you've got control. I mean if you think you've got control of the pockets and you have control of nothing, right? Find some way to get that. So where does this tie back into the vendors is again, security is personal. Your business is in fact, and I don't mean this in a facetious way, a unique snowflake. The constant calculus of employees and how they use things every single day and the constant change of business requirements and configurations, all that's. That's happening, you have to stay with that. And so you have to put it in your context. So in that assumed breach, that post access, understand your business, understand what the risk is in that context, and look for a way to identify that and tune your tools based on those understandings. Yeah.

Bryson, this is fantastic stuff. And you. In my youth, I believed that these were finite games we were playing, that we could establish strong boundaries. It was wrong even back then, but that was our attempt. We would put firewalls on things. We would do as strong identity management as we could. And then we think we won. We kept the bad guys out today. It's just exactly as you said. I believe in boundaries and control points, but they're irrelevant to most operational decision making. You put as many controls on them as you can, but the bad guys are getting in.

Yeah, I mean, so here's why it's so hard. So first of all, anybody that really wants to get in will get in. All right, that's the first breakdown. The second is, well, then why are they so hard to find? Right? Why? Why is it so hard to find them? Even if I know where to look, it's still hard to find. Well, the simple industry challenges that we have overwhelmed our centralized analytics. The SOCs that are meant to be looking through these haystacks for if there's a needle or there's not a needle. And again, going back to the things that haven't been tuned, we actually went more like, with the NSA problem of like, let's just collect all the information we can collect. Well, on one hand, that's better than not having the information to collect. So it does make it easier to do a retroactive analysis after you found out something has happened. But it makes it really hard to do it in real time to look at detection, response and remediation in that phasing. For those of you who have not looked at it, Sunil Yu published several years ago and is currently writing a book on the cyber defense matrix. It is a brilliant, simple way to look at how the phases of defense should be in your organization. So just that's, that's my pitch there to go and look at that. Cyber defensematrix.com I'll capture that and put.

That in our show Notes for this.

Yeah, he also has a good one on the pets versus cattle die triad for looking at cloud, because cloud is of course becoming increasingly important. He is one of the smartest people I've ever met in this Space. And those are good, easy ways to kind of look at everything you're doing through those lenses. So just shout out. A lot of my ideas are influenced by his thinking.

Awesome. Back to your idea about the boundaries. So adversaries get in, but now they're on your turf. So shouldn't it be you have enough instrumentation to detect what they're doing and then mitigate their action inside your enterprise?

It's a great question, Bob. How do you do that? So let's, let's break again, go down to like, what's the problem? First, your defensive solution is their test matrix. That's why dwell time is a thing. That's why your AV doesn't catch stuff. That's why your EDR eventually might catch stuff. Right. We as an industry have put ourselves in the Groundhog Day of understanding yesterday to catch today, to get surprised tomorrow. And the reason is so hard, beyond the fact that your defensive tooling is their test matrix. So they know what you will and will not see, they know how to avoid you is. So that's, that's the host collection point. Anything that they can do on a host, they can be relatively invisible to you. The place they cannot be invisible is the network. They can blend, they can try to look like something else, but they physically have to be there. There is no way for them to defeat the laws of physics and not be ones and zeros on those communication levels. So the ability to correlate what's happening on a host, even if you can't see it directly to network traffic, is the way to find them. And this is why the behavioral aspect of what they're doing is much more important with that event. Correlation to alerting to tip off the detection and response phases is so critical.

Right, Right. And then so this also brings to mind things like designing for network segmentation. If you're a large organization, there's a lot of things to do at the network level. And then understanding your most critical data for your particular business and layering on the right kind of monitoring on that is just absolutely a proven step. Right?

Yeah. So network segmentation challenge is you've increased the policy and the maintenance side. From a user management perspective, any heterogeneity you're introducing your environment is going to increase that. So those are things that need to be handled through, through a technical means. Manually segmenting a VLAN for a user request is not going to work well as well for you as, say, using a bastion box or bastion host. Where else do I want to go with that segmentation oh, particularly in the remote work from home environment. Something I recommend is a lot of folks are all excited that oh, we have all our users vpning in so we have some level of protection as opposed to they're just coming in, of course. Well, yes, that's true. Why not have them VMP V? Why not have the user's VPN into a least privileged zone so you effectively have them coming into a DMZ and then having them authenticate from there to specific services or assets. So a form of using segmentation as a part of your remote workforce to, to improve your security right out of the gate for what's coming in. Because that, that decentralization also makes it harder for you to have eyes and assessment of what they're doing at home. Which by the way, I would also caution probably includes mixed asset use because it's really hard when somebody is working home non stop and their family's there because we've been in this pandemic that they're not going to accidentally do things on corporate assets that they probably wouldn't do normally in the office.

Yeah, Bryce, I want to talk about so many other things including the ICS Village and Hack the Capital. But first I want to ask is anything else on the state of cybersecurity today? I think that was a pretty darn good assessment, but anything else that should be top of mind.

Do I talk ransomware?

Yeah. What's the state of ransomware? Is that another one where there will be a end to this challenge? There will be a patch to our antivirus that will detect all this and knock it out of the enterprise and we will have won.

Yeah. So I want to, I want to pick on the patching thing too. So what I was talking earlier with the foreshadowing of. I think there's been an over focus on patches. The way that patches have been given the just patched it aphorism like just patch it. That's the solution is one just patching it is going can be very complicated based on an environment. First we need to have the vendor release the patch after a known vulnerability. So there's that delay in time. And then most environments are custom. So the ability to put a patch in and make sure that it doesn't break anything, that takes time. And all of this is done while ignoring the fact that a vulnerability is not a vulnerability is not a vulnerability. So something is a 9.8 on the CVSS. So what that just means that if somebody had direct access to it, that's bad, but. Well, don't we have a defense in depth? Isn't this a system of systems perspective that 98 may never be accessible? I might also be able to mitigate it through other means. So just patching it isn't necessarily answered. Going back to your business's unique snowflake, those vulnerabilities and those requirements to patch should be prioritized within the contextual business and not just the. Well, what Bryson's telling me is I'm just kicking the can down the road to Patching something in 90 days versus today. No, it's with assurance knowing that you've mitigated the need for that patch, because that is a possibility through other ways than patching ransomware. So Chris Krebs and I at RSA in February 2020 got up on stage as part of ICS Village and the Cybersecurity Infrastructure Security Agency. You can tell I've worked with them because I can just rattle that name off for cisa. We called out the coming scourge of ransomware and did, oh boy, did it happen in 2020 and is it continuing, of course, in the news in 2021? And why? Because it's lucrative. They are making money hand over fist. Yeah, Darkside, I can't tell if it was the developers or the operators, which by the way are two different things, have been attributed to $90 million in cash in hand. So they ransomwared dozens of companies. The Colonial pipeline was actually the fourth attack in six months on the US energy infrastructure. It's just the Colonial Pipeline one got the attention because turns out you affect consumers at the gas pump. In a hydrocarbon economy, you're going to get attention. The other ones, hospitals, well, we bitch about those in news articles, but nobody's really cared and done anything about it. The gas one, the Colonial pipeline, now it's, it's gotten serious attention. So I'm looking forward to what comes out of the US government taking lead on this. But I do want to note, as I've said several times in the past, do not look past the amount of work that private industry has been doing independently without government to fight off a lot of what's happening with these kinds of gangs.

Would you give me a recommendation for, let's say, a company that maybe has a small IT department, maybe even a cio? And I hope this company is not just going to wait for the government to do something. What can a mid sized business do to mitigate the risk of ransomware to their organization?

So I'm going to give two answers. The first is the overtly self serving and then the second is going to be the technical implementation for backup. So first is Scythe is a modular post access malware platform. We can safely emulate any kind of malware, including ransomware. And it has been, we have received a lot of interest this year. And it's funny because from a site perspective that was just a small little feature that we happen to have. It's like, oh yeah, we also do that. Except for now since it's become such a part of the consciousness that's now actually the primary driver for folks coming to us because nobody else can safely do that at scale with the ability to adapt it to each campaign. The dark side one within two business days we had an emulation plan out shared with the community. We're going on a series right now where we're doing it, including we're going to launch an offering here shortly where companies will be able to download those emulation plans off the website themselves and immediately run it locally for a very reasonable price. I'm not, I'm not kidding, like we're not going to be charging a lot for that. So you can actually see what will happen with your defensive stack against ransomware. All right, taking off the sales hat. Backup, backup, backup, backup again, it's going to happen. Breaches are going to happen. It's going to happen. And assuming you haven't tested improved with us, what can you do? You can make sure that you can recover, have a great backup strategy that has two prongs, one local hot backups so that you can immediately recover, recognizing of course that those are going to be a target. So ransomware knows that, Ransomware knows that that's the biggest thing that's going to keep you from paying them. They are going to try to get to those. So the thing that makes the convenience for you to quickly and easily back up as well as restore, they're going to be gunning for those. The second part then is to give yourself a little bit more leeway and make it harder for that. And it's of course harder is the same thing that we've seen in business continuity planning for a long time. Have periodic off site backups. Yes, you're not going to be doing those every day, whether that's weekly or monthly. It's something that's going to give you a chance to come back versus the cost it could be in. I think it's, I mean the latest ransomware payment that a company just paid was $40 million Colonial CNA insurance, $40 million. Insurance was 40 million, Colonial was 4.4. So compare that cost to that cost and make that business decision. And the final piece of advice back on the backup is test your backups. There is no worse feeling in the world. Well, there might be a few, but up there in the worst feelings in the world is going to a backup in an incident and realizing you didn't have the backup you thought you had. Test your backups. Right.

Well, Bryce, I want to talk more about Scythe, I want to talk about adr, but I also want to hear about the ICS Village. ICS Industrial Control Systems. What is the ICS Village?

The ICS Village, as you mentioned in the introduction, is a nonprofit. So we're a 501C3 and we build critical infrastructure exhibits that we take around to different security conferences. In the last year and a half, that has been completely virtual. We actually did a lot of really interesting research to allow folks to interact with these inherently physical systems remotely. So that was some, that was some really cool R and D that's allowed us to do that. But some fun things we do, of course, is so we bring lots of different speakers together to talk about different aspects of it. We provide a lot of hands on training from. Do you want one on one introduction to what a programmable logic controller is? For those of you who don't know what a PLC is, that is the bottom of the ICS stack where the computer actually makes a change in the physical world. The pump goes this way, that way, things go open, they go closed. Right. It's actually affecting the physical world, which is why this is what the technology that underpins critical infrastructure, because it's cyber physical, we have capture the flag that has all sorts of different technical disciplines. So even if, again, if you're new to ICS or you feel intimidated by that stuff, everything we have has what I call the spouse test. If you've never turned on a computer, we have a tutorial to help you get started. So don't feel intimidated. And then probably our marquee event is called Hack the Capital. It is in May every year. So we just had the recent one here on May 4th, it's our fourth one. And we bring together the technical and the policy communities to debate and talk about all these different issues. One of the things we've taken advantage of in the virtual world that I think is particularly cool is the ability to bring in speakers from other countries. So we had two panels, we had a panel staffed solely by executives from the Middle East. We had a Panel for Europe, each talking about their perspectives for their regions on critical infrastructure. A particular talk that I thought was really neat was Oleg Shakirov from the PRI out of Russia. So we had the Russian perspective on critical infrastructure and it's necessary to understand even the civilian lower level perspectives on these things because at the end of the day, critical infrastructure has no nationality. And understanding and making it safer for everybody is, I think, a human problem.

Awesome. Bryson. I know for a fact that congressional staffers and members benefit from this Hack the Capitol. And it was interesting timing that just a couple days after your Hack the Capitol last Hack the Capitol event was a Colonial Pipeline event. And I sensed that more staffers and members themselves were more articulate because of what you did. So thanks. This good effort.

Yeah. We had six members of Congress speaking at Hack the Capitol, which highlights that they're getting it. I mean, their willingness to put their time and to speak on these issues is a big part of Also a quick shout out. So I, as a part of the ICS Village, in collaboration with the R Street Institute, host a monthly podcast called Hack the Plant and we bring in different guests to talk about different perspectives of critical infrastructure in plain English. And the last one that just came out actually had Representative Mike Gallagher, who was the co chair of the Cyberspace Solarium Commission. And you can hear me insult his Marine Corps heritage in the context of industrial control systems if you listen to the episode.

Well, as a transition to this concept of ADR attack detect respond tools, I wanted to test a thesis out on you, something I believe. I've looked at metrics in cybersecurity for decades. And a lot of people want metrics to help them make decisions going forward. So they look at how can I measure, what can I measure, what's relevant? And there's big frameworks built around metrics like the CIS Top 20 Control or the NIST Cybersecurity Framework. Those can all generate metrics. And over time I've come to believe this thesis. There's only one top metric that matters, and that is, is the adversary able to accomplish their objectives? Now, underneath that, there's a lot of subordinate metrics. You know, how long can they operate in your network? How long did it take them to get into your network? How long can they operate there undetected? How long until you can push them out? But that to me is a top metric. And it's not the metric that you want to check everything with. You want to check it a different way. You want to check it yourself by emulating an adversary. So that is the metric. We need thoughts on that thesis.

Yeah. So I published a few months ago a detection maturity model. And what I was trying to do there was add two things to the way folks typically look at detections, which is typically very simplistic of did something get stopped or not, did I see something or not. And it builds up from there to starting to establish what's the correlation of that tied to particular threats and then going to what you're saying, which is putting it in business context because not everything in your network is created equal. There are things that matter, there are things that matter less and then there's things that don't matter at all. And being able to understand those, those metrics within that context are important. And then starting to tie into what Dmytro Perevic has been talking about with 1 1060. So don't get caught up in the numbers. 11060 stands for minutes to detect, respond and remediate. And that's driven by his time at CrowdStrike and seeing where his consumers could effectively where they could see what was the breakout time for an attacker. Attackers do not start where they want to start. They start wherever they get access and then from there figure out where they're trying to go. And once they do, they tend to move pretty quickly. And then once they've done that, it's really hard to know what you got out. So let's use the example of the Colonial pipeline. Are we completely confident that there is no malicious code left in some other version from that incursion? I don't know. We don't even know what to look for. It might just be lying dormant. It's very hard to tell. That's why Dwell and Breakout Time is so critical to understand. And so the more advanced piece then starts to look at how can you measure that detection and response in the timing because it not only gives you tool insights but but it starts to give you an idea of professional capability and workload. Are my analysts just not able to get to things that are alerts because the alerts don't either have enough fidelity or we don't have enough analysts or there's too much workload or it's not presented in the right way, or our analysts aren't trained to do it quickly? All of those things start to come out of it because metrics by themselves, it's not the number, it's the story it's telling you.

Right? Right. Good context. Now let me mention I, I read this paper you guys did titled Attack Detect and respond. Enhancing security with realistic emulation. I like that. A lot of good takeaways for me, it helped me understand how attack detect and respond tools can be applied to multiple missions that can have direct, positive business impact. And I know for a fact it's just full of true statements like you cannot just rely on antivirus anymore and assume that that is going to cause all your problems to go away. You really do need to test but I want to get your context on this paper. So would you start at the beginning? Why is this called Attack Detect and Respond Tooling?

First, shout out to Ed Amoroso, who was the collaborator with me to help develop that paper. Second is the idea here is so attck tooling has been around for decades. There has been the evolution from simple scripts to gain access and identify vulnerabilities to more complex Red Team tooling that allows you to do different kinds of emulations or simulations at all sorts of different levels of complexity. George Orchia is my cto and then Adam Ashinshi who oversees the Atomic Red Team platform at red canary. The three of us created a website called the c2matrix.com that's a community driven compendium of all available offensive tool sets. There are over 60 of them right now today. And there's probably even more that we don't know about. Point being is there's a lot of different ways to do an attack and test that attack. And the bad news is some of those attack tools are in fact used by our are used by adversaries. We see them. So it's really hard to have an assessment with something that's used against you because you're going to end up focusing on it versus the behavior. But the missing gap was the integration. Part of that is that ATTCK tool, because the value of the attack tool is your security is defined by the threat. And so being able to realistically do that threat at scale and with the agility to adapt to the latest threat innovations is key. But having it integrated to drive business value was the missing component. And that's where we came up with ATT and ck. Detect Respond is how attack tooling needs to be focused to drive what we've been talking about. Where I'm improving my detection and response, I'm not just consuming threat intelligence and putting it into play, I'm bouncing that into what's my understanding of my vulnerability management, what's my understanding of my contextual business risk, and what's my ability to put those together for how I'm defending Myself through the phases of the cyber defense matrix from protect, detect, respond and remediate.

Is this the difference between ADR and the category of breach attack simulation?

So the problem with breach and attack simulation, why we we define it as ADR is that first of all, breach and attack is a redundant statement. Second, that industry as it is is not really doing attck. And that misnomer I think has created, I would say the trough with disillusionment is where it's stuck. It does two things well, going back to my initial premise that we have a lot of these nerd solutions for nerd problems. It allows you to easily do a threat checklist. I can, because it's an, it's an agent based, their agent based frameworks. I can run a local checklist to validate technical controls and then I can replay things through these set points so I can validate my network segmentation. And the other good thing that it does is security is a constant fight against gravity. Things break, things change. By replaying known things, I can identify if there's configuration drift in my security stack. That's what breach and attack simulation does really well. So technical control validation and I want to emphasize that technical control validation and security configuration drift, it's not giving you realistic attack understanding and people are not in the loop. There's no contextual business risk. There's where you've installed things to have those insights and the convenience of knowing that with a button click you are automatically going to be getting those insights.

I see. So it could be a helpful tool. But this is not attack detect and respond tooling.

No. For example, it, it only measures technical detection. You're not seeing how the, the staff side of that is. You're not measuring response at all. You can't measure response with something that is static. Just.

Good. Now back to ADR tools. What kind of organization should be considering these? What size of an organization, I guess would be one way to put it. Is it any company that has a CISO is large enough or any company that has a internal soc or any company with a that's taking MSSP services. How should I view that?

Yeah. So your defensive tooling is only as good as your ability to, as we said, to drive it for you. So from that context it's essential. As Ed Amorosa puts it, hey, it's the tough love. You just gotta do it. Now, I'm not quite as optimistic as Ed of just telling people they just gotta do it. It's gonna make it happen. Large companies absolutely benefit if you Have a red team. This is a common standard for them to develop all of the awesome human innovations that they bring to the table in concert with the community ecosystem that we fomented way beyond us. Right. My tech team will never stay at the edge of innovation, but a whole community sure has a chance and automatically bring those insights as easily as possible into where you're enriching the data sets where the blue team needs it. There's the ability to have lots of predefined threat campaigns. There's the ability to no code create your own, what we call synthetic malware because these are not just like general simulations. This is the payloads that you've created under your control. They're going to go and do those things to give you a realistic idea in a production environment quite safely. By the way, if you don't believe me, I'm happy to provide any of my dozens of rest of reference customers who've been using it for years. But we do get the challenge that. Okay, that, that's great, Bryson. I get it. Large companies can consume this. What do I do if I'm a. An IT manager with. Of a 100 person company with one other IT person helping me? I'm not, I'm not really gonna be able to do that. I'm. I'm so focused on what I'm doing. I outsource my stuff to an MSSP and I just kind of trust that the paycheck gets me what I think I'm going to be getting.

Yeah.

We are working to partner with MSSPs to take what is the upmarket solution down and offer it to the SMBs who I think really do need this the most. This is not a abstract thing. This is an existential threat that they're facing. So we want the platform to be very affordable through all of that automation to give them those insights where the MSSPs are able to offer that as an additional service offering to, to give insights and refine the MSSPs for their customers.

Yeah, it could be very smart and very helpful. And I would also say that personally I've tried your platform and love the fact that I could create campaigns myself. And if I can do it, I think anybody can really create their own pack campaign.

Bob, I'm not sure you count as the typical layperson.

Okay. All right. But the power user, any power user can do it. How's that?

I will give, I will give an example and this is a true story. I'm not exaggerating this at all. So I brought on a Chief of staff a year and a half ago named Elizabeth Wharton. She, she's worked on the Hill. She was, I believe, general counsel for the Atlanta airport. She's a lawyer. She's a lawyer by training. Her husband is Dave Maynard and he is an exceptional technical talent. But I'm not, I'm not gonna pick on Liz. But none of that exceptional technical talent has really rubbed off on her. And after she joined, she was like, hey, I'd like to install the Scythe platform and you know, play with it at home and see what, what happens. And Dave came over and was kind of like, hey, what are you doing? He didn't really know what Scythe was. And, and she was like, oh, I just created, you know, this, that this keylogger does this, this and this. And Dave was like, wait, what, how, how did you do that? And she's like, yeah, no, I just click, click, click. And then I had it. And it's this custom payload and I'm running it and I'm getting these results. And so we joked it's so easy even a lawyer can do it.

And that's good, that's what we need. Frankly. Another topic on this I wanted to cover with you is the Mitre ATT and CK framework. First of all, from my viewpoint, that's been a very virtuous, community driven project. The Mitre ATT and ck, would you agree?

I would agree. Let me give my definition for what Mitre ATT and CK is. It is a periodic table. It is the ability to describe, as we understand them, the elements of the world to create chemical equations that equal an attack going to. The way Mitre ATTCK has been misused is folks look at what is known and go, I'm going to run a thing that does that thing. I've now checked that box and I'm going to run a thing that does that thing. I'm going to check that box and great, you've got this heat map that makes you feel better that has accomplished absolutely nothing in the real world. Attacks are state based, hence the reason I describe. The periodic elements being put together in a certain way equals a chemical equation. Let's take it out of the metaphor. What do I mean? If I am trying to do lateral movement, that's going to be different based on where I am in the organization. That's going to be based on what privilege I have. That's going to be based on whether or not I have stolen credentials or I haven't simply checking a box that says, okay, I stole Credentials and. And then separately from that box, I'm going to do something. What have you really accomplished? Right. We need to understand the ATTCK chain in context and Mitre ATT and CK provides the common vernacular for everybody to understand that. And all of those components that we have identified of which they're it's constantly being added to and they're also adapting it for cloud and container and industrial control systems because they're different applications. What don't think of it as an exhaustive list, but it is a community driven list of a common way for us to put everything that is the combined body of knowledge on ttps.

Great. And then so your ADR capability is. Would you call it interoperable with the Mitre ATT and CK or founded on the Mitre ATT and CK or how would you describe it?

Yeah, I mean going to the list story, this is the thing we do that I think is really neat. Imagine going to the MITRE ATT&CK framework and going, I want to steal credentials from Azure. Okay. Oh, there's a box for that. Click. Oh. Huh. I'd like it to. And then there's a little wizard says I'd like to do it this way. Then it automatically builds that code for you into the campaign that's going to be custom built for you. Great. Cool.

Couple other comments, questions.

I.

We've mentioned the dynamic adversary and you know, prepare to be surprised. They're going to surprise us. So there's this massive new discipline, this field of cyber threat intelligence where folks are doing their best to figure out what the bad guys are doing and then share that data. How can that intel inform Scythe or other ADR kind of capabilities?

Yeah. So of course a threat driven defense is critical and ADR is trying to apply that in the same way that you, an artist, carves a statue out of marble. So this is the David Bianco Pyramid of Pain. For those who aren't familiar with it. It's very easy things to identify from an attacker to very complex things. The easy things are this is a bad hash value because that file of that, that malware always is going to look like that. Very easy to change. But it's a starting point, right? This is a bad domain, this is a bad IP. Right. These are all the standard indicators of compromise IOCs that are provided out of the threat intel industry, which is why we're good at understanding yesterday to save today, but we get surprised tomorrow because those things are so easy to change, but they're so easy to do. The hard part is the TTP level where am I focused on behaviors and today there is no common standard where threat intelligence is being automatically produced in that way. Today it's a manual exercise. There is a 20 to 60 page amazing analyst report written by any of the threat intelligence vendors and there are multiple ones and they put out fantastic things except somebody has to read it and then put it into play. I gave it a talk. I gave a talk at ATT and CKCON a couple of years ago highlighting this and calling for we need to create a better standard for this Sticks which is a common CTI taxonomy or template is has part of it. But we need something better for the folks in our ecosystem. We've created one internally in JSON which is very easy to one click import and have the community build and drive their own. But we need something like that. Whether it's ours or a modification. We are happy to adapt to whatever anybody else creates. But taking the behavioral level components of this analyst outputs and allowing those to be one click emulated through an ADR is a gap in industry currently that has to be done manually.

Bryce, I appreciate all that. I want to ask a couple other questions about cybersecurity and would love your views on people. For as long as I can remember I have seen study after study after study saying that the nation is short about 500,000 people for the cybersecurity career field and it seems like it's going to be that way forever. We will always not have enough people and we need more people in cybersecurity. First of all, do you agree with that statement?

Yeah. So this is the we don't have enough people. So there are a lot of answers to that one. I think that companies need to lower the gatekeeping. There are lots of folks that are interested in doing this. The joke that we have inside the community is, you know, I want a beginner who has 10 years of experience on a product that's only been out for three years. Like you're looking for the proverbial unicorn that for these positions and preventing available talent from getting into it. So we need hiring managers to understand that stop trying to get the perfect candidate that has an inherent bias and instead open the aperture to giving someone a chance with knowing that on the job training for everyone is regardless of skill level is a thing and then tying this back that people don't scale tools do. We need to find ways to get the tooling to be more aligned to the resources so that they're not fighting the tooling but they're working with it.

Thanks, Bryce. And I think another piece of this that I'd appreciate your context on is any advice for the new person who wants to start in the cybersecurity field. And I mean new, not just in junior, but maybe a person who wants to come from a different career path and get into cybersecurity. What advice do you have for folks starting out?

First of all, if you're coming from a different discipline, that's a great thing. Cybersecurity's benefit is having a holistic perspective. Groupthink is the bane of our existence. Coming from a completely different discipline, it's gonna give you a fresh set of eyes and a different analytical way of looking at the world. And we need that and we want you. Please come. That's fantastic. To get started, get engaged with the community. I love the B sides conferences because those are community based, they're local. It's a way to physically meet people, look for meetup groups, look for online CTFs, get out, engage, ask questions and the community is gonna is going to help you. There's so many folks who are happy to help on this stuff. Also back on the conferences, there is career counseling that's involved at a lot of them. And a lot of these conferences can be free or even, I mean very cheap, like 20, $30 a ticket. So great opportunity to network and get professional coaching.

Great. Yeah. Bryson, that's one thing I've noticed about you is you are deeply engaged with the community and you do a lot of that kind of thing yourself. And we can find you all over YouTube giving presentations and tutoring and teaching and mentoring and it's very helpful. It's great that you give back to the community that way. And so anybody who's starting out I think should look on YouTube for some of your past presentations. It's a good way to learn about the community.

Going beyond YouTube, Scythe offers free Purple Team workshops periodically. Next one's June 2nd. There there'll be more coming out. We want to expand our free education lineup and beyond just getting lectured. These are hands on labs so everybody's going to get an isolated network environment with multiple hosts on it. We're going to teach you how to build an attack, we're going to teach you how to launch an attack and we're going to teach you how to detect it. And we're looking at building more advanced techniques there so that the education can continue. And it's going to be covering red, blue and of course purple concepts. So there's going to be something for everybody.

A final question Bryson and first of all, thanks for all this. Really really appreciated. But I'm looking for some extremely virtuous paradigm shifting idea and would love to hear your view on what the community can do to really make a dramatic, positive, virtuous difference in the world of cybersecurity. Any ideas there?

I don't know that I can give you we're going to solve world cyber hunger overnight answer but the most common thing that's the people level is security needs to stop being a culture of no create a security service catalog if you run a security organization so that all users have a predefined way to know what to look for, what to expect in terms of engaging you as opposed to where both sides get frustrated when security gets brought in at the end at deployment going well, hold on, this isn't all taken care of and nobody's happy. So don't blame the user. Engage the user. Don't be a culture of no great helpful and virtuous.

Hey thanks Bryce and I really appreciate the time today. This is fantastic.

Thanks for having me. Bob thanks for listening to this OODA Loop production For the latest analysis on cybersecurity technology and global risks, please visit www.ooda loop.com.