The Cybersecurity Vault - Now is the Time to Improve Cybersecurity Metrics

Welcome to the Cybersecurity Vault. I'm your host, Matthew Rosenquist, CISO, industry cybersecurity strategist and advisor. And today we're going to talk about how the evolution in our industry, including the regulations, frameworks, all those fun things, are escalating the needs Good cybersecurity metrics. And I'm going to be talking with Rick Howard. Rick is the Chief Security Officer at N2K and the Chief Analyst and Senior Fellow at the CyberWire. Previously, he was the CSO at Palo Alto Networks and the CSO at TASC and one of the founding organizers of the Cybersecurity Canon. Welcome, Rick. Thank you for talking. It's been a long time since we've had a chat.

It's been too long, Matthew. Thanks for having me on. This is going to be fun.

Absolutely. Let's talk metrics because everybody loves metrics, right?

Right? We're not going to put anybody to sleep with that. Yeah, no, go ahead.

No, no, no. Put it on right before you go to bed. Cures insomnia all the time. So good cybersecurity metrics, well, they've always been valuable. They've always been needed, but they've also really kind of been elusive, right? Good ones out there anyway. How are you seeing the landscape changing? And we talked about regulations, or I mentioned regulations, uh, and we have SEC cases, right? We've got two this year, one with Uber, one with SolarWinds. Um, you know, how is that kind of driving it, and including like a greater demand for budgets, right? Managers are always asking for, for higher, but how are new metrics needed? What are you seeing in the industry?

Well, I think we've been doing it wrong for 30 years. You and I have been capturing metrics wrong our entire career. And it's mostly because I think that we don't really have a strategy that we're trying to pursue. Gathering metrics is a tactic. Okay? It is, here's some things we're going to do to measure our program. But if you don't have some idea about what you're trying to do with that metric, then I don't know, you're just kind of spinning your wheels, right? So, and we're not there yet. The change is slowly starting to come because guys like us have been around for a long time going, you know, this isn't working. We need something better than this. All right. So, I've been thinking about this problem for a long time. It's not just metrics either, right? It's like, I don't know, Matthew, you know, I've been doing this a long time and we've taken what our predecessors have done. Okay. And some of these really smart people. And we've kind of added on to what they've suggested. And most of us haven't really stopped to think if we were going in the right direction in the first place, right? And so, uh, what I did was I got sick and tired of this, so I started to try to figure out what that is. So I wrote a book this year called Cybersecurity First Principles: A Reboot of Strategies and Tactics. And the whole point of the exercise was to figure out what it— the absolute cybersecurity first principle is, something that everybody in the InfoSec community can agree to, that we're all trying to do with our programs, regardless of your vertical, regardless of your size, right? What is that absolute, that atomic element? And I bet you can come up with some ideas, Matthew, about what you think are candidates for that. I don't know, what do you think might be in the running for this?

For, I mean, if we're talking about first principles, I mean, I've always said the first axiom of cybersecurity is cybersecurity isn't relevant until it fails. So, that's where you get attention and then investment and investment spurs innovation and then you get new tools and services to help out and processes and so forth. And Mitch—

I agree. I agree that's true though. I agree that that's true, but go ahead.

But when it comes to principles, I think— I think we have to understand, and you mentioned it, right? The strategy. Metrics without strategy is kind of pointless.

Yeah, pointless.

You run it in all sorts of different ways, chicken with your head cut off. So, you have to have a strategy, which means you have to have clear goals. Goals should be measurable, right? Exactly. So, now this all ties in. If you've got clear security goals that are measurable, now you need to identify the metrics that support those measures that tell you either how bad you're doing, how good you're doing, or where you need to allocate your resources. Right.

I can even— I'd even go even better than that. And I'll show you and I'll tell you why. But this all started back in the '70s and '80s, though. You know, this is when computers were starting to just become popular. We all started to have computers on our desk. What was your first computer, Matthew? Everybody has that story.

What was the first computer I ever programmed on was a Commodore PET. It had 8K of memory, no permanent storage. The first computer I actually owned, everybody was getting the Atari 2600 gaming machine.

Yeah, yeah, yeah.

And I actually bought instead, I bought a, it was a Commodore, no, not a Commodore. Commodore was the upgrade. It was an Atari 600XL, which is just the keyboard with the processor in it. You plugged it into the TV, no permanent storage, but I could program it. 16 kilobytes of memory and I could program, I think it was Atari BASIC is what I learned, the first BASIC, right? But yeah, and then I upgraded to a Commodore 64 a couple of years later. And I mean, the world of difference.

World of difference.

Yeah.

It's, we're changing the show.

My first computer. Don't believe it.

I know.

We need to cut that because that says how old I am right there.

It does, right? So then I won't even tell my first computer story because it's basically—

Oh, come on. What was it? A TRS-80? I mean, there were so many good ones back then.

It was an Apple IIc. Okay.

Oh, I remember those.

Yes. And I splurged. I remember I spent $3,000 on this thing because it had 2 floppy drives.

Wow.

A color card.

You were riding high.

Oh, I was it, man. I thought I was great.

All right.

So back then, all these big giant brainiac computer science nerds. Okay. And people that you've heard of, right? Bell and Lappadula, those kinds of people, right? They were writing all these white papers about what cybersecurity was going to be, right? And they threw out lots of—

Hold on, hold on, hold on. Back then, cybersecurity wasn't even a term. Information security wasn't even a term. This is system security.

System security.

Yes.

Right? Yeah.

Okay. I just want to be clear for our audience here. Cybersecurity is relatively new for us dinosaurs, right?

Yeah. No kidding. We didn't get the first CISO until 20 years after that. So let's just— but anyway, these guys come up with all these interesting ideas and a couple of big ideas kind of stuck in the community, right? And one of them was we're going to just keep all of our systems up to date and patched. Cyber hygiene. Okay. And we all see how hard that has become and how that has not been that fruitful, right? And the other big idea that came out of all that was the CIA triad. Confidentiality, integrity, and availability, right? And I would guess if you ask any InfoSec practitioner what they're doing with their program, they're going to say some function, some piece of the CIA triad.

And I'm a fan of it. I mean, there's other tangents and aspects to it, but I'm a fan of that triad. Okay.

Well, I'm just, and I am too. These are great ideas, but the question is, are they working? And clearly if you just read the headlines, okay, they are not working. All right. The question then becomes is, what is more atomic than those two things, right? And so that's what I tried to come up with in my book. And so long story short, here's what I think it is. This is the ex-Twitter line, okay, of what the absolute cybersecurity first principle is. And then we'll show you how it relates to metrics in a second.

All right.

All right. So here's what it is, okay. Reduce the probability of material impact to our organization due to a cyber event in some shortened timeframe, say 1 to 3 years. Let me break that down for a second. It's in 3 pieces. First, we're going to reduce the probability. We're not saying we're going to stop everything. We're going to reduce the likelihood.

Right. You can't stop everything. You can't eliminate risk. We manage risk. That's our job.

We manage risk.

With certain objectives.

Right. The second part of that is materiality. Why would we spend resources on things that are not material to the business? So, don't do that. And the third thing that's probably not as obvious is we want to have a shortened time horizon, like 1 to 3 years, whatever makes sense to the business because are you going to get hit by—

Protect the decision timely.

Okay.

Yeah. Got it.

Because are you going to get hit by some Chinese hacker 100 years from now? Yeah, probably, you know, 100 years from now. But in the next 3 years, what's the probability that they will be successful? All right, so that is a more tangible number you can calculate, right? So we can argue about whether or not that is the absolute cybersecurity-first principle. I think it is. But when we're talking about metrics, right, the metrics you should— these are the tactics now that we're going to pursue that strategy. The metrics you gather should be measuring how well you are reducing the probability of that attack.

Those three aspects. Yes.

Right.

Okay.

So that's where I think it comes from. So there's lots of things you can do with that, but that's kind of the overall one of the world idea for my view of the world.

So, okay. No, I like this, right? But it goes back to a much, much older theory, right? You can have it fast, you can have it good, or you can have it cheap. And everybody wants all three. And it sounds like you're saying cybersecurity, you should deliver all three. It should be fast, it should be cheap or affordable, and it should be effective. But the whole inside joke of saying pick two, does that apply to now what you're saying? Does that convey that it is an impossibility? Or, and this is the path that I've taken, it's really a matter of finding that optimal balance between what it's going to cost., right? The friction you're going to introduce and the residual risk that you're going to have to eat at the end of the day. Okay, let's balance those. You want to spend very little, your risk is going to be high, right? You want to risk lower and spending lower, your friction is going to be through the roof.

Yeah. And I think what I've learned after 30 years of doing this is, right, security is just a series of trade-offs. And you can have more of this, but you're going to have less of this. We don't have infinite budgets. The company has limited resources. So, yes, you're going to be making trade-offs here. And so, here's where I think we can go with this, though. You can go to your boss, your senior leader, let's say it's the CEO, and say, "Hey, boss, in the next 3 years, I think the likelihood that we're going to be materially impacted is about, let's say, 20%." We can talk about how we calculate that later.

Yeah.

But let's say it's 20%.

I like where you're going though. Yes.

Yeah. So it's 20%. The first question you ask the CEO is, are you okay with that? Okay. Because the CEO has to manage all kinds of risks, not just cyber risk, right?

And if they say yes, document it and you're done.

Yeah. Maintain, maintain, which I love that job. Okay.

Maintain. So, that's you on the golf course every afternoon. That's what that is.

That's exactly right. Okay. And then, but then if he says, oh my God, that's way too high. I need it to be at 10%. Okay. Then we get our heads together and say, here are some follow-on strategies that we can use to reduce that risk. And oh, by the way, all of these things cost money and time. They don't come for free. Right? So, if you're going to implement—

And friction, right? You may be impacting— release of products or soaking up engineers' time or impacting customers, lots of things, right? Okay.

And I think that's one of the things we failed to do. Our peers, Matthew, and I include myself in that hit list, we have failed to convey those kinds of things to senior leadership. I used to be that guy back in the '90s and early 2000s that some new vulnerability would hit the street and I'd run around with my hair on fire saying, oh my God, the world's coming to an end.

It's the end of the world.

Right. And you know, back when one was released a week, whatever, you know, come on, you know, except for a very small handful of organizations, the world didn't end.

Okay. I'm just saying your instruction was put in the floppy drive for your antivirus and run a scan. Okay, sure, sure, sure. All right, exactly.

So now we can make those. Now we can go to the senior leadership and say, listen, boss, it's your call. You're the guy that runs all the risk in the in the company, right? But here's what I think it is and here's what we can do about it and here's what it's going to cost and here's how long it's going to take to get there, right? And is that worth it to you to reduce that probability down to something you just said you needed, right? And so then they get, then they get to make that decision, which they get paid big bucks to make, right?

So, they get to decide that trade-off because it's their butt on the line at the end of the day. So, you're not giving them a problem. You're giving them a dilemma.

Choose what you want to do here. Choose what you want. That trade-off. Is what we're talking about, right? And so to bring it back to the metrics, then whatever metrics you're collecting should be to inform what that probability is. All right. If you're collecting metrics on, I don't know, how many people logged in, you know, during, you know, the holiday weekend, I don't know, how does that impact your probability material? Yeah.

Yeah. It's not directly tied.

Not directly tied. But okay. If you are collecting the number of pieces of software that has a known vulnerability in it that's very high, you know, on the rating scale being impactful. All right, then we can figure that in and we can calculate that in. All right. And that's the kind of thing I'm talking about.

Okay. So let me postulate an idea here based on what you're talking about, right? Metrics, the only value of metrics is to help make a decision. If you've got a metric that isn't contributing to a decision, it's eye candy. It may be fun. It may be pretty. It may be distracting. It's not practically valuable. So, that would be the first supposition. The second thing is a meaningful metric is one that's tied to that important decision that you're talking about, right? What are we going to do? What do we accept? What are our paths, right, or options? For managing risk because we want to hit that optimal zone, you know, out there.

And I would go even deeper than that. I would say every metric should contribute to what our calculation of probability is for that day. And let me give you an example. All right. So if let's say we, our organization, we calculated it was 20% chance of material impact and then Log4j happens. Right. Okay. Right.

So now the boss— out the calculator, we gotta recalculate this.

We gotta calculate, right? So now the boss is saying, what's our probability of impact here, Rick? Okay. So that means we're going back and seeing every place that we are exposed and trying to close that down. And I can guarantee you that our probability would've gone up significantly, okay, if we had lots of Log4j, you know, thrown around unpatched, right? So that's what I'm talking about. It may allows you to understand the significance of the event that you are worried about.

So I want to challenge you on this here. Yeah, because I agree that is absolutely bar none one of the value propositions of metrics, right? It's the probability factor. But I would argue that metrics play such a bigger role than that. It's not just probability, probability. It's also impact. It's also recovery or costs, you know, of recovery. It's also estimating when you're giving those options to say, hey boss, we can do Plan A, Plan B, Plan C. They should be backed by individual metrics, right? What is the value proposition? What is the ROI or ROSI, return on security investment, for these three options? Right? You had mentioned timeliness. So, you know, probability definitely plays into that. I would say also threat intelligence, right? I love when I see metrics around who's attacking me, what resources are they bringing in, what are they likely to target, right? How can I start? And again, that gets into more, all right, what's the best path or recommended path? So I think metrics play into so many more aspects than simply the probability discussion.

Well, I say, I'll push back to you on that, all right? Because I agree that you have to consider cost and damage, right? And impact. And in my statement, I said, and impact. And in my statement, I said, reduce the probability of material impact, right? And so that's an interesting question. What is material to your organization? Sure. Compared to my organization. Right. And it is different for every type of business, you know, depending on the size, the vertical, the culture of the leadership team. All right. And so in order to make that calculation, you have to know what is material for your business. Right. And so, yeah. So when you catch all those stats that you were talking about,, right? You can see if this hits the radar or not, if this hits the threshold, now whether or not you should be worried about it or not.

So, you're using the word materiality and there are specific legal connotations around the word materiality. Are you using that specific—

Yes.

In that context? Well, even beforehand, right?

Yeah, yeah, yeah.

Non-fiber related, right? If you're finance, you know about materiality and you have to report it to your shareholders. Are you using it in that specific definition or are you saying something that's important, something that's simply tied to the goals of those executives that own the risk? Because if an executive, and I've had this, I've had the CEO turn on the news in the morning, see something, panic, right? And come to me and go, "Hey, you know, that, make sure that doesn't happen." And it wouldn't have been, even if it did happen, it wouldn't have been material. But they were in their brain, that was just not acceptable. Okay. All right. You want to spend the money and eat the friction on that? We can go take care of that. It's not material. Will never be material. But to you, it was important. You own this. You're the representative of the share. Okay. We get it done.

When I wrote the book, the SEC had not published their definition of materiality. Yet. Okay. And so, when I wrote it, it was a more generic version of what each individual business thought was material, right? But now— Okay.

So, it's more a generic term. It's not taken from the pages of the SEC doctrine, specific security, although they already have material defined for other things, right? If there was a disaster or something, a hurricane, you had to report materiality impacts in your—

Yeah. But after the SEC released their new rule, okay. I look like a genius, right? It's in my title, right?

You were already a genius. It's just they recognized it. That's all. That's all. That's all. And I'm sure you have the metrics to prove it.

Yeah, yeah, absolutely. I think that you and I and all the InfoSec community are going to be arguing about what is material for the next 30 years. Okay. And so—

No, I will push back on that 100%. There's already definitions. If you go to any big company and you go to the finance people, you got to go track down the finance people to do the quarterly reporting. They already have a definition of what's material, right? And you know, certain percentage of revenues or percentage of this or that operation down this constant, basically constant. Now we'll argue and debate it if it's in the gray area, but this, right, is material. We lost 20% revenue this year because a factory went down. Because of a hurricane, yeah, we're going to have to report that, right? That's going to be part of the quarterly report. So materiality is already there and it translates. I think CISOs going, oh no, it'll be so vague. No, it won't. No, it won't. It'll be clear.

You might be right about that. All right. But I will say that for our history, us two old timers here, right? Our community has not done a good job.

Speak for yourself. I'm still a young pup. All right.

We have not done a good job of engaging with the business in that way. Right? Yeah. We stayed in our silos—

yeah.

—and had that discussion. And we were talking about this last week; we're talking about SolarWinds and their CISO Tim Brown, right? The SEC accused him of fraud. Yep. Of fraud because he admitted that they had some mistakes in their infrastructure. No!

No, no, no, no, no, no, no, no, no. So, I've been arguing this and debating this. I've actually had some great interviews with people. Okay. Let's hear it. They say fraud. Now, the crux of what they're saying is now every company, every quarter has to produce quarterly reports, right? And you have to state if bad things are happening. And then if you have an incident, you have to report an 8-K form and also, right? So, what they're saying is he signed off, they filed these official SEC reports that go to the shareholders. And the intention of these reports that go to the shareholders is to inform them so that they can make good investment decisions. Otherwise, it becomes insider information, right? If only your executives know a hurricane took down the factory and yeah, they can start selling things, but the shareholders need to know. Same thing for Cyber. So their claims in the 86-page complaint that can be downloaded. Unbelievable. Is that they misled investors when they filed those official forms. They were fraudulent in what they put in there. Well, let's be clear. Doesn't matter what security you have.

No, no. Let's be clear about who they are. Okay. Because they charged Tim Brown, the CISO.

They named Tim Brown as well as SolarWinds. So it's both of us there. Yeah. And they can also name the CTO and CFO and everybody else if they choose. But they didn't do that. They didn't. I agree. I think there's more culpability there, but that's a different argument. All right.

But wait a second. Here's the question. Yes. Here's the question. What CISO do you know? All right. In your vast experience has the authority to sign off on those forms for public release? How many?

Every CISO has the responsibility. Now they don't do it in a vacuum, right? And that's why I completely agree with you that there is more culpability here. It should not just be on the shoulders of the CISO, but the CISO, that is his prime responsibility is representing the security aspect, just like it is the prime responsibility of the CFO O to represent the financials. Well, let's be clear. So if the financials are wrong—

The CISO is not an officer of the company in most cases. Okay. 99% information security officer. It is a title. He is not protected by any of the insurance that the CEO gives the CFO.

Well, no, some are. I am. I made sure I didn't owe any insurance. Are you kidding me?

That is not the case in most places, right?

No, I agree. I agree.

But my only point was, is that most of us have not gone to the lawyers with our risk calculations and said, here's what we think. That's going to change. It's changing now. So, we're all going to learn how to do that calculation. And we're going to understand what materiality is down deep in our soul. All right. So, that's right.

That's right. And if you feel, if you don't feel comfortable with what's on that form that you're attesting to, don't sign it. Don't do it. But that's an ethical kind of thing as well. And I think, you know, for 15, almost 20 years now, right? We've been saying, hey, the security director, chief of security, whatever, really needs to be elevated to that CISO status. They need a seat at the big table, guys. We've been yelling and screaming to this and guess what? We succeeded. You now have chief information security officers everywhere. They're at the big table. They're talking to the boards. But then people—

wait, wait, wait, wait, wait, wait, wait. That is not true.

I don't want the accountability. I don't want the SEC to come after me. Well, careful what you ask for. That's what comes with the chair at the big table. Put your big pants on. Realize you're now accountable. Realize that you're signing legal documents going to the shareholders. And you're that expert in that field. So guess what?

My case is that we are not there. We, the CISO is a title only. It is not, you're not an officer of the company, nor are you a director of the company. And by the way, you don't get compensated that way either, right? So for me to sign my name on that line saying this is what it is, that needs to be an entirely different situation. Matthew, I think that's what we're going to get, what you said. Yeah, we're going to get what we asked for. We may not like it. We like both of us.

Like most of us like being barefoot.

Very true.

It's like you want me to ask for something and I'm thinking, oh, here you go. Sure. Yeah, sure. There's a lesson there.

Oh man. Oh man.

Okay. Okay. Okay. So, um, metrics, usually important, going to be more important, especially with these regulations coming out. I wanted to touch base because I, I, I think this is, we are in a phase of transformation. I can't say a moment because you're absolutely right. Right. This has been going on for a long time. We all know we needed better metrics. We've been trying to push this wagon up the hill. And not get crushed by it, slow progress. But I think we are in a phase where we've got other factors that are driving and emphasizing change and evolution in better metrics. You know, and you mentioned it, you got to have the strategy and the goals before you can even figure out what metrics are going to be meaningful. Right. And I see some changes as well in some of the frameworks out there, like the NIST Cybersecurity Framework version. They just upgraded it from version 1.1 to version 2. And they added a whole governance, you know, ring around what they already had.

So, you know, how do you— Let's talk about that too, because you could consider these various frameworks like the NIST framework or any of the ISO standards or any of the standards that you think about, they should be candidates for our absolute cybersecurity first principles. Right. But when I went—

they should be involved, definitely.

I mean, when I went through that in my mind, what I realized is that most of those things are really just maturity models or checklists. They're not really a strategy, right? Because— agreed— there's a bunch of things that you could do, right? But how does that manifest into an overall goal? Your goal can't be implement everything to the max of the NIST Cybersecurity Framework.

Infinite resources, right? We have infinite resources, infinite possibilities then. Yes. Yes. Just get with that company that has infinite resources.

To your point, though, I think that we might be able to glean some meaningful metrics by just looking at those frameworks, right? Just by, oh, for this particular set of things, this is what they were looking at. What metrics can I come up with to measure how good I am? At those particular things, right? So I do think they have value. I'm not trying to throw them out as if they're bad. No, no, no.

They have value. They're not the end-all be-all solution. Right. But, you know, and one of the things that I like between 1.1 versus 2.0 is they had this whole governance ring, right? But in my mind, if we kind of overlay the metrics, what is that really saying? That's saying, hey, you have to have oversight, you have to understand where you're at, where you're going, and metrics are a key part of that. So I'm seeing some of these frameworks realize, even if it's subliminally and they're not straight out coming out and saying it, that yeah, metrics that support good decisions are really important. Yeah. Right? If you don't have those, you know, our industry is not you flip a switch and you have security, it's all done, right? It's this gradual scale that you have to adjust and find that optimal and stay in that zone as the world changes. But, you know, the CSF moving to 2.0, I think it's emphasizing as a requirement, hey, you need to have better metrics. If you're going to do governance, you can't do governance without metrics.

Well, and it goes back— yeah, and it goes back to what you were saying before, right? What I was saying before too is that what's the strategy? What are you trying to accomplish, right? And you can use the frameworks. Glean metrics out of them, but always, always to drive success as you pursue those strategies. Right. And knowing, by the way, that you're never going to get to the end of those strategies. You know, these are aspirational things, right? You know, your strategy might be like for me at CyberWire, our strategy is resilience because we can't afford to do all the other ones. We're just a little startup.

Oh, that's a separate discussion if you're investing too much in resilience, then you're not doing prevention and prediction and your cost efficiency goes down. Responding, you know, a pound of cure versus an ounce of prevention. Which one do you want to have? You want to have some of both.

If I have enough resources, that's an interesting question, right? However, I do not. You know, our startup has 2 IT guys. The security team is me and I'm not allowed to touch anything because I'll break it.

Right. Good. That's the way it should actually be.

Right. So we can't develop an intrusion kill chain prevention strategy or a robust zero trust strategy. Right. What we can do as a startup is deploy a pretty robust resilience strategy, meaning that we're going to survive it. Okay. We're not going to try to prevent it per se, although we do some things. Right. But we're just going to make sure that the company can survive whatever bad thing might happen in the next couple of years.

See, I can't get behind that. I have to say you have to have a balance of both, right? When I look at that, that continuous capability where you've got prediction, prevention, detection, and response and prediction and prevention is, is, you know, it's avoiding loss and you've got detection and response is minimizing loss. You have to have a balance. You can't say I'm just going to have backups, but I'm not going to have strong passwords. I'm just going to have, you know, hot swaps available, but I'm not going to do front-end user training on good practices and avoiding phishing, right? You have to have a blend in my mind.

I'm not saying that we have open passwords and anybody can just get into the network. I'm not saying that, all right? I'm just saying that if I was— Well, if you did, I'd try.

I'd go out to your site and try password, password, admin password, sure.

And I know after we publish this, I'm going to get a million people trying to do that. I'm just saying that if I was going to implement every strategy, every tactic in a zero trust strategy, right? That's a lot, right? That's too much. That's a huge order.

Huge, huge order. I'm with you on that. Yeah. I'm with you. Okay. So ugly, ugly question then, because I want to get back to this strategy and metrics. Yeah. And intentionally, I know I'm asking an ugly question. Go for it. What comes first? Do you build a strategy and then the metrics, or do you look at metrics to help build your strategy?

Well, I have been guilty of both of those things. All right. And, you know, in my early days, we would just collect a bunch of data not knowing what we needed and just looking at it and going, "Hmm, I don't know what that means, but keep collecting it. Something will emerge." That sort of works. All right.

The light will come down. Divine intervention. And I will see what I need to get done. Yeah.

It'll, it'll just be so clear, you know, but I am gonna recommend strategy first. Everybody on the team has to understand what the strategy is and then the team can decide what metrics can be collected. To pursue it. That's what I think. What do you think?

I like it. I actually think it's an interaction. I think it's a continual process. You go to the executives and go, what bad things do you not want? What bad, you know, what level are you kind of okay with? Great. I'm going to go back and look at metrics and see how close or how far we are away. Then I'm going to come back and report on that. Then I'm going to look at metrics because I'm the security, I'm your security guy. I'm gonna go find some plausible paths, right? And option 1, option 2, option 3, I'm gonna have my preference, but I'm gonna go now pull the data that supports exactly what you were saying. You know, what is the likelihood? What is the impact? What's the, the threat? All that I'm now going to present to you based on the data, right? Um, informed risk-taking with actual data. There's metrics behind it. And once a path is chosen, now I have to go and I have to potentially even create new metrics to know how successful that path, you know, we are as we go down that path. So I think it's got to be a back and forth.

And I do think those metrics might not come from traditional sources like you and I are used to, like, you know, IP addresses that are hitting us or ports that are being used or blah, blah, blah. As you go and discuss material impact with the business leaders in the organization. And it doesn't have to be a commercial business. It could be a government outfit. It can be an academic institution. It doesn't matter, right? But as you discuss what is material to them, you know, when you ask them that question is, what, what can happen to your thing, okay, that will cause it to fail? They're going to have all these ideas about it that has nothing to do with, you know, firewalls and IP addresses and zero trust and all that stuff, right? And so, and those metrics might be more useful than some of the things we're used to collecting.

No, I agree. Okay. Okay. So let me tap into your insight because you've been around here for so, so many years.

I know I accused you of that, but now it's time to come back my way. No, no, we're not talking about me.

We're talking about you, right? The venerable expert here. Give me a couple examples of metrics you like. To be presented to boards? And again, boards aren't necessarily technical. They don't necessarily know how to spell cybersecurity, right? They're business people. They're representing the shareholders. What kind of metrics, security metrics, do you like or would you recommend to the board? Give me one or two examples.

Well, I'll tell you what I don't like first, right? I would never— Wow.

Way to avoid that. Now that's—

Expertise.

That's how you tell a veteran in this industry.

That's a media expert right there. Okay. I will answer that question, but I gotta get around to it, right? I'm never going into a— Oh, look at the time. Yeah. Oh, we'll have to do that next time, Matthew. I'm never going into a board meeting with a list of malicious code or vulnerabilities or exploits. They could care less about that stuff, right? Okay. The only people that like that stuff is the SOC analysts, right? And nerd CISOs, which I include myself in that, right? What I think we want to do is go in there and talk to them about how our strategies are working. Okay. And you can have multiple strategies. You can follow my first principle and there might be 3 or 4 follow-ons that we're going to do. And you can give them actual numbers about how that has reduced the probability material impact to the organization. Now, I say that, but this is new to them. Nobody in the cybersecurity industry has done this before. So you're not going into the big board meeting on the first day and trying to lay that out there. You're going to get laughed out of the room. You're going to have to do individual meetings. Maybe multiple individual meetings with all the board members and all the senior staff that says, here's what we're, here's what I'm trying to do with my InfoSec program and here's what we think it is. And there's going to be pushback. That is completely fine. Right. But as soon as, but when you get them all on your side, then you can walk into a big board meeting and say, okay, here's what we think it is.

Yeah. And identify the advocates on that board because there's always one person that's pro-security. You got to get them buying. Yeah.

First, get that guy, get that board member first.

Yeah, navigating the board is the challenge here.

It's one of the skills that we should have as CISOs, right?

And you know, the other thing that I see is in our industry and when I look at the CISOs and I advise a lot of CISOs and work with them, we have a huge community that has come up from the technical side. Yeah. And they are very adept and comfortable at looking at all those scans and looking at all the very technical operational security metrics. Right. How many hits did we get on the firewall? How many this? How many that? Right. But that's a different language than what the board speaks. Yeah. The board is talking about profitability, market share, average selling price, margins, right? Liability and things of that sort. Two different languages. And if you're a CISO that has grown up on the technical side, you have to learn a new language if you want to resonate In that board meeting, you have to speak their language. Don't expect them to understand yours.

Yeah, let me give you a metric that you bring in.

You have to translate those right into their language.

Let me give you an example of that, right? So I've been asked this question by numerous board members over the past. He says, what would you do if I gave you $1 million or pick your number, right? The technical CISO, would say, "Oh, I'm going to buy 5 more firewalls." Okay. Right? Right.

I'm going to buy this appliance, that tool, this service.

Yeah, yeah. I'm going to get the shiny new object. Right? Okay. But a business CISO is going to say, "With $1 million, I can reduce the probability of material impact by 5 points." Yep.

I'm going to lower our risk. That's what you want to be saying. Yeah. We're, you know, instead of my estimate that we're, you know, going to, you know, have an incident, a moderate incident in the next 6 months to 12 months, it now pushes it out to 24 to 36 months.

Exactly right. Okay. So, that's— God. Yeah. Yep.

How many quarters I don't have to report that to the stockholders? I like that. I like that.

And the other one is that strategy we say we're going to implement and we're going to improve it over the next 5 years. I can get that done in 1 year. Okay. So, you know— I love it.

Yeah. I love it. You know, the other one that I've used is, hey, with this extra budget, I can help accelerate the release of this new product. Right now, it's being held up in testing and SaaS and DaaS, all these other things. With this investment, I can reduce that friction and we get products out the door. Oh, I like that too, right? Oh, right. It's a business benefit.

Let's go one more layer deep on that, right? Because if I went to the boss and said, you're, probability of material impact is 20% and you're okay with that. You're okay with that risk. But then you come to me and say, here's $1 million, what you can do with that? You can say, boss, put that somewhere else. Your risk, you tolerate that risk.

I'm sorry, I don't understand what you just said. That does not make sense to me. I'm sorry. Somebody is offering you a greater budget and you're saying, oh no, good citizen. Go ahead and sit that over in the sales department.

Yeah, that's what I'm saying. Exactly. You're trying to make the business better, not your infosec program better, especially if the CEO's already said, I'll accept that risk.

I'm okay with that as long as you do that knowing, just knowing that if the sales department were given the same situation, they would not give the money to you. I know. Just knowing that you're still doing the right thing. You're doing the right thing for the company.

You have to do what's good for you.

Just know, just know deep in your heart that that's a one-way street. There will not be reciprocity there. Yeah. Okay. You're not even gonna get a Christmas card from them. No.

They're gonna hear about that and laugh at the next SEC meeting.

Yeah. Like, oh, and that's why security's not a profit center. Okay. Okay. Last question here, because I know we're going long on time. For the CISOs that are listening here, right? Some may be thinking, my security metrics, I think they're pretty good. Some may be thinking, I know they're bad. Some may be thinking, I really don't know if they're good or bad, right? What would you recommend them in how to evaluate whether their metrics are meaningful or not? What is that litmus test that should be the first step?

Well, I'm a huge— over the last 10 years, I've transformed myself from what I used to be in terms of thinking about cybersecurity to what I think it is now. And you and I, I think, agree on this. It's everything's about risk. So if you have a hole in your education about what risk is, it's time to get smarter about that, right? And there are a number of books that you could read to get up to speed on that. There's the Jack Jones and Jack Troyne book about the FAIR methodology. A great primer on risk. There's the book from Doug Hubbard and Richard Syerson, How to Measure Everything in Cybersecurity. That's another great primer on cybersecurity risk, and there's a bunch of others. So if you have not—

Andrew Jockwiesc, what's his book?

It's about metrics specifically, right? Yes. It's got metrics in the title, right?

He's one of the luminaries in our industry. Yeah, yeah, yeah. So yeah, if you're going to read a book on security metrics, Definitely, I would put that on the list as well.

All 3 of those books are in the Cybersecurity Canon Project that I belong to, right? They means— and the Canon Project is a bunch of security leaders like us. They read the books, they try to put them into 1 of 3 categories, right? The categories are Hall of Fame, meaning all of us should read this book. Should read it. Yeah, it's a must-read. The second category is a niche book, meaning it's a good book, but maybe not everybody has to read it.

Like, it might be— if you have a specific problem.

Yes. Yeah, exactly right. And then the third category is do not read. All right. Because there's lots of—

it will mislead you. It is a waste of your time. Yeah. So if you're—

you've decided to read a book this year, maybe you don't want to read a bad one. All right. So we do the work so you don't have to. All right. And so we publish those book reviews. On the official Ohio State University sponsored webpage. So if you're interested in a book, go look up the review to see if it makes sense to you. And all three of those books that you just, that we just talked about, Jackwith's book, uh, Jones and Freund and Syerson and Hubbard, they're all Hall of Fame books. Okay. Oh yeah. Uh, and so, um, yeah, start your education if you are new to this idea.

And you recently launched a new book as well, Cybersecurity First Principles. Where can listeners get a copy?

You can get your copy at Amazon. Okay. You can get a hardback if you are one of the old-timers and need paper.

That like paper on your hands, a little bit of ink. Yeah. I'm, I'm one of those. Okay. Yes.

You can get a digital version, a Kindle version that I prefer when I'm reading these days, because it allows me to take notes. Uh, and then if you're an audio person, uh, there is an audio version, uh, from, um, Audible. All right. So all three ways that you can consume Awesome.

And I will put a link for that actually in the description when we push this out. So, everybody can click on the link, get your copy, whether you want to, you know, page it, eat the pages, read it, listen to it, whatever you want to do. We'll get that out there. Rick, thank you so much for coming in. So much fun, man. It's always great talking with you and bantering back and forth. I love talking with, you know, older people who have been in this industry for so many decades. It's a wealth of knowledge pulling from your head. Thank you very, very much.

Let's not make it so much time between our next meeting, Matthew Rosenquist. Too much fun.

Too much fun. Fair enough. Fair enough. And thank you all for watching. Be sure to subscribe and catch all the Cybersecurity Vault episodes where we chat with industry leaders like Rick to dive into the most relevant and interesting cybersecurity challenges, perspectives, and best practices. See you next time.