Obsidian Security | Defending Critical SaaS

Hi everyone. Welcome to Grey Matter, the podcast from Greylock where we share stories from company builders and business leaders. I'm Heather Mack, head of editorial at Greylock. Today we're rebroadcasting our episode featuring Obsidian Security CEO and co founder Glenn Chisholm. The company, which protects critical SaaS applications, just raised $90 million in Series C funding. Greylock has been an investor since the firm led the company's series A in 2017. Greylock general partner Sarah Goa spoke with Glenn and New York Times cybersecurity reporter Nicole Perlroth in early 2021, shortly after the SolarWinds cyber attack.

Hi everyone, I'm Sarah Goa, general partner at Greylock. Welcome to Grey Matter and I'm super excited to have with me today Nicole Perlroth and Glenn Chisholm. Nicole has been an amazing reporter with the New York Times and Glenn is is the co founder and chief product officer at Obsidian Security. Welcome Nicole and Glenn.

Thanks for having us.

Thanks for having me.

Nicole, perhaps you could start just talking a little bit about your role at the New York Times and the book you just released.

So I started at the New York Times in 2011 and back then it's almost quaint to think about what I had started writing about, which was Anonymous and their DDoS attacks on everyone. And you know, DDoS attacks, you just don't hear about them too much anymore unless they are huge. But basically, very quickly we started seeing a pickup in these nation state attacks. I had always been told that there were only two types of companies left in the United States. Companies that had been hacked and companies that didn't know they'd been hacked yet. And I set out to get to the bottom of it. And lo and behold, the Times itself was hacked by Chinese hackers. And I was able to embed with our security team and Mandiant, which was still sort of a fledgling startup at that point, and the FBI and unwind that attack. And a couple years later, you know, we got access to some of the Snowden documents. And I write about this in the book. We were told that we needed to work out of a windowless room. That was the one precondition. And if you've ever walked by the New York Times headquarters in New York, you know that it is a building that is just full of front to back windows. There's actually every conference room in the building is floor to ceiling windows. And so the only space for us was either to work out of a bathroom or to work out of Arthur Sulzberger's storage closet. And so for months we worked out of the storage closet, left our devices at home, walked in and poured over these top secret documents. And one of the things that kept coming up was clearly the NSA had a backdoor into every piece of commercial technology on the market. There are many, many references to that. But they also had these littered references to our commercial partners, our security partners, our malware developers in the private sector. And at that point I'd always heard that there was a gray market for hackers who find zero days and the tools to exploit them for governments. But clearly what I was looking at was some sort of proof of this. And I spent the next seven years essentially trying to get to the bottom of this market. And that is what the book is about. I tried to write a really character driven account of the market and, and I picked people who symbolized one slice of the market. You know, the people who sell these things, the people who set up these programs within government, the arms brokers, and also just the ramifications for these tools and stockpiling vulnerabilities for the United States as some of the markets have drifted abroad. And these tools are no longer just used for classic espionage, they're used for surveillance. They're turned on their own people and increasingly they're used for attacks on critical infrastructure like the ones we're unwinding right now and seeing in a small town in Florida.

I am excited to talk more about it, but I'd say seven years later, it's terrifying to discover there's no bottom right. It's amazing, amazing reporting and an amazing book. But that's part of my conclusion from reading it so far. Glenn, you've been a practitioner yourself defending massive enterprise networks and critical infrastructure. You've been CTO at silance from early to scale scale. Tell us a little bit about Obsidian Security and why you started the company.

Several of us came together and we all had different backgrounds. Obviously when Nicole was busy writing about Anonymous, I was busy trying to deal with it and defend against it. And when Nicole was experiencing, you know, nation state activities against her, her company, we were defending against those nation state activities against us. And I think it's, you know, it's one of those things where you get shaped by these experiences and you get shaped by having to sort of protect against these various problems. And, you know, sort of came to the conclusion was as we talked to Marcel as a group and how we thought about problems was, you know, we'd seen that for a long time, you know, these attackers, excluding Anonymous, whose motivations were a little bit, you know, more obscure. But the attackers generally, you know, they went for the data and you know, that data used to sit on the networks and servers and you know, we protected that and then that data moved to the endpoints and so we protect them. And then that data moved into SaaS applications and we didn't protect them very well. We sort of, we authenticated against them, but we didn't really protect them.

And Obsidian's focus on securing SaaS, traditionally people worried about who logged into the SaaS applications and they might use something like an Okta, a Microsoft or a Google to make sure that everyone comes through one point. The problem is you have to worry about everything that happens after that. What happens if a credential is lost or stolen and someone steals it and gets into SaaS applications? What happens if malware runs and someone uses that to use a computer to steal information out of the SaaS applications? Obsidian's focused on making sure that only the right people are using your SaaS applications. No one breaches those applications. Making sure that your data is protected and secure and you know where it is and who's used it, making sure that the applications are configured and managed appropriately and then most importantly, allow you to detect and respond to anything that might happen in your SaaS environments. Securing SaaS is important because it is this intermix of the connections between all the applications and third party vendors that, that create the risk, not just who logs in and when they log in.

I'm sure both of you must have this terrible sense of I told you so in terms of starting a company that was really about protecting SaaS applications like Office 365, wherever your files and your email might sit today and ringing the alarm bell on the implications of this sort of giant economy that nation state attackers had had start and fed. And then right before you released this book, Nicole, you're like, well, Sunburst, Solar Winds. So let's get into it. Perhaps Glenn, you could start by just explaining a little bit what happened with Sunburst and Solar Winds.

You know, what it's fair to say is that we saw a nation state attacker that the US sort of intelligence agencies attribute to Russia break into a company called SolarWinds. And SolarWinds is a really interesting company in the fact that it's very ubiquitous. It wasn't one of those brands that ever everyone knew and loved, but it was a brand that everyone in it knew and used. And that software was everywhere. It was installed in a large number of places and the Attacker broke in there. And their intention was obviously use that as a way to get elsewhere. And you know, what we've subsequently learned is that the attacker was inside their network, inside their source code. They deployed modifications to that source code that then went into their production software that then gave the attacker access to as many as 18,000 potential victims. Now, as we know, the attackers resources are limited. Everyone has a budget, even an attacker. And so There wasn't like 18,000 victims, but there were 18,000 potential victims. And what the attacker did was as they gained access, they triaged them. They understood the environment. And you can see that because some of the environments they didn't want, they dropped additional malware. They looked around quickly than they left. And the environments they did want, they expanded their attack. And you know, what we saw in places like Treasury DHS, SolarWinds, was they moved themselves into the mail environment and they started monitoring the email, the communications, the files, and accessing that material because that's where invalid material is. And particularly in places like treasury dhs, you know, policy, communication strategy. But what makes this attack interesting is that wasn't the only supply chain entry point. They attacked, they attacked FireEye, they attacked Microsoft, they attacked a number of organizations that are publicly admitted to being victims. They attacked people like mimecast and this, you know, many of these were to give themselves second stage entry points, but it also gave them other pathways into these mail environments. So you see mimecast, which directly connects to someone's mail environment being used to suck out the mail. And so the attacker was very widespread in their campaign. They were very deliberative and they really went after the supply chain to allow themselves to amplify their own resources. So it's really interesting in the way they managed to amplify their attack through the source.

Glenn, you're reacting with a lot of Zen for somebody who's been doing some serious incident response for two months. But appreciate the overview of the situation, Nicole. You know, you've just been covering breaches for the last decade. What's the significance of this one? Why is this any different? Why does it matter?

Well, I just want to echo that. Glenn's accent really makes this sound a lot more Zen than.

Than it is.

So. You know, my book ended right before we found out about Solar Winds and Sunburst. But essentially the book is really the prelude to this attack. I mean, one of the simple solutions I propose at the end of the book is something like a bill of materials, just understanding what touches your network, what software is in your network. And as I've been calling up the victims at these agencies, a lot of them didn't even know. They'd never heard of SolarWinds. They didn't know it was in their network. I mean, obviously IT administrators did, but they didn't know. And they didn't know that a lot of IT gets built abroad. Not to say that Eastern Europe is somehow fundamentally more dangerous, but most of the customers had no idea that their build servers were also in Belarus, Europe's last dictatorship. And this is a problem I've been highlighting for a long time. And you think back to Heartbleed, there was a vulnerability they discovered in an open source encryption protocol and it was used by the FBI, it was used by Amazon, you name it, they used open ssl. And it had this giant gaping vulnerability in it. And when I did some digging, no one was looking over this except for two guys named Steve who were barely making enough in donations to pay their electricity bills. And the idea of open source software had been, you know, with enough eyeballs, these protocols will be secure. But in this case, there weren't any eyeballs on this code. And you know, we had that happen several years ago, but essentially that's what's happening right now with SolarWinds. You know, and when you dig into SolarWinds, we're learning that they did not have good security in place. They were warned internally that if they didn't make a bigger commitment to security, the results could be catastrophic. And now we're watching that catastrophe. So, you know, this is a big wake up call. It's a big wake up call to take stock of what's in our networks. It's a big wake up call to understand how seriously our vendors take security. And, you know, we're nowhere near that. But I think this attack is so pervasive and it's so bad that we have no choice but to pause and reflect on all of the things that we've been adding in terms of complexity of software to our networks.

Glenn, you know, the Pentagon, the State Department, treasury, dhs, like these, some of these organizations that you name major technology companies. You know, in security, we talk a lot about the haves and the have nots in terms of defense resources. And these are the groups that are the haves, right? They're supposed to be smart about defense. So part of my language but like how did they get so totally owned, right, with dwell time without detection being definitely more than six months, possibly even longer. Right. They've ignored the warning call that Nicole just described. But how could it have happened?

It's a few things. Our attention gets fairly easily diverted. And there was this sort of narrative that was spreading around that these nation state attacks were diminishing and there was a reduction in them. And you know, that narrative started to spread. And I think people become complacent very, very quickly in security because it is a difficult thing to think about through the attack scenarios. As one sort of contemplates these, I think. So people start looking at where the ball was, not where the ball's going. And I'm not a particularly sporty person, but following, you know, going to, scanning where the puck is. It was important, particularly in security. And it is very obvious that we weren't looking there. And so, you know, what we found is like a lot of these environments, the attacker was inside their365 environments for an extended period of time. What we did see though is we saw sophisticated organizations did fend off the attack with appropriate defenses. And then what we also saw was a company like FireEye that had sophisticated security team were the ones to detect it, they were the ones who identify it and then more importantly than any of those things, they were the ones to responsibly disclose it and do it in a way that was the absolute best way to do it. And so, you know, what is critically important here is the sharing and spreading information. I think Nicole did touch upon that in a large sense. And so I suppose the short version is we became complacent on nation state attacks. We became very fixated on these Chinese industrial espionage type attacks and the government moved the their eye away from where they should have been in the cyberspace and moved it towards other areas and in doing so allowed us to suffer badly.

I have certainly heard from many practitioners and leaders of significant security teams over the last five years basically say we solved it, right? Like, you know, basic truce, Obama brokered truce. And clearly there's more than one actor out there, right? And that complacency seems to have come back and bitten us all. Is it unavoidable?

Nation state espionage is a constant norm. And you know, I suppose the, the absence of information should never have been taken as confirmation of the problem going away. It just should have been taken as the fact that we weren't looking hard enough. And you know, considering the criticality and importance of this, the idea that a foreign nation state can sit in DHS and the treasury, particularly when that nation state has been repeatedly accused of meddling in US elections and while that particular sort of counterintelligence process was going on. The nation state was inside. Organizations like that is deeply concerning. And it's a failure from corporations all the way through to government. Corporations need to be responsible for their own protection, their own infrastructure. And as I say, as they evolve their infrastructure, as they evolve their capabilities, they have to evolve their security capabilities. They can't just be worried about what they're worried about last year.

Nicole, I want to go back to something both that you said and that is in the book, which is really espionage, including modern digital espionage is kind of accepted as a fact of life. But something very different is happening today because the dragnet is so much bigger, right. It's not, oh, we might have some very scary visibility as an attacker into coming policy changes. It is all the communications and digital behaviors of all people all the time. Right. And now you have the risk and the reality of kinetic attacks. Right. What does it mean to take cybersecurity more seriously given that pretty fundamental change?

You know, I was thinking this morning, someone said it must be so frustrating to watch these attacks play out when you know that they were inevitable. And I was thinking about how in Eskimo language, you know, they say they have like 100 different words for snow. I feel like there should be a hundred different words for frustrating in the security industry. You know, when you see these attacks that you know were going to happen that were inevitable, but we didn't do anything to change our patterns of behavior, it's incredibly frustrating. And frustrating doesn't even get to the half of it. But I think in general, 20 years ago, once we started rolling espionage into digital technology, which was inevitable, it was all fair play. Because if Russia was using one software and if we found a vulnerability in Russian software, we would stockpile it and use it to spy on Russians, no problem. China was using Huawei. You know, Syria, Sudan, North Korea, they were all using Huawei and we were using our own routers and switches. But because of globalization, we've all migrated to the same systems. So when you find a vulnerability in iOS software that doesn't just enable you to spy on terrorists or on Russian officials at the Ukraine embassy, it creates a vulnerability for Americans too. And I think our eye was really on let's spy on as many systems as we possibly can and we will turn over the low hanging fruit to companies to patch and will occasionally raise a red flag and say, yeah, you need to go patch this. Or this is a major vulnerability you need to work on and turn on two factor authentication. But as we've seen in each Passing attack. We are very good at offense. We are here in the United States, the most sophisticated cyber superpower on the globe. You know, we might be tied with the UK or Israel, but probably chances are we're still the biggest player in this space. But we are also the most targeted. Most attackers, nation state attackers and cybercriminals see systems of interest to profit on to steal intellectual property from in the United States. And so we are one of the most frequent targets that, and we are one of the most vulnerable because we are the most digitally connected. And, you know, I found this McKinsey report. We're hooking up the Internet of things at a rate of 127new devices per second. I mean, that's stunning. And nowhere along the way did we stop to say, should we be hooking up 127 pacemakers and water treatment facility controls to the Internet? And I think right now the answer is obviously no. We should have stopped a long time ago and made it impossible for a remote person to come in remote in and up the level of lie in our drinking water. So, you know, and it took an attack. And fortunately, these attacks that we're witnessing right now are, in a sense, close calls. Solar winds, you know, as bad as it is. We assume it's espionage. They were after emails. They weren't looking to pull off a notpetcha ransomware destructive wiper attack, but they could still use that attack, you know, that access for a more destructive wiper destructive attack. The water treatment facility just so happened a guy was sitting at his computer watching his cursor move around on his screen and was able to catch it. And they say they had sensors and that it would have taken another 24 to 36 hours for it to reach the drinking supply. But do we really want to wait until someone's not at their computer or they didn't catch this? I don't think so. Or do we want to wait for Russia to flip the switch and make a few subtle changes to their code to wipe out our federal IT networks? No. So I think right now is the perfect opportunity with a new administration coming in and the fact they've inherited these IT networks that they just can't trust, that we really have no way forward but to take stock of our own systems and vulnerabilities.

Yeah, I guess here. You know, Nicole, you and I may not fully agree, and part of it for me is like, fatalism, I suppose. And part of it is actually, you know, I'm a technology investor, right? Like, I work with young Companies that are just shipping all the time as fast as they possibly can. And I feel like a part of the, the history of technology's impact and really its velocity and triumph of low cost, this fragmented supply chain of software speed and convenience and features over security. I don't know that that is going to change a lot. Right. I think we're going to keep hooking things up to the Internet. And as a extraordinarily paranoid person myself, I'm still pretty online, right? Like I monitor my own network and I'm like cool 36 devices and I like to be able to check in on my baby when I'm not at home. I realize that's an exposure surface for other people, but I still want the Internet. And so maybe I'll also ask Glenn to weigh in here, which is like if we assume we are going to continue with some similar pace of adoption of this just connectedness and technology adoption and increase in data on the Internet, like what can we improve about security and defense if it's not cutting back technology or slowing down technology adoption?

I think that one of the nice things about some of the younger companies, the way they're doing things now is the speed, the inclination is to push secure code. And I think that we've started making changes that weren't around. When I went through school, there was no such thing as defensive programming. It just didn't exist. It wasn't a concept and I'm aging myself. But, but the reality is, is that it starts with the creation and if the creation is secure, then that helps the model. You can still end up with vulnerabilities and you still end up with one. You still, you will still end up with vulnerabilities. But you know, I'm generally far more comfortable with an agile, fast moving, rapidly changing product than I am with one that static stayed and isn't moving. That is because the overall momentum can often help with security and hopefully the team is more agile and adaptive and you get a better quality output. The haves and have nots problem is also pervasive. Realistically speaking, we have a water plant in the state. What is the chances that to have. It's zero. They don't have high quality security people because they haven't thought about it, they haven't moved in that direction. So the creation of more humans that can do the work, it's the better training of the ones that build the products. It's the movement of technology. You know, I'm fatalistic as well in the sense that I accept the Fact that it's going to keep going and if it's going to keep going, then what I prefer to do is to like be looking where it's going to go and defend that rather than do anything else. The one that worries me, as I said, is where you have stayed slow moving and the problem with that is the best example of stayed in slow moving is critical infrastructure. And so until they become more agile and adaptive, we won't solve the problem, but we're not going to probably be able to solve it by removing them from the Internet. We're not going to solve it doing the same thing we're doing now. So I think agility and speed and also development of capabilities is the only option.

Yeah, Nicole, you brought up not Petya and I was thinking just the words critical infrastructure mean a different thing to me in 2021 when we're talking about security than they did five years ago. Glenn, you're working with many health care organizations today. If your EHR is compromised, like health care is critical infrastructure and that is just as scary like not being able to get to medical records or Internet connected medical devices. That is certainly just as scary as a Internet connected water treatment plant of which they all are now.

Right.

And so I do think that what we broadly think of as an industry as critical infrastructure is going to get to be a much broader definition over time. As an investor, one of the things that we think about are like who are the organizations that are willing to resource cybersecurity, right. Who are actually going to staff high quality teams and buy the product and make the organizational investment to have a fighting chance against, you know, maybe nation state attacker, but just generally making yourself a less attractive target. And there's financial services, there's large tech companies, there's certain parts of the government, there's not many groups. Right. And I asked you guys some about some of the hope that some of the change you hope to see. The change I hope to see is more people recognizing the attitude of well, what's a little data loss is not accurate today.

Right.

And I think that's true for many industries that are not sort of heavily cybersecurity regulated today. Nicole, what would you add, like what other change would you hope to see from a government perspective or a private sector perspective?

Well, it's really hard. I mean security by design, you know what Glenn just called defensive programming is something we've talked about for a long time, but we haven't really incentivized companies to practice security by design. And there's A couple suggestions out there. You know, there's things like tax credits for organizations that practice secure software design. And Glenn's right. It just comes down to how we build the code. And for too long we were shipping, shipping, shipping. We were moving fast and breaking things and just getting product to market. And now we're seeing what the trade off is for security and for attacks. I also think we do need to take stock of what's in our systems. I think we need to know what's hitting our networks. I think we need to know how much of it is built domestically. You know, we need to know where it's built, where it's tested, where it's being updated from. I also think that we need to stop leaving ourselves more vulnerable. That's it. And that means at the individual level, it means turning on two factor authentication and not clicking on crappy links and using different passwords for your bank account and your email account and everything else. And at the business level, it means doing secure coding and authentication and inviting hackers to test your code and having a channel for that. And at the government level, it means stop leaving us more vulnerable. When you find a critical vulnerability in Microsoft Windows that we all use whether we know it or not, sure, use it for your espionage for a little while, but don't stockpile it for five years until it gets hacked and dumped on the Internet for North Korea and Russia to pick up. So that's it. It's just stop leaving ourselves more vulnerable. In the case of the water treatment plant, we don't know exactly how they got in yet. We don't know if it was a nation state hacker. I joke that it was probably like a Patriots fan or someone, but it was right before super bowl weekend and chances are it was probably started with a phishing attack and they didn't have two factor authentication turned on. It probably is going to come down to some really basic things. How do we incentivize organizations to just do the most basic things? And that would not have stopped Solar Winds, but it would have stopped, you know, 50% of the attacks that we face, if not more.

Glenn, you are working with a number of organizations in the aftermath now with, with whatever you can say. Avoiding the specifics are any thoughts on how different victims of the attack are handling the experience?

Yeah, look, I mean, I think how you respond to an incident is indicative of the quality of the organization and being able to admit that there's a problem. Get in there. Look, understand the scope, get the breadth. Understand what was lost, which I think is critically important. Being able to quantify that, being able to share the information and the understanding with others is hugely important. And so I think, you know, the best organizations are the ones that are following that process. They're in there, they're trying to understand the depth and the breadth of this problem. They're trying to understand, understand how the attackers came in, and then more importantly, are the attackers still there? Because, you know, persistence is the problem. And you talked about dwell time and you know, for a long time, you know, there was all these. Dwell time was the phrase of the day for a number of years, and everyone talked about it and it was all about reducing dwell time. And of course it should be about reducing dwell time. And, you know, this is indicative of what you have to do. You have to build something securely, you have to deploy it in the right way, you have to monitor it, and you have to then expect that it's going to get breached. And if you do all of those things well, when something goes wrong, you're actually very well placed to cope with it, to find the problem and to mitigate the effect of it. So instead of relying upon someone sitting at the screen and hopefully seeing the cursor move, if you have the right detection capabilities, you detect it. Instead of that happening in the first place, if you deployed it securely and you managed it and you monitored it and you protected the configurations and you understood the way data was moving around and who the people in the system was doing, what they were doing and how they were moving in the system, then you would never have had the breach. And as long as you had secure software deployed, then that helps protect against the very first part of this, which is vulnerability to breach to detection. So I suppose the short version is good quality. Companies look for that breach, they go and try and scope it, they understand the depth and breadth of it, and then they hunt for where the attacker still is, because the expectation that the attacker is where they found them, that's where they were when they found them, they're probably other places too. You didn't spend this much time doing what the attacker did to get kicked out if you got found on the first instance. And I suppose that's the important bit here, is the good organizations are looking for where else the attacker is, and the organizations behind that are busy cleaning up the mess. And then the third bucket are the ones that said, well, we don't really have a problem, we don't think we're on that list. And the problem is the people in that bucket are the Most dangerous ones.

My lesson in that, I'm just going to say it's my first hack I covered at the New York Times in 2011 was the hack of the U.S. chamber of Commercial Commerce, which, fun fact, was leading the lobbying effort to water down a bill that would have strengthened our critical infrastructure security because they argued it was too onerous. Anyway, they were hacked by China. The FBI came in. They did a full sweep. They thought everything had been cleaned out of their network. And then a couple months later, one of their printers started printing out reams of Chinese characters. And then their thermostat in their corporate apartment, which had been acting funny. When they went back and did a little bit of forensic analysis, saw it had been communicating with a Chinese IP address. That was the first attack I covered at the New York Times. So ever since then, I've learned the lesson that Glenn is saying, which is, once they're in, you really need to find out just how far into your systems they got. And that is what the truly scary thing is about SolarWinds this attack is they were in there for nine months, if not longer, before FireEye caught them. How many back doors did they plant? And if it ends up being the svr, the unit of Russian intelligence that we've heard, it is, they also hacked the White House and the State department back in 2014, 2015. And what I remember about that attack from talking to the people who were on the ground there, was that at one point, they even took over the RSA Net witness tool that they use for their forensic investigation so that they could basically, you know, let investigators think that they had removed every backdoor when they hadn't. They were just seeing what they were detecting and making sure they were doing something else, and they were really staying one step ahead of them. And that if that is the group that we are now dealing with that was inside our systems for nine months, you know, it could be months, if not years, before we can confidently say that we've actually really eradicated them.

I think one of the things that actually makes being a defender the hardest is the psychology of never assume victory is a very difficult one for organizations to accept. And I think, you know, people who have been working in cybersecurity for a long time, they're in this, like, furious, getting to the bottom of it moon, like you were saying, Nicole. But they will assume that they don't get to the bottom of it. And I think the. The combination of two things makes it such, like, an important mission for me personally, where I think the Pace of technology change where that loop that Glenn was describing of let me shrink my surface area and protect myself, let me respond quickly.

Right.

And let me continue hunting. That is really hard when your surface area continues to move. Right. I think the continued expansion of that surface area, the shift of that to the cloud and SaaS applications, that just seems to be. That combined with the psychology of like, we can't actually win in any permanent way, I think is really tough for everybody from agencies to large organizations to accept. But I actually am much heartened by this Sunburst episode because as Glenn said, it seems possible to have like, temporary victories. Right. And like, we did find out it was happening at some point with no world ending disaster as of yet. So that's a. That's as much as I can sort of garner for optimism right now. Let's go to just thinking forward a little bit. I'd say one of the big scary takeaways for me from your book, Nicole, and generally from working in infosecurity, is that like you said, the US is, we're number one. We're the big dog in terms of cybersecurity, especially offensive capabilities versus other nations. But the takeaway for me is it, you know, it doesn't take our. It doesn't take our trillion dollar defense budget to catch up. Right? It takes a building full of hackers. And now you don't even have to own those hackers. They're contractors. They can live in Argentina. And so how do you think we can address that from a policy perspective? Because that seems like a pretty challenging structural issue where the US Is the biggest physical defensive power in the world, but we don't have the same dominance on cyber.

Yeah, it's a really hard problem to solve any way you look at it. Even Brad Smith at Microsoft has been pretty vocal proponent of a digital Geneva Convention. And when you talk to American officials, they say there's no chance in hell we'll ever sign a digital Geneva Convention because we do our own attacks from Cyber Command out of the Pentagon and our espionage from the nsa. Even if Russia or China or Iran agreed to those norms, a lot of their dirty work, like you said, Sarah, gets outsourced to these contractors. And in Russia, some cybercriminal groups do the government's dirty work, and they maintain a degree of plausible deniability that we don't have here. We assumed that by being the most clever adversary in the cyber domain, we could outsmart everyone else. We could set up an early alarm system to head off attacks before they Hit our own network. And what we're learning now is actually that's not true. You know, we missed this nine month long attack on the Department of Homeland Security, the very agency charged with keeping us safe. So the answer is very long, drawn out and boring. But essentially it's if, you know, we cannot outsmart our enemies, you're right, they can do in cyber, with a couple million dollars, they can do a lot of damage. And it doesn't have to look like stuxnet. It could just be wiper malware, it could just be ransomware on a very strategic target at the right time. So all I'm saying is I don't think we need to back off offense. I mean, clearly our adversaries feel that there are no costs to some of these attacks. And our answer for a long time was sanctions and indictments, but that hasn't worked as a deterrence strategy. So I don't disagree that we need to continue to pursue offense and impose costs for some of these attacks. But I do think if we're going to continue on this active defense or offensive strategy, we need to recognize that, you know, we do live in the glassiest of glass houses and it doesn't really matter if we're the best offensive player if whoever can come back and, you know, contaminate our water supply in an hour. So, you know, the time has come to focus on defense. We don't have to stop offense, but we have to focus on defense. And a lot of that is just the boring stuff we've been talking about, about secure coding and authentication. And, you know, it's not pretty, it's not sexy, but it's very necessary at this stage. And, you know, what's it going to take really? Are we going to wait for the water to actually get contaminated or can we do something about it right now?

Glenn, perhaps just for a broader audience here, I think there's this overall question that, you know, a layperson or even a smart technology person often has as to, you know, I don't run a water treatment plant. What do I care if they have my email or my documents or about the surveillance, Right? And so maybe you could just describe why that data matters and what attackers might do with it. Because I think even just going back to opm, the broad collection of data of your organization, your emails, anybody who's ever had a clearance, I don't think it's obvious to people why we need to care or how that data gets used.

The heartening thing in some of this is the enhanced focus on privacy. And as people start to think more about that, it's a first principles movement to start to think about security, security in a more fundamental way. And there's lots of things to be heartened about here, so it's not all doom and gloom. Why do I care? Well, you know, I'm not an important organization. I'm just an air conditioning contractor. Well, an air conditioning contractor cost a major US retailer hundreds of millions of dollars because that was a pathway into their network. Why do I care? It's just emails and documents. Well, if I can impersonate you effectively and I can target your aunt, who happens to be someone at the State Department, because I can impersonate Sarah effectively, that is a problem. If I can gather information, I can assume identities, I can assume control, I can assume and I can start to understand my adversary. And a large part of this problem is that it was 50 years ago it was impossible to understand your adversary sitting in Florida right now. It's very easy to understand your adversary sitting in Florida because you can do this information gathering process. You can start to understand people, you can start to take information looks like low value, low quality, and if it's well targeted, it's very useful. And then you get into systems like opm. Well, if I understand everyone inside OPM systems and OPM happens to have all the resources, happens to have all the records for everyone that isn't a member of the intelligence agency and somebody working at an embassy doesn't appear in the OPM records, well, who do they work for? They probably don't work for State because if they work for State, they'd be in OPM's records. So I now know who the assets are, and the undeclared assets may be, and that's a very simplistic version of it, but I mean, you just imagine the depth and quantity information that I can extract from human resource records, from personal records, from banking records. If I need to understand where you're going, what you're doing, your bank is perfect for that. You pay with everything, with a credit card. I know exactly where you've been, exactly what you like. And this is the problem, that small amounts of information add up to high quality, valuable information when it's targeted. With an attacker that's thoughtful and considered, and you cannot describe this attacker as anything other than thoughtful and considered.

So last question for you is, give me one prediction for the security industry over the next couple years. In reaction to what's happened with Sunburst.

My view would be, is that the One positive I take away from attacks is the visibility and the profile they create. And the visibility in their profile they create brings a new set of people into the security community. Twenty years ago at defcon, the security community looked a certain way. It was a small group of recalcitrants that had a certain set of motivations. Some of those people are in Nicole's book, and some of those people are very active in some of those communities. But what you've seen is this incredible influx of extraordinary talent from extraordinary people. And the more that we get those people to look at defenses, to look at capability, the more that we bring up that talent, the more that the person that was going to create the iPhone is now creating security defenses. That is a fabulous outcome, because that is where you win. Because those people are the ones that bring the original ideas. Because at the end of the day, what you're actually up against is you're up against people on the other side of the fence. And if you look at your phone, it's infinitely more secure than your computer. And a lot of that is the security model was built in by design. And whether that's the security team at Google, a number of whom have now moved on and gone to other places, whether that's the team at Apple, or even the things that Apple's been doing in the last few months to improve the security by some incredible young researchers that have been doing some great work, I think that that's. That focus, the attention, the drawing of talent, that's the outcome that we want.

Yeah. I'll add my own prediction to this, which is, don't worry, I won't actually say the words machine learning, but I will imply it. The potential for us to really have the infrastructure and the technology to look at our environments in their totality and be able to understand our networks and our environments better than our attackers do. I feel like that is here from a core technical perspective. Right. And that wasn't five years ago in the same way. Right. You know, people may have had Net Witness or they may have had ways to do forensics, but it required the talent that Glenn is talking about, the talent that shows up, you know, mostly on the offensive side, but still in your book, Nicole, to do, you know, expensive and painstaking work, and I feel like the ability to actually automate some of that and then to, you know, join that intelligently with other people trying to defend their environments, like, I am pretty optimistic about that. So, you know, be it on the access side like Obsidian or in other domains, and so I'm. I'm hopeful about that. But I deeply appreciate both of you spending time with me today. And please go read this Is How They Tell Me the World Ends. It's an extraordinary book. Thank you, guys.

Thanks so much, guys.

Okay, everyone. That concludes this episode of Gray Matter. Find our episodes wherever you get your podcasts or get episodes and blogs on our website, greylock.com and on Twitter at greylockvc. I'm Sarah Gua, and thanks for listening.