1 Apr 2021 · 28 min
John Kindervag, who created the Zero Trust framework at Forrester, explains why traditional security models failed after breaches like Snowden and Manning. He walks through his 9-point framework for implementation, starting with mapping transaction flows and defining a protect surface, then building policy and monitoring around that. The advice: start small with one protect surface to learn before scaling.
Chapters
Security.
Hi there. Welcome to the Cloud Security Podcast, and thank you for joining us today. Your hosts here are Tim Peacock, the product manager for threat detection here at Google Cloud, and me, Anton Chuvakin, a reformed, mind you, analyst and member of cloud security team here at Google Cloud. You can find this podcast wherever podcasts are distributed, at our website, whenever we finally launch it. You can also follow us on social media at twitter.com/cloudsecpodcast. Our guest today, and please pay attention, this is important, is John Kindervag at OntoIt, formerly of Forrester fame and Palo Alto Networks, who was the first to define the concept of zero trust in his 2010— think about it, 2010, 11 years ago— paper. So we have a few questions mostly focused on zero trust and of course its past and the future. So let's start from a somewhat painful but necessary question. Let's define zero trust, perhaps contrast how you define it back then and how you think about it today.
So zero trust hasn't changed, right? It was still a fight against the old trust model where we had trusted parts of the network and untrusted parts of the network, as we would see implemented in, say, an old Cisco PIX. And so you had to define policy based upon a trust level. So the internal interface of a PIX was Trust level 100, the highest level, and the external interface was trust level 0, the lowest trust level. So you could go from a high trust level to a low trust level without any policy. And I thought having that variable is painful, and it means that there's no outbound rules and it's highly insecure, and we allow people to have access because of this trust model. And so trust is just a human emotion that we've injected into digital systems for no reason at all. And people confuse all the time human trust and digital trust. I mean, going back to 1984, Ken Thompson, who we all know as the co-creator of Unix, in his Turing Award speech that year, talked about the problem of trusting trust.
Yeah.
So trust is something that shouldn't be in digital systems, and that was the main thesis Of the report, and then it led to how do you build systems like that. But mostly it was about thinking that this concept of trust actually incentivized bad behavior because all data breaches and almost all negative security events that have ever happened, the root cause is the trust model. You think it was a spam email, but it exploited the trust model. You think it was ransomware, it exploited the trust model. Snowden and Manning were insider attacks who exploited the trust model. So That was my fight, and then it's led to a lot more stuff.
I think that that's one of the more wide-ranging attempts to define something on the podcast so far, and I want to celebrate that we've made it 8 episodes without bringing up untrusting trust. So you've, you've broken that barrier. I think it's never going to be the same for us. But what is zero trust? Could you do that again?
Zero trust is elimination of the broken trust model that exists in digital systems today.
Okay, I like that. That's concise. And how has that changed since 2010?
Well, it hasn't changed. As Chris Hoff says, there's just a lot of people who lost the plot, right? So they'll say, oh, the goal of Zero Trust is to make a system trusted. No, it's not. It's to eliminate the concept of trust so we look and constantly validate it. Part of it was inspired— I was talking to a CISO many, many years ago, and I said, what's your strategy? And he says, well, It's trust but verify. And I said, okay, trust I get, that just means you're not going to do anything, but what are you doing for verification? He said, well, we actually don't verify because that'd be rude because they're trusted. And so then I started, why do you do trust but verify? And he said, Ronald Reagan said so. And so I looked it up and Ronald Reagan never said trust but verify. He said a Russian proverb. And since I've got Anton on here, he's going to say it in real Russian because I butcher it every time I say it. The actual phrase is what, Anton?
It actually rhymes in Russian. It's "довиряй, но проверяй". It sounds exactly that: trust but verify. But in Russian it rhymes, so that's the cool part.
But it means the opposite thing, and so you can see the video of Gorbachev and Reagan, and they're both laughing, right? He's making a joke. So trust but verify was always a joke that Ronald Reagan made it made. But we, of course, yeah, we need to do trust but verify because that great cybersecurity expert Ronald Reagan said so.
I now know, by the way, why when I showed John the questions, specifically the next question, John said, we're going to argue about this one. And now I know why. So the question we prepared was this. Sometimes I hear people say that zero trust means you cannot trust anything, which is, in my opinion, possibly mistaken, was false, because you need to trust the zero trust system itself and other things like identity data, endpoint state data. I think from John's definition, I'm off. Maybe there is a way to build a system that will just work and you don't need to trust it. So could you help us here? What needs to be trusted for the zero trust system to function, if anything?
In digital systems, we say trust is a 4-letter word, right? So don't use the word trust. Trust is binary. Once you say, well, I don't trust Anton very much anymore, he's untrusted, right? But I can say I have confidence in the system. So I'll substitute the word confidence because it's measurable across the spectrum. And then when When you talk about the system, right, I can measure the input and output of the system and see whether I get the expected results. This is something that we do like in supply chain or something. You know, you're testing, if I send an input to a device, am I getting the proper output? Because I'm going to see somewhere inside the packet that something has been changed of what I expected to see. So we can test that and measure it and validate it. So the word I tend to use is validation. How have I validated A system so that I can have an extreme level of confidence that it's doing what it's supposed to do. My saying is trust is a vulnerability, and it must be mitigated. And the reason is because people exploit it. As soon as you're trusted, then nobody does anything. And I was in Switzerland, but in the German part of Switzerland, and everybody was saying, "Oh yeah, it's like you know farfignugen or some long German word," and they were trying to explain why people do this trust thing. And finally, they were all talking amongst themselves about how do they translate in this very. long word into English. And finally the guy said, lazy. People are just lazy, so it's easier to do this, right? And so now we're validating the system. And I also have a saying that people aren't packets, right? So we have anthropomorphized everything. We would say colloquially, Tim and Anton and John are all on the network, but none of us are on the network right now. A packet that is asserted to come from a machine That is asserted to being used by us is riding across the network. People aren't packets, right? So maybe in the movies, Tron, Lawnmower Man, people are on a network, but even in The Matrix, they got to plug in, right? So it's those simple things that cause people to rethink things, and then they will lock stuff down. I mean, look at Snowden and Manning. What did they exploit? Did they have any great hacking techniques?
No.
Did they create new malware? No, they exploited the trust model. Once you're on that network, SIPRnet, you have access to everything on SIPRnet. So Snowden is setting up all these SSH tunnels. No one is noticing that because nobody's looking at his packet post-authentication. And so we need to validate the asserted identity of the user, right?
Mm-hmm.
That's an assertion. It's not a real user. And then we need to constantly look at that traffic and see if it's behaving correctly and apply policy at layer 7 about that behavior or that expected behavior. And then, of course, we need to look at that authentication because authentication has entropy attached to it, right? So those are different ways that you can look at it.
So when I used to say that for the ZT system to function, you need to trust identity system or identity validation, I really should not have said it. I should have said, For the zero trust system to function, you need to have a system whereas the asserted identity can be reasonably seen to belonging to the right person or some such thing. Yes.
There's that philosophical question of ontology, right? So I was talking to somebody from the, let's just say the intelligence community, the IC, and I was joking. I said, well, hey, if somebody comes up and puts a gun to my head and makes me get up from my computer and then starts using it, do they suddenly become me because the computer is equal to me? And so now they're John, right? And he laughed and he says, yeah, that It happens a lot more times than you think. And I said, it does not. What are you talking about? And he says, John, it's not a real gun. It's a virtual gun. I can say to you, hey, either you get me this intellectual property or I'm going to tell this dirty secret about you. Or I can say, hey, if you get me this intellectual property, I'm going to pay you $1 million. It's just traditional spycraft into the digital realm, right? Robert Hanson was trusted. Aldrich Ames was trusted, right? So identity is an assertion, it's fungible, and it has entropy. And it's important, but it has to be constantly validated, and then it has to align with the policy-derived behavior expectations.
So steering a little more tactically here and away from the philosophy of all this, you've mentioned identity systems as a prerequisite Behavioral expectations, policy. When you've seen organizations do a ZT project, what are the things they need to do to start with? How do they make it go right? What are the most manageable and important things for security teams and IT teams to grab onto at the start?
Yeah, I tell people you need to know 9 things to do Zero Trust.
9?
There are exactly 9 things. There are 4 design principles. First one is align with the business, right? What is the business trying to do? Focus on business outcomes. Second one, design the system from the inside out. What are you trying to protect? That is the most important question. If you do not know the answer to that, it doesn't matter how many controls you have. You just get into what Rick Holland calls expense in depth, right? Instead of defense in depth, it's expense in depth.
I like that.
The third thing we do is control access, right? So you do access on a need-to-know basis, privileged access, all those things we talk about. But that's not enough, right? Snowden and Manning proves that that's not enough. Then we inspect and log all that traffic, and we do it at layer 7 so we can apply a policy at layer 7 that includes access control and inspection. So zero trust is ultimately a layer 7 policy statement that has to be enforced by layer 7 controls, because all attackers know how to bypass layer 3 and layer 4 controls. They all do, right? And so that's pretty trivial for them. And so if you say, well, In my cloud, I'm just going to turn on a stateless ACL in iptables. The nation-state attackers are going to laugh at you. Oh, you think you're protected? Hmm, great. Be happy with that. So those are the 4 design principles, and then there's 5 steps on how you deploy it. So the first thing is you focus on a protect surface. So what I've taken is the concept of attack surface and completely inverted it. And instead of saying, what are all the threats out there? What do I need to protect?
Hmm.
It's a much smaller thing. I can shrink it down orders of magnitude to something that is very small and easily known. So I usually use a metaphor of how the Secret Service protects the President of the United States. You know, like at the inauguration parade, there are 4 people in the protect surface: the President, his wife, and whoever else, his children, whoever was there. And that's all the Secret Service cares about. That's all that has to survive inauguration day. And that protect surface is decoupled from transport. So sometimes they're going to be on Air Force One, sometimes they're going to be on the helicopter Marine One, sometimes they're going to be in the limousine, the Beast, sometimes they're going to be walking around. They might be on a rope line and doing handshakes, but that's still a protect surface that the Secret Service understands. And so what you put in a protect surface is what I call a DAS element. That's an acronym that stands for data that you need to protect, anything sensitive, assets that you need to protect, IT/OT, IoT, IM/OT, IIoT. They're all the same. They all run on TCP/IP, don't they? The third thing we protect are applications that typically run sensitive data, ERP, CRM, HR. And the fourth thing are services, that's the S. So I think of DNS, DHCP, Active Directory, Network Time Protocol, things that are incredibly fragile. So we take a single DAS element, we put it into a single protect surface, and we build it out one protect surface at a time so we can do this incrementally, iteratively, and non-destructively. So that's step 1, just know what you need to protect. If you don't know that, none of this stuff is going to work. It doesn't matter what you do. The second thing is you map the transaction flow so you understand how the system works. And I learned that the hard way when things went south a couple of times because we're working on a system that nobody understands, right? What's this server here? Wow, we probably don't need that. Let's take it out. Well, it was integral to the system. So understand how the system works. The third thing you do is you architect the zero trust environment no matter where it is. It could be in a public cloud, private cloud, on-premise. It could be on an SD-WAN. It could be on an endpoint. It could be in a SaaS environment. Just what do I need to protect? Where is it? And then figure out what's the right architecture. So zero trust is a bespoke or tailor-made architecture for each protect surface. Then the fourth thing we do is create policy. I have a method about that called the Kipling method: who, what, when, where, why, and how. It's my personal homage to Rudyard Kipling, 'cause he gave us that idea in a poem in 1902. I keep 6 honest serving men. They taught me all I knew. Their names were What and Why and When and How and Where and Who. So that's where we get that, and that resonates across all languages and all cultures, so far at least. Everybody has that concept. And so I can define things, and I have documents written about that. And then finally, we monitor and maintain it. So we take all the telemetry. You know, the first time I ever met Anton, he was wearing a button that says, I love logs, right?
I love logs, correct.
I mean, he's the only person I've ever met who actually loves logs. He does.
I actually feel pretty similarly.
I wrote the freaking book about them.
Yes, he did.
I'm with you, Anton.
Now's the time. Now is the heyday for logs because now we can correlate them all. And now that we're seeing what's going on in the system, we can refine each of those 5 steps and make an antifragile system out of it. So if you've ever read the book Antifragile, Oh, Nassim Taleb.
We're going all over the place with the authors today. This is great.
Yeah, totally.
So the more that system is under load, the more we're going to see how we need to refine it. And we go, oh, we need a new controller, we need to update policy. And so we've got this feedback loop. And so Zero Trust is an antifragile system.
Wow.
And so once you've done those 5 steps for one protect surface, you just do it for another one and another one and another one. And I have a thing called the learning curve Where I talk about how you do it one protect surface at a time. You start with low sensitivity resources early on so that if you break something, it's safe. So I call those learning protect surfaces.
Mm-hmm.
And then you get up and you do some practice protect surfaces, you know, things that have a little bit more sensitivity, not mission critical. I'm going to get better at it. We need to practice. So Tony Scott, the former CIO of the US federal government and Microsoft and VMware and Disney, he's become a friend of mine. And he told me this. He says, I tell people, I tell people, you get to zero trust the same way you get to Carnegie Hall. You practice, practice, practice. Yeah, yeah, yeah.
I love it.
After that, that's when you do the high-value assets, the keys to the kingdom, the crown jewels, whatever you want to call it. And I used to tell people, start with those crown jewels, and then I found they were too fragile. And there was nobody in the company or in the organization who knew how it was built, and you would see things go south. So give people some experience before they do this, and then you just follow those 5 steps. So I've made it simple. I've given you 9 things. And I read other people talk about it and I go, whoa, I don't understand this and I created it. So I can't imagine what the average poor person out there who's trying to figure this out with all of the competing stuff. So I'm calling what I do authentic zero trust, right? Hmm. Cool.
You have the brand Authentic Zero Trust.
I like it. Locally sourced artisanal zero trust.
Artisanal zero trust. Yeah, there you go.
There's actually some really cool parallels here between what we heard from Max on episode 4 and what we're hearing today.
Yep. Yep.
In particular, I'm hearing similarities in the agile and the repetitive approach to this. The like, learn how to do it on something low risk and then build up the flywheel of skill here. I also really like the parallelism between understand your data and your assets at risk and protect surface and do one thing at a time. There's— this is a well-considered model.
Yep. Yep.
Well, thank you. Thank you. Well, it's even longer than 2010 because remember, I spent 2 years researching this, talking to lots of people, validating it. Test marketing it with speeches and webinars before I ever wrote the paper. So what you learn in the analyst business is how to do primary research, how to ask the right questions to the right people, go to the original source, which a lot of people don't do. I mean, I went to a ton of people, people who were experts, poke holes in this, right? And those people all helped. And so that's what you learn to do when you're in a place like Gartner or Forrester. Or whatever because they give you the freedom to think big thoughts and they teach you a methodology on how you can do that in a way that doesn't make you look like a nutter and you can create things that you couldn't create this kind of stuff in a university, right? Imagine if you had to do peer review and all this stuff and everybody said— they'd say the same thing that they said to me early on, that's not the way we've always done it. That was the—
Yes.
Big objection. That's not the way we've always done it, right? Imagine Einstein trying to come out with something today, right? No, no, that can't be right because that's not the way we've always done it. And that's kind of the history.
And he would face outrage in social media, I'm sure. Right, right, right.
What do you mean space and time are flexible, right?
Hey, I will say that this is a great learning experience for me today because I'm finally seeing the value that these analyst firms can bring other than really annoying to deal with acronyms. You know, my life as a PM I feel so beholden to the analyst firms, but it's really nice to see the real value that they bring to the community here.
So apparently I failed to express this, but then John shows up and he's like, dude, this is how you do it. And Tim is like, yeah, totally, very clear.
Exactly.
Okay, thank you, John.
Exactly. Anton and I have been doing this for months now. I never got it, but now I do.
Oh, well, good, good.
Well, I'm not offended at all.
Anton is a much more brilliant person than I am. I have to make things simple that I can understand, right?
So one other question we sometimes ask some people is this: what is the first step in the zero trust implementation? I think your previous question kind of answered it. But the funny part is, from other people, we encountered some difference of opinion. So, for example, somebody said asset management, somebody else said identity, somebody else said what you said, namely the crown jewels. Well, it's kind of connected to assets. Do you have a pithy short answer about what's the correct first step in a zero trust implementation that has the highest chance of success?
Determine what you need to protect. I mean, it's that simple. I was just on this big call and they had all these people who had various opinions on zero trust and they were all mostly from vendors and I was from a vendor from Palo Alto Networks, but they're all giving their opinions about what they need to do and I'm just sitting back listening, you know, because it's kind of fun. And finally, I just interrupted and I said, excuse me, what are you guys trying to protect? And it just ground to a halt. No one had ever considered that because everybody was trying to position, here, you need my endpoint protection because it's better at zero trust than anybody else's. And I was just like, who knows, maybe that's true, but that's not the question here. The question— until we know what you need to protect, we will always fail, right? And that's why I have that DAS thing, and that's why I have the learning curve, because Don't start with the high-value assets because you'll probably fail and you'll blame the failure on the zero trust model instead of the fact that you don't understand how your system works, right? So give yourself an opportunity to fail. I have a friend who's a CIO in Florida and he wrote an article for Security Roundtable for me because he actually has a degree in classical organ, right?
Where is this going?
Well, think about how much he had to practice to be a classical organist. He didn't just decide, Oh, here's the keys and now I'm going to go play a concert, right? He had to practice. And we train people in our business, but we don't give them the ability to practice and learn what they're doing. And that's the issue, right? And so give some people the opportunity to practice. That's where it's going. You know, you need to practice something to get good at it.
Okay.
I actually really like that. There's kind of like almost defending the notion of zero trust against the failure to begin with. succeed as you get through the tough part. So speaking of defending against failure, what's the most spectacular failure you've seen in a ZT project? And also, what's the most spectacular success? What are the ends of the spectrum here?
The biggest failure I've ever seen was in a retail environment. Somebody, as we were going through this process, said, what's that old server there? Let's pull that out. And this is for the POS system, right? So we're getting back to PCI because I'm a recovering QSA myself, right?
Mm-hmm.
So yeah, if you've ever done PCI, you know it's a 12-step program, and, and only people who've done PCI get that joke. So they just pulled this server out. Ah, it's an old server, let's get rid of it. Well, it was the polling server between the point-of-sale service and the backend system. And so everything went from the point-of-sale server, it was pulled from, from that into the polling server and then sent to the database in the backend. And suddenly in 1 minute 3,000-some-odd stores went down.
Yikes.
And that was when I learned map the transaction flows. That's probably the biggest failure. The biggest success probably that I've seen is at a manufacturing organization, seeing them really struggle because manufacturing facilities are run by computers, right? And inside the computer, there's a board called a PLC that's a controller. And that runs software. And so there was this big fight between everybody because it was running on Windows XP. There's no longer going to be any patches, and you couldn't patch anyway because it's 24/7 system. Even if we upgraded, the software wouldn't run on it, and the manufacturer isn't going to make drivers for it, right? So suddenly it's this rock and a hard place situation, and I'm with the global head of manufacturing, and I'm explaining how I can build a zero-trust environment around this. to put a micro perimeter around that system and protect it without having to upgrade it and do anything, and it'll still run 24 hours a day. And he got it. I mean, he got it like this, and suddenly he became the champion for the entire company's digital transformation around zero trust. Because where did they make their revenue? From manufacturing a product. And so, yeah, he's going to be that guy. In fact, a lot of times the CEOs, members of boards of directors, People who run the business get zero trust easier than the technology people who are tied into some legacy thinking. So that's probably been the biggest success.
That makes sense.
I've heard that from you before, by the way. This is one quote I recall from you when you were at Forrester, actually, that the CIOs sometimes get it faster than engineers, which struck me as strange and illogical, but now I actually get it.
Well, because I'm asking somebody to do something different than they've always done it, and I'm asking them to change, and no one likes change, right? I'm enabling security. secure business operations and your ability to make money. And so I did this primary research at Forrester on business alignment, because everybody in IT says, we want to align with the business. Well, if you look at what the priorities are, if you have a bunch of people stack rank them, and you say, business leaders, what are your 3 top priorities? The outcome was increase revenue, increase profitability, and stop data exfiltration of our stuff. The IT people were catch viruses and stop threats. And the last things that they were thinking about was increasing revenue, increasing profitability. So how do we align? Zero trust aligns us to the business thinking. So I'm going to enable the business to do things securely instead of being the department of no, that's what security is. I'm going to be the department of yes, let me do that for you in a secure way.
And so funny enough, we're kind of getting to time and I had this simple, short, crisp question I want to ask you. What do you think will happen in the next 10 years? I'm sure you can handle it in like, I don't know, 3, 5 seconds perhaps because it's so simple.
His answer to what's the first step was actually very pithy, so maybe this one's going to be good too.
You've lived this life for 10 years, 11 to be precise, and then plus 2 more when you did the research. So what do you see happening in the world of zero trust in the next 10 years?
I'm going to answer it in less time than it took you to ask the question. We're going to see near-universal adoption of zero trust concepts in these systems starting real soon. There's going to be huge amounts of external gravity, whether they're compliance or business things, to adopt zero trust. zero trust mindset. And you're seeing that when you're starting to see lots of people talking about it. I mean, you know, the CIO of Google was one of the first guys who said do zero trust. And in fact, a lot of things that I started to do came from somebody called Google, and Google said call me, call John, right? So I got a lot of business from you guys, so thanks. But you're starting to see the top military officers who run DISA starting to talk about zero trust. You're starting to see a lot of that stuff. NSA releasing their guidance, which was pretty darn aligned with authentic zero trust versus, you know, multi-factor authentication equals zero trust that other people do.
Oh yeah.
So you're going to see a near adoption of it at a strategic level. And then there's lots of different ways you're going to deploy it based upon what the protect surface is. So there's room for everybody to play this, but just don't say that you have a zero trust product. say my product works well in zero trust environments.
I like that. So I think with that, we're just about at time. John, I can't thank you enough for joining us on the podcast today. We had a conversation that ranged from Kipling to untrusting trust to a 9-step program, which was really 4 plus 5, which does add up to 9. It's been, it's been a great show today. You can find this podcast wherever podcasts are sold. You can follow us on Twitter, twitter.com/cloudsecpodcast. Myself, I'm on Twitter, @TimPeacock. You can follow my co-host Anton, @Anton_Chuvakin. So please, as always, tweet at us, argue with us. If you like or hate what you hear, we might just invite you to the next episode, which I promise is more fun than arguing with us on Twitter. And so with that, John, thank you once again. And to our listeners, we will see you next time on the next Cloud Security Podcast.
More from Anton


EP289 Software Engineering vs. Software Craft: How Google Scalably Eliminates Classes of Vulnerabilities


EP288 CISO Tested, Board Approved: Cloud Resilience with General Mills CISO Noah Korba


EP287: Creating Trust at Global Scale with Local AI: Reken-ing with Shuman Ghosemajumder


EP286 Building an AI-pilled, solo vibe-coded, Clickhouse-based SIEM with Dan Lussier


EP285 Scaling Lessons from Leading the NSA to Defending the World with Morgan Adamski


EP284 Closest Alligator to the Canoe: How Transforming SOC Became P0 for Lloyds Bank


EP283: How Google Cloud CISO Chris Betz Uses LLMs to Defend Billions of Users from Vulnerablities


EP282: Debating Coupled or Decoupled SIEM with Alex Hurtado and Christopher Witter


EP280 Hyperscaling Cloud Security: Wiz Joins the Cloud Security Podcast by Google