Episode 13: Expert Practitioner of Analysis Carmen Medina

Informing your decisions with intelligence analysis and insight brought to you by the team@ooda loop.com.

Hello. Carmen Medina, welcome to the Oodacast.

Hey, Bob, how are you?

Very well, thanks. And I'm so excited to have you here because, you know, we want to focus this podcast on people who want to keep learning and thinking and understanding the world around them. So we invite thought leaders like you into the conversation. And I've already got a bunch of questions I want to ask you, Carmen, but before I get into that, I wanted to give people a feeling for your background in your bio. So can I hit a few high points and then ask you if I left anything out? You spent a career analysis in the intelligence community at the Central Intelligence Agency, and you worked in several positions there, including head of the Strategic Assessments Group, and you then went to become the Deputy Director of the Central Intelligence Agency for analysis.

Correct.

And to me, I was in the intel community in or around it most of my life, and I always regard that position as the number one analysis position in the community. You're a leader, a manager, a thinker, a trainer. And after great work there, you went on to actually teach in the intel community at the center for the Study of Intelligence.

Correct.

And since then, you've been a writer. You've written books, including Rebels at Work, which is a handbook for leading change. And follow on to that, dispatches from Rebels at Work. You've also consistently been writing on your blog@recoveringfed.com you've worked in industry with Deloitte and your own practice. And did I leave anything out there, Carmen?

The only thing I might highlight is why or how it came to be that I had expertise on being a rebel at work. And that's when I was at CIA from the mid-1990s when the Internet arrived, I thought, man, every knowledge organization is going to have to change to adjust to this new reality. And the CIA was slow to pick up on that. And so I was sort of forced into this position of being a heretic at CIA, arguing for something that the organization didn't believe in. And I made all the rookie mistakes. And that's what led to the writing of the book Rebels at Work.

Rookie Mistakes. Like what?

Well, like. Well, the first, most obvious one is not realizing that by arguing for the Internet and it's important at CIA, I was arguing against the prevailing theology. So the CIA is about secrets and controlling information, and the Internet, particularly in the 1990s when it came out, was about open information and no secrets. And if I had just put Two and two, together and gone. Okay. I can see why the CIA is not going to be receptive to this. I think I would have proceeded differently, but I didn't make that connection until after the fact, which I'm embarrassed to say.

You know, by the time I met you, you were already very senior. And in my few interactions, you were incredibly polite, friendly, always smiling. And I tell you, when I was a more junior person in the intel community, I didn't have that together. And for some reason, I too frequently would be the skunk at the party. And I know that's the wrong way to be a rebel.

Absolutely, yeah.

Were you always so polite and friendly and courteous?

Well, I, you know, that's a good question. I mean, I'm an introvert anyway. That's my natural disposition. And I try to be cheerful. I think my sense of humor is always on, or my sense of whimsy or the absurd. But when I was trying to make progress with these heretical ideas at CIA, I became less nice and, you know, ended up making enemies or alienating people that I should not have alienated. So. But I think once I became, I really enjoyed being a sort of strategic level leader. I think this is going to sound pretentious or obnoxious, but I really do think it plays to my strengths, which is I like to think big thoughts, but I'm not good in the details. Right. I'm not. Not particularly good in the execution of it. And so strategic leadership positions are sort of suited my strengths and hid my weaknesses. Cool.

Hey, Carmen, another thing I want to talk about is you've been. You write a lot on your blog at Recovering Fed. And some of my favorite topics there. When you write about thinking.

Yes.

And how we think. And you've been doing that for years, ever since you transitioned out.

That's true, yes.

Recently you've been writing about how we think in the context of the coronavirus.

Right.

In fact, two blog posts in particular, Part one and part two on that topic. I wanted to ask for a little more context around that.

Sure.

Like you mentioned in the first post, this concept of cognitive traps.

Right.

What did you mean by that?

Well, I think that we all have a, for lack of a better word, neurological disposition. I mean, we're learning a lot about how our brains are wired. And then, of course, our experiences affect us in how we think. And so we all have tendencies. And it's very important that we. Nobody can be objective. The idea, the goal of a perfectly objective analyst is just fallacious. It's a Dream, the best we can get is to try to achieve objectivity about our own biases, to understand them. So I have a bias that I'm an optimist by nature and it's very hard for me to accept pessimism. For example, something I got wrong. I just didn't see how Yugoslavia was going to break up in the early 1990s. I couldn't figure out why people who had McDonald's would fight a war. And that was definitely my pessimism bias showing up. A bias that I think has affected us during this COVID 19 period is this bias that I think almost all of us have, which is to just kind of assume that worst case scenarios don't happen. You know, worst case scenarios are unlikely and in reality a worse, you know, the impact a particular situation is going to have is independent from its probability. So there they are, two variables that are independent of each other. And yet in common thinking. Well, not just in common thinking. Among policymakers, it's very normal to think, oh, well, that's the worst case scenario. So it's not going to happen. And I think that's what happened with COVID 19. People could see what was happening in China and then Italy, Iran and then Italy. And for reasons that completely escaped me, they assumed it would not occur in the United States. It's now, in retrospect, it's. You can't really explain it, but we lived through it, so we know that happened.

Right. In fact, early on, way before Italy, before Japan, before South Korea, there was Wuhan.

Right.

And you were looking at that then and thinking something was up.

I sure was. You know, I recall being at a conference in mid January, January 16th, Minnesota, and there was a contingent of attendees there from Seattle. And I was thinking, golly, if the virus or when the virus comes to the United States, Seattle is going to be a likely initial port of entry because of the Asian American connection. And of course, it was Lunar New Year, so there was a lot of travel going on. When I looked at what was happening in Wuhan and that the Chinese government was closing down, essentially its Chicago, right. Its major industrial city, closing it down and closing Lunar New Year. Right. Stopping the celebrations. I was thinking, wow, why would a government do something like that? That is for the Chinese Communist Party, whose legitimacy depends upon economic growth, it just made no sense to me. And I mentioned in the blog post that this is one of my favorite heuristic devices as an intelligence analyst, that actions reveal intentions and motivations, that when someone does something, they're doing it For a reason. And so try to figure out what that likely reason might be.

Right. Man, that really resonates with me. It really does. And I just wish I was better at that personally, and I wish our policymakers were better at that. Additionally, you write in the second blog post about the fact that sometimes, even when you have all the information and all the evidence, you come to the wrong conclusion.

Right? Yeah.

Can you explain what you meant by that?

You know, that's an interesting phenomenon because you would think that scientists would be trained at the appropriate use of evidence. But we know, for example, that the scientific community is full of this replication problem. Right. That someone will conduct one experiment and think it has some kind of universal meaning and then nobody else can replicate it. So when you draw definitive conclusions, you really have to have strong body of evidence. And the corona crisis shows that, you know, everyone, not just non experts, but scientists and politicians, are all prone to cherry pick the evidence that suits their bias. And you know, I loved when people would say, oh, it's no more dangerous than the flu, only 500 people have been killed. And I'm like going, well, it's just gotten started. I mean, there's this element of time as it unfolds, right. We're going to have more casualties. Even that Stanford study, the first iteration of the Stanford study that people point to as one that is not as bleak as so many of the others. The first time I read it in mid March, it predicted that only 500Americans would die. And I remember reading that at the time and thinking, it's just bogus. There's no way it's going to be just 500Americans. And yet a lot of people, a lot of experts hoping that it would be true, like glommed onto it.

Right. You know, it reminds me of so many cases from history where people have gotten the information they should have had for a forewarning of some event and didn't take the event or didn't follow through. And one is the famous Soviet spy, Richard Sorge, who was in Japan, who some people say he was the greatest spy.

Right, right.

And he got all the information available through his network in Japan saying that the Nazis were going to invade the Soviet Union, got that all, all the way to Stalin, who just disregarded all of it.

Right. Well, another good example of it in my professional career was when Saddam Hussein invaded Kuwait and people had picked up, using satellite imagery, all these Iraqi movements. And I remember one person trying to explain that away by saying, oh, it's just for show. And I'm thinking, well, it's only for show if you have a satellite to see it. Otherwise it's just stuff being done in the desert where nobody is seen. So it just doesn't make any sense. But again, I think we're really in danger of falling into a cognitive trap when something happens that breaks norms. Right. And because we norms are norms for a reason, that's how the world normally operates. We don't normally have pandemics and we have a tendency, this is another way of saying the worst case scenario is viewed as unlikely. We have a tendency to just wave away evidence of norm breaking. Oh, that's not happening, right?

Yeah, yeah. This is fascinating. And see, this also relates to another topic you wrote about, which is tools and the fact that, you know, like those satellites looking over the desert, if you didn't have that tool, you know, you would have had that data.

Right.

You also, you write about the fact that knowledge tools really inform the decision making. And what do you mean by that?

Well, everything that we know and we forget this is a function of our tools for knowing. And our tools are always incomplete. Only an omniscient being God has a complete set of tools. So right now our tools for understanding the virus are models. But these models all have some kind of bias built into it based on the assumptions that the modelers make to create the model. Right. And so we have to be very humble. We have to know, understand that our knowledge is always incomplete because our tools are always less than perfect. And for intelligence professionals this is particularly true because our tools, secrets and intelligence collection are naturally biased. Right? They're naturally biased because we pretend, I think, that secret information is better than open source information. They're naturally biased because, you know, we can only see what the satellite sees or only hear the communication that's intercepted and we don't know what percentage of reality any of those things represent. So our tools are incomplete, our knowledge is incomplete. It only reflects a certain unknown percentage of reality. And yet we have to make decisions based on that.

Right. So mental models are also tools.

Yes.

And I think maybe that is our great hope if there's training going on, because obviously we can study this. Daniel Kahneman, for example, Jay Richards Hoyer, all that great work that you did at the center for Intelligence. Maybe our hope is that we can teach analysts to understand all of these biases and knowledge gaps and problems with tools and improve the outcome, Right?

Yeah, absolutely. I think that a great analyst is really in a sense a philosopher of knowledge in that he or she spends a lot of time trying to understand the limits of what is knowable. Right. And it's difficult, though, because policymakers crave certainty. They crave certainty and assurance. Right.

You know, for the CEOs in our community, that's what they crave also.

Yes.

They like to have a nice, predictable pipeline of coming business, and they want a good, accurate assessment of their competitors.

Right.

And now in this pandemic, you know, even though it could never be truly accurate, and I think most good CEOs are good at ambiguity anyway.

Right.

But now it's all ambiguous.

Right, Exactly. You know, we have, we, you know, we don't know whether it's going to be a sharp V or a very long and shallow U recovery. We don't know if there's going to be a second wave. We don't know if we're going to have a vaccine. We just don't know. And so therefore, we have to try to assume the posture that offers the most resilience. And if you're lucky, and I think some companies are doing this, if you're lucky, you actually, one of the best ways to grapple with uncertainty is to create a new reality for yourself, to kind of create this new way of doing things that takes advantage of the uncertainty. And I can't think of them right now, but I think there are some companies that are trying to figure out how to do that. So, for example, if you embrace working from home and rather than see it as some kind of necessary nuisance, actually go, okay, what are all the new things we can do now? How can we cut costs if everybody's working from home, for example? Right.

Yeah. Well, I'll tell you, Carmen, one of the new realities for my company, OODA, is this very podcast, because Matt DeVoe and I were planning our big conference on the 19th of March, and we're so excited about that. And about two weeks prior, we realized that this pandemic is coming sooner, Vice later. We always knew it was coming, but we knew we had to cancel.

No, you guys made a very good decision.

Well, thanks. Thanks. As soon as we did, we realized hour after hour that was the right decision. But then we had to think, you know, what do we do? Well, we want to keep interacting with our community. We want to do these kind of video things, or we invite thought leaders like you to interact with the community. And it would not have been done except for the pandemic.

Yeah, I think that that's one of the best practices for dealing with uncertainty is you have to. You can't stop creating because, and you know, you have to create an experiment with lots of different things because some of them will stick, some of them will work, some of them won't, and that's okay. But some of them will. And I think if you keep churning, if you keep moving your legs, you know, keep your legs moving, I think that you. Can you stand a better chance of prospering during this time.

Carmen, I want to ask a few other questions about things that I just know that the average business leader or CEO would really appreciate your perspective on. One is you talked about the streetlight effect.

Sure.

What is that?

Well, the streetlight effect, it's an old joke, and it's usually told that there's a drunk person on their hands and knees looking for something, it's night, and the policeman finds them and goes, well, what are you looking for? And this person says, I lost my car keys. And the policeman says, is this where you lost them? And. And the person says, no, but it's the only place I can see. So it's just another reflection on this problem of evidence that sometimes we just take the evidence that's available to us and use only that to make a decision. And at a minimum, I think you need to go through the thought exercise of asking yourself, okay, this is all the information I have. What percentage of reality does it represent? And, you know, people might say it represents 10% of reality, or someone might say 50%. But just asking yourself that question, I think will give you some perspective on the streetlight effect.

Yes, I think the streetlight effect is just so relevant for business people because, you know, they'll look to their internal databases, and if that's all you're doing at this particular time of crisis, you know, you might be expanding your horizon.

Right, exactly.

Another one you write about is the paradox of warning.

Oh, yes, that's a tragedy. Right. So the paradox of warning is that essentially, if you are a warning analyst, you're hoping to be wrong. You know, in other words, by warning, you hope that the policymaker will take action that will prevent the bad thing from happening. Right, Right. But then if you do take the action and the bad thing doesn't happen, then people will say, well, you were wrong. You shouldn't have warned anyway. So that's the paradox of warning. And you're seeing that right now with a disease where, you know, sadly, actually, the best case models have proven to be wrong. Because we already have more than 60,000 deaths, we're easily headed in the US to 100,000. I think by the sometime in June. But there are still people who are going, see, it wasn't so bad because we haven't had 2 million deaths yet. And that's the paradox of warning. Hopefully we won't get anywhere near 2 million deaths because we will have taken appropriate actions. But once you take appropriate actions, then the Warner person who warned is at risk of being labeled wrong.

You know, another that I see again and again in the cybersecurity world, that exact effect is a CEO will take appropriate action, like stand up a security operations center and fund a great CISO activity and do vulnerability management. And this group will secure that organization. And then the CEO will be biased to think, oh, well, see, I'm spending too much on security.

Exactly. Right. Yeah. It's a funny thing, you know, because if you want a resilient organization, you can't spend 100% of your time and money on just executing your plan. Because by definition then you're not resilient. You don't have any kind of strategic reserve or capability. And yet people are. You know, it's one of the things this crisis is teaching us, that this drive to be just in time, absolutely efficient, can have a catastrophic cost when a worst case scenario happens.

Right.

And the other thing I would just add, given how humans are behaving, you know, a lot of people are complying, but a lot of people clearly object to any rulemaking by authorities on how they should conduct their lives. I think that has implications for the cyber community. Right. People are. A lot of people are skeptical about expertise and professional advice. And even during the pandemic, they have remained skeptical about expertise and professional advice. So that should give cyber professionals pause about, you know, the motivations and the thinking ability of the average person in their business.

Right. This does give me pause. And it also raises another issue, which is going back to your discussion about actions can reveal intentions.

Right.

If you look at what the cyber actors are doing right now, there were some in the community who hoped that maybe they would back off a little bit. Well, the criminals are doing what criminals do and they're getting more aggressive.

Yes.

And they're doing, you know, cyber fraud, regular fraud. I do DDoS attacks against organizations that are trying to make vaccines.

I know, so counterintuitive.

But they're just attacking and attacking, attacking. So to me, you know, your point about actions show you something about intent. This is their intention forever.

I think that's a really good point. Right. So they're sort of malevolent, destructive motivations, you know, have become evident.

Yeah.

Cool.

Carmen. I want to switch in just the final couple of questions, please, if that's all right. Ask about you and your sources of information. Any advice you have for, you know, what I should be reading or what our other readers should be reading or viewing or thinking about good books or good article sources.

Yeah, yeah, I tease. I don't tease, actually, but people think I'm teasing when I say that Twitter is my knowledge strategy. And I really value that stream of random information that I get through Twitter. And what it does is direct me to sources of information that are more likely to have unusual, offbeat early reports on important developments. I don't know if you read about how the. I think it was the health minister of Taiwan, and Taiwan's done a great job on the pandemic. How they first got wind of it through a Reddit post.

No, I didn't see that.

Yeah. And then the next day they said, okay, something's going on and we need to start doing getting prepared. Well, I, you know, don't do Reddit a lot, but I do look for quirky and different things. I think it's okay to follow the mainstream media, but they are always three or four days behind the times on important stories. Just recently I came across a resource, a scientist type, I think, who's blogging on a science platform, who's giving kind of day by day reporting on vaccine developments. Very technical, difficult for me to make sense of, but I know that this is really ground truth in a way that the diluted stuff I get through the media is not. So I like to look for the offbeat. And I think that I don't have a job with any organization, so I can spend 100% of my time looking for offbeat information. But I think that anybody should say, okay, 20, 30% of my time. I'm gonna go off the beaten track. What I often do is I. When I find an interesting article from one source, I put it on my reading list and I just keep going back to it on a regular basis to see what other new and different things they're reporting.

Great. Well, Carmen, I really appreciate your context on that and everything else.

Sure.

Great. Well, can we do this again sometime?

You bet. Absolutely. Bob, it's good talking to you.

All right. Good talking to you. I'll see you.

Stay safe. Thanks for listening to this. Ooda Loop production For the latest analysis on cybersecurity, technology and global risks, please visit www.ooda loop.com. Sa.