Unspoken Security Ep 60: Why Are We Trusting AI with Intelligence?

AI needs to be a tool to help us like plus up what we're doing, to enhance our collection capabilities, enhance our vetting capabilities.

Welcome to Unspoken Security, the raw and gritty podcast for security professionals who are looking to understand the most important issues related to making the world a safer place, including intelligence-driven security, risks and threats in the digital and physical world, and discussions related to corporate culture, leadership, and how world events impact all of us on and off our keyboards. In each episode, host AJ Nash engages with a range of industry experts to dissect current trends, share practical insights, and address the blunt truths surrounding all aspects of the security industry. Legends are made. Hello and welcome to another episode of Unspoken Security. I'm your host, AJ Nash. I spent 19 years in the intelligence community, mostly at NSA, and I've been building and maturing intelligence programs in the private sector for about 10 years now. I'm passionate about intelligence, security, public speaking, mentoring, and teaching, and I have a master's degree in organizational leadership from Gonzaga University. So I continue to be deeply committed to servant leadership. Now this podcast brings all these elements together with some incredible guests to have authentic, unfiltered conversations on a wide range of challenging topics. It's not your typical polished podcast. My dogs make the occasional appearance. People argue here. We even swear, at least I certainly do. And that's all okay. I want you to think of this podcast as a conversation you'd overhear at a bar after a long day at one of the larger cybersecurity conferences that we all find our way to. These are the conversations we usually have when nobody's listening. Now today I have an exciting guest, a good friend of mine, Faith McGregor. She's an intelligence guru and author currently serving as senior manager for threat intelligence at the Cigna Group. Built her intelligence career over 14+ years in the US Army National Guard, followed by roles in intelligence, governance, and strategy, uh, US Bank, Atomic Data, Wells Fargo, and Cargill. Now beyond that, and perhaps more interesting, uh, Faith is an award-winning author of historical fiction and historical fiction fantasy. Anything you want to add to that bio, Faith?

No, I think you covered it pretty good.

Well, we're off to a good start then. So, uh, listen, I want to jump right in. We have a really interesting discussion today. Both of us have obviously intelligence backgrounds, military backgrounds, and we both live in the modern world of where things are going. And all of a sudden, AI is just everywhere. When I left the government to the private sector, my joke to people was that everything apparently had become intelligence. The label was slapped on everything. It didn't matter anymore. Data and information was all intelligence, and people were network intelligence analysts and security intelligence professionals and business intelligence analysts. And I think my dry cleaner was a dry cleaning intelligence specialist. And so, uh, now it seems like AI is the thing. It's just getting shoved into everything. It's the new hotness, right? So today I want to talk about the future of intelligence in the age of AI. So now before we go too far, I think most of the audience probably has a pretty good understanding of what intelligence is, so we won't belabor the point, but for those who might not, we should quickly at least talk about what the foundations of intelligence are, starting with the intelligence cycle, right? So we've got planning, collection, processing, analysis and production, and distribution. Can you talk a little bit about some of the standards of tradecraft that go into intelligence, some of our processes, so people know the difference between data, information, intelligence, things like that?

The community will argue, but I mean, at its core, data or information is what we used to say in like traditional intelligence. It's just information. Information is just data. It's not corroborated. It's not fact-checked. It's not, you know, what we'd call backstopped or an operational test. It's something that we do in human intelligence where someone may tell you something And then you're like, well, is that true? How do I know? Well, someone told me it was true. Well, I want some extra independent sources to verify, usually not the same source. So usually to get to intelligence, to turn data or information into intelligence, it's vetted, it's cross-checked, it's information that has been verified through multiple unique sources. So if a human source tells me one thing, I wanna see a satellite image that also backs up what they're saying. And then maybe a cell phone interception or something like that, that triple corroborates that. Then we can start to say, okay, this is now verified information. It has been tested and checked, and now we can call it intelligence. Now we can make a decision off of it. And that's really what— if I wanted to sum it all up, like, intelligence is information that you can make a decision about.

The purpose of intelligence is informed decision-making, right? And I think you made excellent points there, and that Data is just raw, right? It's a point of context, perhaps. I used to talk about in technical terms, a list of IP addresses is data. A list of IP addresses with some context associated with it is information, which is usually being sold in feeds as intelligence anyway, but really that's information. And to get to intelligence, you have to be able to validate it and say, well, this is the list of IP addresses. This is the threat actors associated with it. These are the actions that have been taken on this infrastructure. These are the malwares associated with it and the signatures. So some context to it, and maybe even this is what we, might expect to see next, right? That much deeper picture. And as you said, through multiple sources, right? Not a single source. I didn't just grab a thing and call it intel. And it seems like that's been a real struggle in the private sector, partially because I think it's been a while to get enough people with longstanding intelligence training and tradecraft and experience to come out of the government, which is where almost everybody comes from originally. But then also the marketing, uh, you know, vendors have turned a lot of things into intelligence that aren't, because intelligence sells much easier than telling people it's data, right? So it's more expensive to tell them it's intelligence. And I think that's caused some issues from what I've seen. And then we talk about standards and tradecraft, right? In the intelligence community, we have these very rigid structures that everybody's trained on. Intelligence community directives, ICD 203, 206, 208, things like that. The private sector doesn't really have that. Uh, I know some experts who've come out of the government, myself included, I guess, have tried to push that forward. Uh, what are your thoughts on where we are in terms of maturity of tradecraft and consistency of messaging in the private sector when we talk about intelligence?

There are enough people that are former government or former agency people or civilian public sector experience. They've got good classic intelligence training and like my current team, we got a good mix of some who have that experience and some who have worked for civilian agencies, some DIA folks and the like. And I think the biggest challenge in private sector is the speed. I think what no one likes to admit is that intelligence is a process and you can't skip any of the process. So sometimes intelligence is slow. And my last part of my career, I did human intelligence counterintelligence, the slowest intelligence that there is, right? Humans is so slow and no commander wants to be like, do you have the answer yet? So, well, I've been nurturing a source and doing this, or if you're doing private investigation, well, I've been doing this investigation and I haven't found the gold nugget yet, but it's I'm confident we'll get there. And I think sometimes that, that kind of waters down the intelligence in the civilian sector. Like when we look for it, we usually pull articles, right? Hey, CrowdStrike article and this. And if you read those articles or some, you know, tech blog or something, and if you read them carefully, they're all citing like the same source. Hey, there's this blog and we're citing the IBM cost of data breach 2025. And then you read this other source and it's like, You go down to the fine print and like, oh, this is IBM cost of a data breach 2025. So they're not independent sources. They're all just quoting IBM. So really you only have IBM's perspective. And it's hard to find another group since everyone's doing their own thing. It's very hard to go and find a second source that's looking at the same thing in the same way. Because everyone's giving you, especially in cyber. They're giving you different statistics in a slightly different way. Like, how are they actually measuring it? You don't know. And without being able to see the mathematics behind it and see the algorithms and how they came up with that, it's hard to do source validation. It's really hard, which is what turns information into intelligence.

You made a good point. Circular reporting is another thing. Intelligence community, we look for, right? It's something looks like it's 5 or 6 sources, but as you said, you dig in, you're like, no, it's all the same source. Like, it's just Bob told Susie, Susie told Mary, Mary told Sally, but it all can't start with Bob, right? It was just the rumor from the one one person. Having other people repeat the same story from the same original source doesn't validate it. And we lose track of that a lot in the private sector, I think, uh, again, because you see the blogs and the journals and vendors quoting others, et cetera. And it's like, well, no, uh, you know, IBM said this and then CrowdStrike said this and then FireEye said this and then Flashpoint said this. You're like, yeah, but they all said the thing IBM originally said, as you pointed out. They didn't, unless they were adding their own findings or validating in some way. And there's nothing wrong with quoting other sources, of course. It just needs to be understood that you still just have the one source. You can't say because other vendors quoted it that somehow it has more legitimacy unless they're going to say they also were involved in the research or they validated it somehow. But you do see that happen. We see a lot of things that are being sold or consumed that are these intel feeds and you dig in and you're like, ah, it's not really intelligence, it's data or it's information or even it's just news sometimes. News with some perspective put on it is useful and can be. I think it's a challenge sometimes where people don't know how to accurately calculate the true value of that, right? A news story about a threat from a foreign nation is news. If you have a senior intel person on your team whose career has been dedicated to studying that nation and they can add context based on past knowledge of cultural differences or sounds like this, but when they say this in that culture, it really means a different thing and has to be interpreted a different way.

Okay.

You've added something to it that has value that is specific and might be getting you closer to an intelligence piece because of that person's deep research and deep knowledge. But otherwise, if you have a team that's just taking a bunch of headlines and putting them into one product and sending them to you, they're not sending you intelligence. They're just sending you a nicely packaged news feed that's curated to your needs, which is useful, but not intelligence. I think the difference between data information intelligence and the news, I've spent a lot of time talking about over the years, and it still ends up being very frustrating. And I think you and I share a frustration, which most of the world doesn't appreciate, just in the terminology. Cyber threat intelligence is a term that anybody who knows me knows I hate. I will use it occasionally because you have to, because that's where the market's gone, but it's, I don't particularly care for it. It's intelligence. Cyber is a subset of it. Threat is a subset, but these are not their own things. So cyber threat intelligence, you know, I get that it's become a normal term, but to me it's all intelligence. Then you break it down into its subcomponents. And I know if I'm not mistaken, you feel similarly about this. You wanna talk a little bit about how you see intelligence and what the subcomponents really should be feeding into it?

I, yes, we 100% agree on this one. Yeah. I traditionally intelligence, if you're not an intelligence professional, like intelligence is just, It's a big umbrella term. And within that you have things like, well, most people have heard of OSINT, open source intelligence, human, human intelligence, CI, counterintelligence. But there's like lesser known ones like MASINT, mass spectrum analysis, ELINT, electronic intelligence, which kind of morphed into the modern cyber, at least on the public side, sort of, or part of it. A little bit of that in communications. But yeah, like I remember when I was doing my first master's degree in intelligence, I was writing about, or I wanted to write about like intelligence operations and building an intelligence program and all that kind of thing. They had the term in some of those, this was like, I don't know, 10, 12 years ago now, but I had this term cybint, cyber intelligence, which is what we use, not CTI. And it's just another discipline. And really what we need is probably more of a, I don't know, like a higher level intelligence team within private sectors that do multiple disciplines. And we do it like I'm cyber threat intelligence, but do I report on geopolitics sometimes? Of course we do. Especially when it's relevant to me what a cyber threat actor would do. But we have people who live in different parts of the world and I can get information from them. Like, hey, this happened outside my window this morning. You know, that's human intelligence. I'm vetting that source. Or if you had a fraudulent worker and your HR was unsure about it, they can bring them to the intelligence team. We can not only just do the technical stuff under the hood to see what was going on, but we can interview what we'd call an intelligence debrief that HR person and be like, well, and we might be able to pull details from that person that would give us more context and help back up the, you know, the hard data that we have, giving us another source. And we just don't do that a lot because cyber is so fast-paced, which we talked about before, and we're constantly responding to things. And, you know, we don't have 17 intelligence agencies to help us. We have most programs, I'm better under 100 people. I mean, like, whole cybersecurity programs, not Intel. Most Intel programs are usually 2 to 3 people.

Very underfunded and undersupported, right? So you're only a handful of people doing a little bit of everything, right? I mean, Intel team that works with an insider threat team, I mean, you can call that counterintelligence. It's the same concept, right? We do signals intelligence. We do OSINT, as you said, right? So open source intelligence. We're really— most of us are just all-source intel. Professionals at some level pivoting from one source to the next to try to get a holistic picture, which is another reason why I've always advocated for just saying you have an intelligence team, you know, and the rest are pieces of that puzzle. Anyway, I don't have to get on that soapbox. Anybody who knows me knows about my thinking on having a CIO, the Chief Intelligence Officer, at the C-level and having everything roll into that. So we won't repeat all those things for those who get bored hearing me talk about it.

I do love that idea. I just want to publicly go on the record saying the Chief Intelligence idea is probably where we need to be at some point if we really want this to mature.

I agree, of course, and I appreciate that. I know some people are working on that. So, so that gives us our baseline for those who are just catching up. And for those who are intel professionals, you probably skipped all of what we just had there and now you're like, okay, good. Can we get onto the next piece? Right? So if you jumped ahead, good for you. If you needed to hear some of this to understand the baseline of intel, that's awesome. So now we could jump into where we're really going, right? Which was the discussion about AI and how that fits into this. So, You know, a lot has changed just in the past, changes every day it seems like, but certainly in the past year or so in the AI arena, right? We've actually had ChatGPT launched about 2 years ago now, I think 2 or 3 even. So AI has been out for a while and people have used it for varying things and it continues to grow and expand. We've gotten into agentic AI and now we're seeing these mass layoffs across industry as people are being replaced by AI or theoretically replaced. We'll see how that works out. And so let's talk about specifically the intelligence zone. Now, keeping in mind, we said the purpose of intelligence is inform, to help leadership with inform informed decision-making. These are critical pieces that we are providing, right? Leadership's going to make strategic decisions. Sometimes it's shifting entire corporate structure. Sometimes it's how you handle your security in acute manners or in chronic manners. These are generally pretty impactful decisions that are going to be made based on intelligence. So, what do you see right now as risks and rewards of this exponential increase in the use of AI, specifically in intelligence work?

Yeah, I think Honestly, I think the final answer is still like, to be determined. TBD is my short answer. True. Seeing how it works. There are changes coming in really fast. Mythos has scared people into, and maybe rightly, like a little healthy fear of what threat actors can do is not a bad thing. I think when it comes to intelligence though, like there's really, as we talked about, you know, sources and articles. Quoting the same source like IBM, how do we know that AI is using vetted information that is coming to those conclusions with not just an opinion that it scraped on the internet or an article that is subjective? What sources are they plugged into? If you don't have access to classified sources, like how do you know that that information has been vetted? So you get an AI report or you ask AI to give you a little brief or a summary of some event. Like, how do you know that all that stuff is correct, right? Because the most important thing we can do in intelligence is validate our sources and make sure things are correct. And I will tell you, like, I have used some really top-of-the-line AI programs within vendors, and the summary is not— what it says in the summary is not backed up. And then you go and look at the sources, and it's not quite what you thought it was, even though all the pieces are there. That AI made the leap and then that kind of pushes the analyst to make a leap. And then you're under a tight timeline, so you make, you make a like, hey, I think this is where it's going, but you don't really know. And kind of a follow-on to that is in the corporate world, you don't get the time to say, well, I think this is what's happening, but give me like 6 weeks and I'll get back to you and let you know for sure. That's never gonna be an acceptable answer, right? So, you're forced to make snap judgment calls pretty quickly, and that can be stressful. It can be hard to really put your stamp on that information. And we've seen examples like the recent— my favorite one to quote is the EY report that came out. Oh, and they had to recall it because several of the conclusions in that report from AI were incorrect. I think it was like 40% or something of them. So not like the entire thing was wrong, and that's a company with good funding. They put out a lot of good reports before, and they charge a lot for them, and they charge a lot for them. So if something like that can't be trusted— so I think it comes down to trust, right? We talk all the time about zero trust. Well, why are we trusting AI? Like, what— I'm not saying we're not— AI is not going to go away. So how do we put something in place, a system, controls, whatever you want to call it, to ensure that that information is valid and trusted, can be trusted from a trusted source. And I don't think the term human in the loop, I don't think that's enough when it comes to intelligence. Maybe for logging easy data or low-level SOC stuff like, uh, that's for complex critical thinking, decision-making that is gonna affect budgets and businesses and jobs. I don't think it's there.

I think you're exactly right. Some of the tenets of intelligence are timely, accurate, and relevant. There's actually 7. Complete would be another one. I think those are the 4 I talk about the most. As we start having more AI involved, timely seems to be the piece that everybody cares most about, right? Oh, it's very, very fast. How accurate is it? How relevant is the research you're getting? And is it complete? Now, those are much harder things to come up with. Even if you work with good AI platforms and you've put in the effort, you've really worked on your prompt engineering and all this, it's always a challenge. It's a challenge even for humans to say, do I have everything? I have everything relevant, at least within a timeframe. Normally we have what's known as information cutoff, as I can't wait forever to report something. I decide I've gotten as much as I can get from this timeline and go forward. But with AI, you don't really know. AI goes and does its thing and it comes back and you have your sources and it's like, well, did it really get everything that was possible for this? It almost never does. It's going to cut off at a point and say, this is good enough to validate the question. And it may not be, there might be more out there. Then you get into that accuracy and you made an excellent point. Even if you curate all the sources. If you feed an AI platform just sources you've pulled, I pulled 8 products, they're all valid and vetted and I think they're great. Just gimme a summary. There's still a reasonable chance most of these platforms will hype, will come up with things that aren't part of that concept or aren't part of that structure of data that you've given it. It does that sometimes. We haven't been able to figure out the why. Technology's working on it. The platform creators have said they don't completely understand their own technology. They're honest about it. And yet we're going, well, no, we're in a hurry. We gotta do it. We gotta run. And like you said, there's not enough time to validate. Everybody's in a hurry. Everything's got to be fast because timeliness seems to matter more than anything, which is incredibly dangerous when we're talking about things that are going to be massive decisions that are going to be made off of this. And then it puts so much pressure on people that you maybe don't have enough time to validate it, or maybe you get lazy. Humans are designed to be efficient. So I've been working on this platform for a while and I've validated all the sources on these 28 products and everyone's been perfect. The 29th product's the one where I go, well, you know what? I validated a couple. It probably is good because I'm in a hurry and I've got somebody chewing on my neck. Ship this out. And then the next one, you validate less and you validate less. And suddenly you're just shipping things directly out, not validating anymore. And that's how people operate. And you're going to make massive decisions as a company based on things you don't know if they're even close to accurate necessarily. The EY one's a really good example. It's shameful, frankly, for EY, the amount of money they charge and get paid and the brand that they have. You didn't put out 1 or 2 mistakes. Like you said, it was like 40% of the product was just flawed, just inherently flawed. Which means they've offloaded an awful lot to AI. Human in the loop can't fix that fast enough. You can't go back and check everything and fix it all when it's that flawed. And it ends up going out the door and they charged a lot for it, probably, for it to pull it back. And yet we're seeing more and more of this. People are just pushing forward and saying, no, we're going to, we're going to, AI can do it. We don't need people. And keep in mind, there's no intelligence. AI is again, a brand. It's marketing. It's not intelligent. It's not sentient. It can't reason, at least not yet., you know, it is amazing technology, does a lot of amazing things, but it is not intelligence. You know, if you had an intelligence analyst that every once in a while just wrote a report and put it out that was based on nothing, or just added some fiction to it, you know, most of it was good, but just added a few lines that were just completely imaginary, you'd fire that person the first time it happened. It gets— they get a lot of talking to, and then they have to explain how they did it. And if they couldn't, by the way, AI won't be able to. It'll just say, I don't know why I did that. You'd fire that person. You can't trust them with the content you're going to use to make massive, massive impact decisions. And yet we're just shuffling more and more people out the door in place of this technology that we don't understand. So listen, I also don't want it to be all negative, right? Some good things are happening with AI, obviously. And there are some really valid ways to use this really powerful technology. So let's talk a little bit about that. What do you see as some of the positive ways that AI is impacting intelligence analysts and ways people are being able to use it to do a better job? Companies that are doing a good job of integrating it. What are you seeing as some of the positives right now with AI when it comes to intelligence work?

Yeah, and there is some good news. Like back in the old days, we used to have this kind of reachback capability where if we were deployed, we could reach back to a unit in the United States and get information. But usually to get that, you had to kind of draft a really complex query or a query letter and get them like, this is what I'm looking for. Do you have this information? And some humans would go and find it. And then as we got better, we started having computers do that. And then you'd have to write like basically simple code or SQL language and write a simple little query to pull the information you need. So one way I see this working great is with Recorded Future. They, when they first did this, they were one of the first intel vendors to adopt like a GPT model. And so what they do is they have their insec group of human analysts that curate reports and do all the source validation and stuff like that. Classic intel reports. And then they put it into their system. And then you can use their Record of Future AI to find those reports. So then you're doing a kind of an advanced query to find information, and it's not just pulling data, it's pulling validated source reports, validated intelligence reports, what we'd call finished intelligence, right? So now I can make, I can put those in my assessments and I can make that based off of like good conclusions. And then they have their sources too that have been independently verified, hopefully. Now, The Record of Future still does like kind of the classic, you can build a report with AI and stuff like, everybody's doing that. But I do think that's like a great use of intelligence, artificial intelligence in the intel sector is kind of curating reports. We have, cuz we have all this data and sometimes we've talked about analysis paralysis. We just have too much data. Mm-hmm. How can we possibly go through it all? But AI can help us sort that. AI can help us sort through. Find me the one IP that was masking its, or find, can you verify this? Where was this being logged in from? What host country? And maybe AI can actually help us get through some of those ways that threat actors mask where they are.

Yeah. Well, you make a good point about just, we're swimming in data and information and we have to consume so much, right? And if you're working in a mature organization, it's even more, right? It's nice to have funding and resources., and also comes with the burden of more, right? I got all these vendors and all this open source and I've, we've got all these collection systems and it's just an overwhelming amount of content coming at us. We rarely have enough people still. So yeah, I agree with you that when tuned properly, and that's a key piece, I accidentally have become a bit of a prompt engineer. If you play with technologies, you either learn how to use them or you struggle. And so you learn over time how to ask the right questions to AI to get better answers. And the platforms change and they get better too. But, you know, I think one of the things you talked about with that paralysis component, a lot of times a challenge I've seen in the Intel space, myself included, is you just, you keep going down rabbit holes deeper and digging. And that's why we have information cutoff to force people to stop researching and actually start writing. And then sometimes just getting to writing, it's hard. How do I, how do I want to structure all this? How do I put it together? How do I just put pen to paper, digital now, but to get things going? And I find that AI is great for drafts. If nothing else, just get you started to say, okay, now I've got a draft to work with. Sometimes it's easier to edit something than it is to author something originally. Now, of course, you got to assume everything that came in that draft is flawed, just for the sake of being safe about things. Just assume it's all going to be flawed, but at least you have a structure to work with. Then you start checking the sources and validating and moving some things around. AI doesn't do confidence language at all, as far as I'm concerned. Not well. It'll throw things in there, high confidence, low confidence, but it's just making shit up. It doesn't understand the intelligence community directives. It's not applying standards. So you're going to have to go through and do those things. But I've always found that some of the summarization I think you mentioned is good. Some of the drafting can be really helpful. If you program it properly, I feel like it can do a lot of really good research. Like you said, it's nice if you have a corpus that's already validated content. So at least you don't have, you don't have it out there making stuff up, but even some of the open source, if you're able to put a lot of time and effort into building the right models to really limit what sources it can go to and force it to validate the URLs, and then you still have to check them all anyway. I have found that it can be really helpful as a research assistant. And you can even get into time parameters and saying from this time to this time, kind of create information cutoff and give it a wider range. And at some point you may just have to accept that I hope this is complete, but again, timelines are timelines. I think that there's been some improvements there, but for those who think, hey, I'm just going to hand it to the AI, here's my tasking and here's what I need you to do. Go do the whole thing and write it up. And that's going to be an intel report. I haven't seen any that I would trust certainly to make decisions. I don't know, maybe you have, maybe you see things I don't. Have you seen anybody yet that could just have the easy button and here's your answer and let's go make strategic decisions based on what AI tells us to do?

Not at all. But I will say I've seen people make decisions off of a news story or a headline, and we're making decisions off of a headline and it's like a, I don't know, there's probably really good examples. Maybe I shouldn't go into all of them. Yeah. News correlation is a big part of what Intel does in the corporate sector. Is, is this news item, is this a threat to us? Is this, does this matter? I think AI is. Nice to pull in some quick information and then you have to go and validate it though. But you know, my, my bigger concern is if the AI solution is really expensive, cuz we're cost benefit analysis, right? Mm-hmm. If the AI costs 10 times more than a human analyst, but it's really good, but then you have to have a human analyst go over every single thing to do it and you could have would another second analyst be cheaper than an AI solution? I think there needs to be some, like AI, like any new technology, it needs to come down in cost. It needs to come down in, it needs to get a little more user friendly. It's pretty user friendly, but it's gotta have some more controls and parameters around it. Or you can, I almost would love to see like, and here for all the AI people out there who are making this stuff, who are smarter than me, give me some controls. Could I take just for intelligence? Could I tune my AI and say, I'm only looking for this or this, but tune it with intelligence parameters? Yeah, I only want something that is within this amount that has been validated by two or more sources. And, and then how is the AI gonna check that it's been validated by two or more sources, independent sources? But if there's ways I can do that, and I know there's people are probably working on that already thinking about it, but. You know?

Yeah, I don't know if there's enough demand yet is part of the issue, right? Because I just don't think there's enough people still that understand the difference between data, information, and intelligence, as we opened, to be able to, to levy those kind of requirements. I, you know, for what it's worth personally, I've been playing around a little bit with trying to create, okay, search parameters and creating collection tiers of collection, right? These are tier 1 sources, these are tier 2, these are tier 3, to try to teach AI confidence language. If it's got at least 3 tier 1 sources, then we can say it's got moderate confidence or better and start working through that. Hasn't been successful. But at least I've been successful in getting it to limit itself to sources. So that's helped me. If I can just say, hey, listen, here's the 20 sources or whatever, these are the only sources you can use, then at least I know what's coming in. As long as they're valid, I gotta go check, make sure it didn't hallucinate, just make up URLs. At least I know it's coming from sources that I already trust, that I've already vetted, you know, which can be helpful. It's a bit of a shortcut through it. But yeah, I haven't seen anybody with the easy button yet. They'll sell it that way. And there's plenty of people that think that's the answer and people are going to be selling it. AI intelligence, you don't need an intel team. For anybody listening, don't, don't buy that. Don't make decisions based on that intelligence. It would be a terrible move. You know, I think this overreliance on outsourcing because there's a belief it's cheap, cheaper than people. It's not, as you said, it's, we need the technology to get less expensive and get a little bit better, like all technologies. And what we're hearing right now is this technology is about to get a lot more expensive. It's sort of been subsidized so far. And now that they've gotten into these corporate environments and convinced people that this is going to save them a lot of money and they've been offloading a lot of people and moving things over to AI. These AI platforms are helping companies become very dependent on what they offer. And all of a sudden they're about to get put on the elevator for price. And companies are already starting to say, well, wait a minute now, our tokens are expiring so quickly. And people's tokens don't expire. People on salary, yeah, you can't work them hundreds of hours a week, but you can work them more than 40 and 50 a lot of times. And people don't really have expiring tokens. And the AI platforms do. And nobody knows what platform AI tokens actually cost. And that's, so that's changing. So I think it's interesting. To see where this is going in terms of maybe too much overreliance on some of these things and too much trust. People have tendencies to trust machines that act like people more than they trust the people. And so it sounds like from what we're saying is there's a mixed bag here, right? There's some good possibilities and good opportunities, but this technology is not everything it's being sold to be, at least when it comes to intelligence. And again, if intelligence is used to make decisions, it's important that those decisions are based on the best available content, you know, vetted, validated intelligence, not just whatever's in the news cycle. Like you said, some people are still doing it just based off news. And I have found that, I don't know, I imagine you've seen the same thing. A lot of my job when I've worked in organizations in intel was just helping people, specifically the CISO, understand what not to worry about. Before you even get into the discussion about they need to worry about, it seems like every morning was explaining why this news story about is not something you have to worry too much about. We don't have to make a lot of reactions. Did you experience, or have you experienced a lot of that on the intel side too?

A little bit, I guess. I don't know. I think we keep talking, we keep getting back to this trust. Like we keep bringing up the word trust, right? And trust is, if you don't have trust, you're going to fail in intelligence. And, but intel people are human and they're going to make mistakes. They, I was in Iraq once and it wasn't me, it was a different unit and someone made an assessment that someone was going to attack the base in Kuwait. Highly unlikely to anybody who understood enough, but they saw the information and they're like, this could be the thing. Now after that, you make that, you make a mistake like that and the human is going to be like, okay, so the next time I go into this again, I'm going to quadruple verify my sources and then I'm going to ask 3 friends to validate this and then I'm going to call another buddy. In Singapore and be like, hey, are you seeing what I'm seeing? Your next product is gonna be, and then every time you make a mistake because intelligence is not perfect, you're gonna get better. And the AI is just going to, like a program or anything like that, it's just gonna, it's just, it's programming didn't change because it got something wrong. Like not yet.

Yeah.

We're, we're not at live learning, whereas a human analyst learns. And also I think someone show this to me and prove that it works, but AI is sorting related information that makes sense. What about unrelated information? The little reserve— you remember the story about the reservist found Saddam Hussein? He was looking through reports that people just weren't even trained intelligence professionals, just completely ignored unrelated, uncorroborated stuff. But he saw something, he saw some patterns, and he took a giant leap and maybe it wasn't a giant leap. The information was there, right? It was there. We talked about 9/11, like the information was there, but could we— it wasn't shared, but then we had the information and it's— yeah. So how do we parse that using something like AI?

Like, yeah, I mean, I don't know the answer. No, it's a really good point. The AI, the learning thing, I think is a really good point, right? Everybody who's done intel for any period of time has gotten something wrong. I mean, that's, that's the nature of intelligence. Like you said, we're not perfect. It's not 100%. There's a reason we have caveat languages. There's a reason that very few people will say high confidence. Nobody I've ever known says this is certain to happen. There's a— I've never seen that happen in Intel. High confidence still has to have a lot of sources and a lot of support, and it doesn't mean it will necessarily happen. Still, confidence language is something we live by and we work hard at it, but we've all gotten something wrong, and it's hard, and you take it personally, frankly, if you're an Intel professional, and it's you you have a lot of introspective discussion with yourself, frankly, about, all right, what do I do differently? How do I fix this? Sometimes, depending how bad it is, you might have gotten a lot of external feedback on how to do things better and how to fix it. And that affects us as humans, right? We either, as a dedicated professional who wants to get better, or maybe just out of fear, depending on what it is— there are consequences for getting things wrong. Even though everybody understands we're not perfect, we're not predicting the future, we don't have a crystal ball— there can be consequences. AI doesn't have any of that. There are no consequences for getting things wrong. Most of these platforms don't remember one project to the next unless they're designed to do so, and they don't really learn. So it can in fact make the same mistake. You can teach it, and to an extent there's memory, it can be created. There's a lot that goes into it, but it doesn't work like humans yet, as you said, and their motivations aren't the same. So, you know, that's a difference. And again, so if you're dealing with somebody who's been, you know, if you're hiring somebody who's got 10, 15, 20 years of intelligence, they have all those battle scars, they have all that iterative learning. It's ingrained, it's built in, it's happened over time in ways that can't even be described through resumes and interviews. The AI doesn't, doesn't have that. It doesn't have the same kind of dedication. It doesn't understand consequences of its actions or consequences of somebody taking action based on its judgments. It, we haven't advanced to that point yet. So, so I, I think those are really good points you've made that it's a concern, right? We will get there perhaps, but we're certainly not yet. And it's dangerous to start running down that path because it also means if the AI makes a terrible mistake that you make a terrible decision based on, it'll do it again. Like, given the same opportunity in the same context, there's a chance it'll make the same exact analytic error or the same exact leap or worse. You know, again, AI is notorious for— they call it hallucinations. I would call it fabrications, frankly, where it just creates things to fill in gaps. You know, that's a— humans can do that, but human intelligence people specifically don't do that. We have to call out these are gaps. We don't have this information. We don't have this knowledge. Ask any AI platform a question. I've yet to have one say, I don't know. I don't know. I don't have an answer for that. I don't know. Humans will do that. Intelligence people will say, best research I've got, I really can't give you an honest, honestly, I can't give you a confident answer either way. AI will always give you a confident answer. And that is disturbing. Again, if you hired an intel analyst that was as confident and cocksure as AI is and immediately had spit out answers to you it was certain of, you'd fire that person. And so, so I'm worried about where that's going to be as we go forward. And I think that's kind of the big question is where do you think we are going forward with this. Again, we've talked about where it is today and where we think it's going to head. What do you see in the next, let's say, 5 years or so as, I don't know, best case and worst case scenarios as it comes to AI integration or taking over of intelligence? And I think maybe we can even talk about the government space. We're starting to hear about AI being used in government intelligence and classified work and planning and some of those things. What do you see as scenarios?

Well, I guess we'll do worst case first. I mean, there was just, I haven't independently verified this or done all my source verification, so it's just a news report, not intelligence report, caveat there, but that's, that the government is using kind of Claude-based models to help with target acquisition with kinetic strikes. And so now, like, I remember in the olden days, if our group got mortared in a base, like it got to the point where you could do all the triangulation and we could, we have the technology to be able to shoot back from where that round came from. And all that stuff, and then you needed like an intelligence officer and a lawyer standing next to that fire team saying, yeah, 66%. And then the lawyer will be like, I'm not comfortable with that. So like how, yeah. And when AI is making decisions so fast that you don't have time to like validate. And I think, I think we've talked about, we talked about at the very beginning of the podcast, time. Right? Sometimes you just have to slow down. And no matter how much you want it to go faster, you make a fast decision, that's how you make a bad decision. And I think teams need to get comfortable saying, "I need to get back to you," or "I need another day," or "The data just isn't there. We, we can't say this." I remember when I, one of my first assignments in combat arms military unit. Like where I grew up, one of the 194 Armor, Minnesota, shout out. The classic intelligence folks like always wanted to give a, like a percentage confidence level and stuff like that. And the combat commanders are like, I wanna know what you think, S2. And you're put on the spot, you're in front of a briefing, it's 100 people and it's like, what do you think's gonna happen there, S2? I don't know. Or, but they say, well, here's what's likely to happen. Here's what I think where this could go. I don't know that's right, but I have a theory that this could go this way, and I wanna put sources and collection assets on this to see if we can verify that. Right. And some of it's gotta be based on like real knowledge. And I think, and I think that can segue into the good parts, right? I think what could be good is not like blanket replacing stuff with AI and not human in the loop. It's gonna be, AI needs to be a tool to help us like plus up what we're doing to, to enhance our collection capabilities, enhance our vetting capabilities. Like it's like anything else. The Army had a system called the Baton Hide where we could do retinal scans and finger scans of people like on the spot and see if they showed up in a database. Had they been arrested before? Like instant, right? In the field. And it was fantastic because instead of, well, I'm gonna take this report back to someone, then I'm gonna send it back to the States, and 3 to 6 weeks later I'll get a report thing. And by then either the bad thing has already happened and you couldn't prevent it, or the information is useless. And so I think technology in general is great. Like it, it can be a way to, to plus up and kind of what we'd call like a force multiplier, right? Like it's not meant to be the thing. It's meant to be the thing that helps you. Like, air power is great, smart bombs are great, but at the end you always need the infantry to walk in and take the town. There is no substitute for, well, we bombed it all, it's all good now, everything's fine. Nope. It's, if you don't go in there and verify it, right? Yeah, we walked in and, oh, how many movies have we seen where they walk in and intel says it's fine and they walk in and they're like, not fine.

Right.

And there's an entire division of Germans behind the building and You know, then drama ensues, makes for great drama, but that's not what we're looking for in, you know, in operations. So like, I think we need to use it that way and it, there's gonna be good ways to use it. We just gotta, well, and you make a good point about the fear.

I, the big fear I have is that speed component, right? The people are in a hurry and you mentioned in the field, you may give the intel assessment that we have a moderate confidence assessment based on conflicting reporting that this may or may not happen. So we have low confidence that this is gonna happen, something like that. And then the combatant commander looks at you and says, just tell me what you think. And we're not doing that usually. Like, it's okay, we're going to go back to the same language. I can't just— this is what I think, right? What I think is caveated intentionally. If I just say I think this is going to happen or I think it's not, all the caveats went out the window. You're going to go based on my assessment, which is not the way actions should be taken. But I fear what's going to happen as we get more into the AI components is those combatant commanders are going to say, ah, the hell with the J2 or the S2. And for everybody who doesn't know, the 2 is intel, J3, S3, that's operations usually. The hell with him. He's so mealy-mouthed and he's wishy-washy. He's always got his caveats. Claude, what do you think is going to happen? Claude's going to say, I think this is going to happen. I love Claude. Let's just go with that. The AI is going to give you that confident answer to make a decision as opposed to the caveated answer that is the smarter and more thought-provoking and something that's been validated and gone through a process. I fear that we're going to see that this bias towards speed and confidence as opposed to being more prudent in our decision-making, which is obviously it's a problem in cybersecurity, but when you get into kinetic issues, it's a much bigger issue. If you have somebody who's in a hurry to make a decision and they're willing to go with the very confident words of a platform that may not have validated anything, doesn't understand how to make these kind of assessments and intelligence, and then you're gonna put bombs on targets. I've seen a couple times in my long career at this point, significant errors that resulted in significant harm. You go way back to the Balkans, this is public knowledge, the US accidentally bombed the Chinese embassy. There's a whole lot more story to that story that I won't go into because it's a whole show on its own. But ultimately, the official story is that the US maps were out of date and the agency responsible for that, their targeting was out of date. And so we hit a building that we didn't want to hit. And then most recently, we just hit a school full of children in Iran. And one might say, well, see, humans make a mistake. Balkans was human. The Iran one reportedly may have been AI helping with targeting.— but that— we've only had the AI thing for a short period of time, and that's one of the most catastrophic, colossal combat errors I can recall. Now, again, not validated. Don't know if AI was involved in that targeting. I'm just trying to make the point here that it's— that's what we're hearing in the reporting. And bombs on targets have very significant consequences, not just life and limb, which obviously is significant, but further than that. If you blow up a school, you blow up a hospital, you blow up innocent civilians anywhere, accident or on purpose, it creates longstanding Problems, international relations problems. People don't forget these things and they don't care that it was a mistake. You know, death came from the sky. And I fear that that's what we're going to run into is, like you said, the combatant commanders, I just want to know what you think. I just want to go make a decision. I don't need all the caveats. I don't want to think through this stuff. Tell me what to do. And I feel like the technology is much more happy to do that because again, it doesn't understand consequences. So I worry at this rush to put this into places where significant consequences happen, specifically intelligence in the government space and combat arms and law enforcement, for instance. And where that goes. So I'm hopeful we don't go too fast in that space. It seems like we're driving there in a hurry now. It may be up to the platform providers themselves to help reel that in, but that's not in their best interest business-wise. So I think that's something we're gonna have to continue to watch and see. I agree with you, this technology has great possibilities for us and can make Intel people smarter and faster and better as a tool, but not a replacement for all of the things we currently do. The technologies aren't even close to being there. Now, that being said, we do have to wrap up. We're actually a little long on time. It's been a good topic and a good episode, and we probably could have kept talking, but I think— I don't know that we made any conclusions on here other than AI is really cool and does some cool things and also is really dangerous, could be a big problem. And I'd say don't trust it too much, but, you know, think of it as a tool instead of replacement. But I do want to wrap up, and we always have the same question we close with, with all of our guests. The name of the show is Unspoken Security. With that in mind, you don't get any pass just because you're a friend. What have I want to hear from you something that in your life that to this point you've never told anybody before, something that so far has been unspoken. Hmm.

On the spot. Yes. Well, intelligence professionals never tell people things. There's always things that we are never going to tell anybody, but true.

Yeah. Nothing classified here, please.

Right, right, right. No, if I just think of something fun, light, and personal, like, yeah, all this work doing intelligence is it's a, it's a 24/7 job, right? You're always doing it. You can be Sunday morning, you're sleeping in, and then you look, open your phone and you see a thing and you're like, see a certain news story and you're like, oh, I'm gonna have to make a report on that later. So you don't just pass it away. You open it up and then you start reading more. And then pretty soon 3 hours of your Sunday is gone and your, your bosses have no idea that you've done that. But that's, that's just the life of intel. You know, I always thought I was gonna be a inner city girl forever, and I live in St. Paul and have my technology and all that stuff. But the more and more I go through this, I'm getting closer and closer to desiring to buy a small farm, like far away from people and just, I don't know, raise sheep and plant fruit trees and, oh, maybe just kind of be a retired spy for a little while.

Well, I think, I think you're right. By the way, it's validating to hear somebody else say the same thing that I do all the time. Like, it's— I go on rabbit holes constantly. You pick up the phone, you think you're just going to read something light, next thing you know it's 4 hours later, like you said. And I realize I don't even know why I'm doing this. I don't even work for anybody that cares about this right now. I'm just— I went down this path because it becomes human nature as intel professionals, becomes our nature to want to know more about things. I want to dig in. Or you read some wild story like, no, that can't be real, and then you want to validate it or invalidate it, and you can start digging through all the details and And I've burned more hours and days than I care to count. But I think what you're saying, I've also heard a lot in our industry, both in the Intel side and just in tech in general, of people who are just saying, I think maybe I'm ready to not do this anymore, to just check out, to get a farm. I've heard many people talk about farming, whether it's agriculture or livestock, and just, you know, just get away, just unplug and be gone, which maybe says just how hard this is. I don't know if people realize how hard this is. Industry is and how hard this work is if you do it for decades. I think a lot of people define hard work as working in a factory or working in the mines or really heavy labor, which is incredibly hard work. And by the way, so is farming, all very hard work. But I think people underestimate how hard it is sometimes to do some of our work because you sit at a desk and you may drink fancy coffee and whatever, and there's good perks in some of these jobs. And I think people don't recognize or understand how hard it is mentally and how it doesn't ever shut off. You know, and people burn out. It's very difficult to do this for long periods of time.

Just in my circle, like, I think I could name off right off the top of my head, I know one intelligence professional became a welder, uh, one is, um, did move to a farm and they're just doing that. Like, and that's just within my little group. And I fear that AI is going to push people out and then like it Intel's not something you learn in a year. I've been doing this a long time and I still have a lot to learn. And there's just— and once that— once, once you get away from it, like, it, it takes so long to get back into it. And if we lay off all the people for AI and then we try to bring them back because it wasn't working, that resource isn't going to be there anymore. And then you're going to be talking about a huge risk. And so, yeah, I mean, we talked about that for years in cybersecurity, but specifically in intelligence, basically a 24/7 bot to do whatever you need to be doing, right? And burn people out and they're just going to be done, or they're going to be, I want to retire early, or I'm going to switch careers because I can't do this anymore, and I don't want to do this anymore, and nobody ever thanks me for what I'm doing anymore. And, you know, it's like you can do a thousand things right, but one, one little thing is off, and then it's like, you know, Yeah, people don't remember the victories in intelligence.

It's— there's a reason it's called— we're silent professionals. You do a lot of things every day that are very successful and people don't remember those. And it's fine that people focus on tragic things that have gone wrong. I should— there are things that need to be focused on. But you're right, it does feel thankless. And over years, that— there's an accumulation of that and just a feeling of, why am I trying anymore? Why am I doing this? You've done a lot of good, but you don't think about it because most intel people are pretty humble and they just move on to the next thing. And nobody sits down, does the calculation of how many lives weren't lost because of a thing. Because it's very hard to calculate prevention. When something bad happens, it's easy to calculate harm, but it's hard to calculate prevention very well. So the metrics are low either. So it's amazing how often you have people that are very good at this and have done amazing work, and you have leadership saying, I don't know why we're paying these people. Like, this bad thing happened, they didn't prevent it, why do we have them? And it's, it's very tiring after a while. So yeah, I think you're right. I think we're having some brain drain issues. And I think despite what some may think, you don't learn this in a year or two, and it takes time, and it takes energy and it takes experience, some good, some bad. And I think we are losing that. I think you're right. I think we may find ourselves in a position where we have a very hard time in the future if we keep pushing too many people out. If you kick the junior people out, nobody grows up to be mid and senior and seniors die eventually. So, and, or get tired and burn out. So I think you make a good point. I think it's interesting going back to what the topic was here, which is the unspoken thing that you were talking about getting a farm and, and moving away. So I hope that doesn't happen anytime soon cuz I really enjoy talking to you and hanging out and hopefully we can continue to do good things in this community.

No, I got, I got some more years left in me. I'm not ready. I'm not ready to turn it off yet.

Outstanding. Well, I appreciate you being here today. I'm sure everybody else did too. Thanks for taking the time to chat a little bit about intelligence and AI and where we are and where we're going and the good and the bad. And hopefully this was, this was thought-provoking and helpful for others. Any last things you want to drop before we close out today?

Just a shout out to all my fellow intelligence community. Find something outside of intelligence that is completely unrelated to make yourself happy. To turn your brain off. That's why I write historical fiction fantasy. You gotta have something that is not related to this, you know?

Yeah.

Keep yourself— keep yourself going. Yep. So you can come back fresh and solve the puzzles.

It's good advice. Yeah. Turn your brain off or at least turn it on something else for a while. All right. Well, this, that with that, we're gonna close it out again. Thank you very, very much for being here, Faith. Thanks for taking the time and having this really interesting conversation. I look forward to getting feedback from everybody. How others saw this and felt about it. And for everybody out there, again, thank you for taking the time for listening or watching, however you chose to, to tune in today. If you like the show, please provide feedback. If you don't like the show, okay, provide feedback too, cuz I still want to see what we can do to make it better. But please like and follow and share this with other people so we can continue doing this show and bring amazing guests with interesting ideas like Faith and others. So thank you all for tuning in. I appreciate it. With that, I'm gonna close this out. This has been another episode of Unspoken Security.

That's a wrap for this episode of Unspoken Security. If you enjoyed this episode, follow us wherever you listen to your favorite podcasts.

See you next time.