Cyber Criminals, Biased AI & Autonomous Rockets: Things That Go Bump in the Night

Foreign.

Welcome to Intellicast, powered by Scip. Welcome to another episode of Skip Intellicast, a podcast about strategy, intelligence and leadership. I'm Cam Maki with Skip, and today we'll be talking with Matt devost. Matt is CEO and co founder of OODA llc. He's a technologist, entrepreneur and international security expert specializing in counterterrorism, critical infrastructure protection, intelligence, risk management, and cybersecurity issues. Matt co founded the cybersecurity consultancy Fusion X and was previously president and CEO of the Terrorism Research Center. Welcome, Matt.

Hey, thanks for having me.

Yeah, really appreciate it. Now, let's maybe jump in. For those in our audience who are not familiar with ooda, the concept of OODA, and the decision support loop, you named your firm after it. So maybe talk a little bit about what OODA is and why you decided to name your firm after it.

Yeah, absolutely. It's got a great backstory. OODA as a concept stands for Observe, Orient, Decide, and Act. And it's a decision cycle that. That was articulated by an Air Force pilot named John Boyd. Interesting backstory with him in that he was a fighter pilot, fought in the Korean War, and then was a dogfight trainer and was undefeated. In fact, not only was he undefeated, he was famous for being able to start at a position of disadvantage and convert that to a position of advantage within 40 seconds. So he got the nickname 42nd Boyd. As you can imagine, that attracted all sorts of attention from the Department of Defense. And they brought him back to Washington D.C. and made him sit in an office in the Pentagon. And he came up with, you know, what he believed was the concept that allowed him to be so successful was that he went through this decision loop. Observe, Orient, Decide, Act. It's actually much more complicated than that. I would encourage people to go out and read the papers, read. There's a great biography written on him, just called John Boyd and basically determined that he could do that faster than. Than his adversaries, and that was what gave him the advantage. So when we started the firm, we thought it was a great homage. I had been in the Pentagon 25 years ago, part of a group called Disciples of Boyd, looking at the application of the OODA loop into other contexts. There's been other great folks like Chet Richards, one of the first people to interact with Boyd on the OODA concept, who has applied it to business in general. And we felt like it was a great framework for establishing a consulting company because at the end of the day, we wanted to be able to help our customers make more intelligent Decisions. So that was kind of the origins of it. We did get blessings from the family before we did it. And even though I owned the domain, I wanted to be comfortable that they were okay with us using kind of his heritage in that capacity. And they definitely gave their blessings.

That's great. And I gotta believe they appreciated the reference and the homage to him. That's a great story. Now, in terms of maybe talk briefly, if you don't mind, Matt, about your firm and some of the things that you do.

Yeah, we are, you know, boutique consultancy and I think that's just a fancy way of saying we're not too big, comprised of really senior leader experienced folks that come out of the private sector and the government. And we provide services on kind of a variety of ranges. Some we do direct consulting. So we do still work with large Fortune 500 clients on addressing hard problems. Those can be cybersecurity problems, it can be geopolitical technology transformation. So that keeps us engaged with the threat environment that exists out there, what our clients are dealing with and the solutions environment as well. And then we do some work on the advisory side. So we do work with companies ranging from small startups to multi billion dollar corporations on growth and strategy, how to grow the company, how to adapt in the market, how to adapt their products, how to create products for the market. And as part of that we do some M and A support as well. So we'll work on due diligence for deals and also some buy side advisory where there are companies that know that they need a capability in a particular space but don't know who is in that space, what would be a good fit. And we'll help them with that process. And then the last piece is we also run an expert network@ooda loop.com, which would provide some expert content and analysis, but then also run a network of practitioners and executives to share ideas. We have an encrypted private chat room that we use and then we have a monthly member meeting where we talk about a wide variety of global risks and opportunities. And it's a Chatham House rules way for them to get together, interact and share information.

Sounds great. So yeah, you say boutique and small, but it sounds like you got a lot of things on your plate, which is impressive. So Matt, maybe I want to start by asking a little bit more about cybersecurity. Now you've obviously throughout your career played a pretty important role in different aspects of cyber. We see it scip. We work with a lot of corporations, a lot of companies do Have a CISO or chief information security officer and they're usually in it. Now what are your thoughts about that being a really effective countermeasure given that so many cyber threats do wind up being behavioral or internal? Are companies organizing to defend themselves from cyber incursions in the best way?

Yeah, some companies do better than others. Right. And what we need to recognize is that for a lot of companies, this is a blended threat. You have pure cyber attacks, but then you have cyber attacks that are intended to have physical consequence. And then you have physical elements of a cyber attack. If you're in the critical infrastructure space, obviously there are physical consequences to your attack that manifest themselves through cyber attacks. And the Colonial Pipeline is a great example of that. It ransomware in the billing systems that resulted in them shutting down the physical infrastructure, not being able to use that pipeline to transport resources to the East Coast. So you need to make sure that with cyber risk. One of my key talking points in dealing with executives and board of directors is that it is a risk that you need to manage like other risks in the business. A lot of time we want to view it as kind of magic or different, but it really isn't. And the concepts of risk management that you would apply to other aspects of your business apply here as well. And typically in risk management, you want to have a multifaceted, dynamic view of the problem. You don't want to have an isolated or narrow view. So you also want to make sure that all of the stakeholders. The CISO is a great role for spearheading the technology risk in the organization, but you need to make sure that that position is elevated enough to articulate the impact to areas of the business, but that also that the stakeholders in the business on the operations side have an ability to get the CISO to understand the business operations aspect because you, there are trade offs that you make. There are things that you're trying to do that enable the business. And you need to make sure that you have kind of one common message, one common risk management strategy.

And that's. Yeah, I like what you're saying there about really being. I mean, it's a risk, right? The sky's not falling. There's no silver bullet, as you said, there's no magic around it. But like any risk, it needs to be managed. And I think the thing that a lot of organizations are struggling with with cyber is that there are so many entry points, so many weaknesses within their organizations. Right. You know, it can be, I got a manufacturing background, you can have that H vac system in the plant that, you know, is that that point of entry? And so you're really seeing this at the board level. If you've seen, you know, do you see generally in your clients that this is something that the boards are allocating proper resources and attention to?

They are, they're definitely allocating, you know, appropriate attention in those instances where there's, you know, a great risk. I think they're, they're giving the appropriate resources to it. In most organizations. The only point that I am freeze, you know, increasingly frustrated at, at the board level is the lack of expert at the board. I get brought in by board of directors who say, oh, we want you to come interact with us because we feel like we've been taken technologically hostage by our CISO because we don't understand what they're saying. And the point that I always like to make is it's time now to actually have experts like me represented on the board itself. There is not an organization out there that doesn't face some sort of cyber risk. It's a key component. They should be having experts on the board as an independent board member that has that risk and can help the business manage to it. Because there is at times a translation issue and there is also, you have to, you mentioned the attack surface is so big. You have to start looking at the representative risk based on the actual impact of the business. What would be catastrophic or consequential for the business. And let's manage to those risks first. In a lot of organizations we see they want to take this kind of unified approach. We had an assessment. The assessment found 70 issues that we need to address and it's going to take us five years to address those 70 issues. But if you sit down and you say, well, what is actually critical to your operations? What is actually, you know, are the systems here that are consequential, you know, to the organization or would be catastrophic or they were if they were attacked. Let's figure out a strategy for how to defend those first. And then let's also focus on measures from a cyber defense perspective that are actually meaningful. And I was guilty of this as well. If you'd seen me brief 20 years ago, I would have walked into a board of directors and said, hey, we did an assessment and we found 50 high level vulnerabilities, and we found 75 medium and we found 100 low. And then six months later you walk in and say, okay, we only found 45 high level vulnerabilities. And you view that as an improvement. That's a good thing. Yeah. And it is an improvement in that you are mitigating vulnerabilities. But it isn't an improvement because I might still be living with the same exact risk as I was previously and folks were measuring the wrong thing. So, you know, as my career developed and especially as we're doing stuff at Fusion X, I would walk into the board and say, here is the threat actors that you need to be most concerned with because they target your industry or they are motivated, whether it be politically or financially, to target you. Here is the ttp, the, the tactics, techniques and procedures that those threat actors deploy here. I came in with my red team and we use those ttp and here's the attack surface that we found. And then here's how we leverage that attack surface to move within the business and achieve an objective. So we would operationalize the attack. I could tell the bank we stole millions of dollars from your bank. I could tell the company we stole your intellectual property, or we simulated the theft of your customer data with a PII breach and then tell them, here is how noisy we got until your team detected us. So we were able to be on the network for six weeks and at this level of threshold, and then we would go back in and we'd say, okay, you've eliminated that high risk attack surface. So we had to innovate. Here's how we innovated. But more importantly, here's how much more quickly your team detected us. Here's how much more difficult it was for us to move laterally. And if you can focus an organization on metrics like time to detection and efficiency of response, you can greatly diminish the impact that a real attacker can have against your organization.

Yeah, no, and I think that's a great point. And like anything, it's a risk that is managed. And a lot of companies, let's be honest, are really sophisticated at managing some risks, whether it's the insurable risks like property casualty. I mean, there is a pretty decent market out there now for cyber insurance. You picked up. I want to pick up on something you said a moment ago, Matt, about, about, you know, talking to the board and saying, here are some of the likely actors now, this whole concept of cyber threat intelligence. So, so as I think about, you know, the average corporation, you have a competitive intelligence person, for example, they are awesome at, you know, looking for weak signals at the marketplace for, for triangulating, even making predictions based on tons of external data. So that seems like a skill set really suited towards cyber threat intelligence. On the other hand, you have your typical risk manager, which they're great at what they do, but a lot of companies are insurance buyers. Maybe talk through the concept of cyber threat intelligence. Who does it in organizations? What are some of the best practices or weaknesses that you see?

Yeah, absolutely. I'm a big fan also of having intelligence driven security organizations. Right. So that you are actually managing your risk based on a realistic threat variable. Right. When we think about just the classic measures, way to think about risk, when you think about threat vulnerability impact as being kind of the three primary components, if you're not engaging in threat intelligence, you're really operating blindly with regards to that threat variable. So threat intelligence helps to inform that in a variety of manners. And from a cybersecurity perspective, it gets a little bit complex because you have geopolitical, which obviously influences what threat actors are doing or what they're going to target. Some of that geopolitical changes on a dynamic basis day to day based on what has that country's attention. Some of it doesn't change. If you're in the defense industrial base or in the aerospace industry or advanced pharmaceuticals, you can assume from a threat modeling perspective that you've got nation state adversaries are going to be targeting your organization on a consistent basis. Then you have kind of the non geopolitical, the kind of the criminal element that you need to manage to as well. And the collection on that is a little bit different. Here's where, you know, information sharing and analysis centers and the ability to, to share intel between organizations with the government as an intermediary to a certain extent becomes valuable because my view typically in that domain is only what's impactful to me or what I've seen. And I want to have a much broader view. The cyber criminal targeting my competitor yesterday might be targeting me today, it might be targeting somebody else tomorrow. So you want to develop that kind of broader industry sector view and then also understand how those threat actors are communicating. I mean in the cyber crime space they are in the business of making money. And that means that there are markets and ecosystems that exist that you can develop a human type approach and to penetrate and observe to get those indicators. Is there a threat actor who is trying to buy information about access in our network? Is there somebody that is offering it for sale that would be indicative of the fact that we might already be compromised? There's that type of collection that you can do as well. And then there's kind of the machine speed threat intelligence, which you know, for most organizations is fully Automated, that is the new vulnerabilities that are being released, the new exploits that are in the wild, the new signatures for the detection of that where you want to be able to ingest that and protect your network. Maybe it's a new command and control, the IP address that you want to block any connections in your organization from going against, or an MD5 hash for an application that is a new variant of malware. So there is a kind of an automated machine speed threat intelligence component as well in order to be able to keep up with the pace of that technical data that's coming in.

Yeah. First off, wow, it's a little overwhelming, right? I mean, as you say, this is pretty much any kind of company is vulnerable to these types of attacks. And there's a lot of great research out there that, that it's some shockingly high number of boring mom and pop industrials in the Midwest. Like 80% have had some incursion in the last year. So it's by no means just the Lockheed Martins or the bank of America's of the world. Now if you could generalize, Matt, do you find that there is. Is the motive for these types of acts typically economic, in other words, like stealing IP and then selling it in the black market, as you mentioned, or with nation states, Sometimes economic, but sometimes it's just they want the IP to help industries in their own country.

Yeah. When you're talking about nation states there again, I hate to keep adding complexity to it, but there's a couple of different variables here. There's one in that it's kind of the preparation of the battlefield for potential cyber conflict. So adversaries will engage in penetration primarily against critical infrastructure. Let's say, for example, a power grid, not because they have the current intent to target that power grid, but because they can Envision A future 3, 5, 10 years in the future where they might want that capability, where they might want to use it as a deterrent or to distract us from a regional security issue. And in order to have that capability, you have to be attacking and penetrating now. So there's that aspect of it. Then there's the traditional espionage. They want to collect information on individuals and programs and policies, etc. And so they will target organizations with the intent of engaging in what we would consider to be espionage. When Willie Sutton was asked, why do you rob banks? It's because it's where the money is. Well, now, today, money is now in cyberspace and so are our lives. If you just think about the social media platforms and the dearth of information that is available about each of us. It's massive. You know, when I, when I present to groups, I have a little triangle slide that I put up that says, you know, here in, I think the year is 2015, three breaches that I received breach disclosure notices on and, you know, contain almost the totality of information about me. One was the OPM breach, which, you know, allowed for the compromise of all my personal data and relatives and Social Security numbers and places that I've lived and, you know, all that fun stuff. The second was a healthcare provider breach, which gave access to doctors appointments and health conditions and medications. And the third was a major credit bureau that had all of my financial data. So if you just think about it from an espionage perspective, you can paint a pretty incredible picture of Matt DeVoe just in the breaches that occurred that impacted me in 2015. So, of course, the traditional espionage actors have moved into that space around intellectual property. Yeah, there's an espionage aspect to that. Obviously, for years we've had examples where we've engaged in espionage to understand advanced technologies, particularly defense technologies and adversaries have as well. But then you also have different cultures that have different concepts of intellectual property where they believe that just because what we've created. If you use the United States as an example, a bulk of our innovation has occurred over the past hundred years. You could look at if from the perspective of an Asian country that says, well, we've had innovation for thousands of years and intellectual property protections didn't exist, and the world benefited from the innovation that we were able to bring out into the world. So why should our society not benefit from the innovation in your country based on the fact that IP protections exist? So we're just going to kind of violate those and steal that innovation so that we can deploy it in our own markets. And then at the crime level, it really does get down to the economics and the financial. I mean, the criminals are in the business of stealing money. It could be that they're engaging in attacks that allow them to steal money directly. We see these business process attacks or business email attacks where they get in, understand who you are, how you wire money. We see social engineering attacks where they call up and convince you to pay your invoice for a service provider to a different bank account. We've seen where they hack banking credentials directly and steal the money or engage in fraud against your customers. And also we'll see where they will breach accounts and then sell that data that they have on the private market. But at the End of the day, primarily financially motivated at the cybercriminal space.

Yeah, that's pretty alarming, the number of tools at their disposal. So bringing it back to the perspective of, of folks who would not be a ciso. But again, our audience is primarily market competitive intelligence and strategy folks. So these matter. Not the people who are going to be monitoring servers or they're tech savvy, but they are not going to be CISOs. Are there any activities that they should watch out for that a competitor could conduct? Offensive cyber activities that might actually be legal or ethical, or is that not a thing? You know, obviously hacking and stealing IP is illegal, at least here in the U.S. but are there any kind of softer cyber offenses that a competitor might conduct that CI people should be aware of?

We see that, right? I mean, one is that kind of tried and true insider recruitment, you know, where they try and get an insider to leave the organization. You see a tremendous amount of that. The other is just mining the open source data that is available and being able use that to deduct patterns or develop profiles around executives or determine intent in a particular market. That is one thing to talk to clients about for a long while now. I think I did the first ever open source intelligence briefing for the FBI back in the mid-1990s and the capabilities have just evolved since then. I would be looking at social media profiles for executives. I would be looking at the emergence of new relationships. I mean, you can determine quite a bit about potential M and A activity by just watching relationships between respective companies. If all of a sudden they start to have more connectivity, if they start to have check ins at restaurants and you know, the city where the headquarters is located for the company you think they might be acquiring, there's all sorts of data that is leaking out every day that's 100% ethical to look at. But a lot of times these executives are surprised with regards to the picture that you can paint from a competitive perspective based on this information that's leaking out in the public domain, or not even leaking out as being deliberately posted or these transactions are being deliberately maintained. If you think about LinkedIn connectivity and other social media network connectivity.

Yeah. So I like that because we've gone from talking about an intentional breach into a network to, you know, maybe there's an Excel sheet that you left unprotected, you know, shame on you, but it's out there in public. But then you're talking here, Matt, about someone essentially just detecting patterns where, wow, there are a lot of folks from, you know, Coca Cola Coca Cola executives who are recently friending on LinkedIn, whatever, you know, Pepsi or pick your brand. So yeah, what from a, from an awareness or maybe even a coaching perspective, what can companies start to think about doing so that if they do want to be smarter from a counter intel perspective, what are some of the things they should watch out for?

Yeah, I always recommend a couple of things. One is do that assessment of the key executives to see what information is out there and what their habits are and expand it to include maybe members of the immediate family. That's a, from the perspective, you know, it's kind of twofold. One is that, you know, the kind of the competitive intelligence just being gathered from open data, you're going to get a good sense of what's out there and what their kind of general, you know, privacy hygiene looks like. But also this helps you in helping prevent more targeted attacks that are illegal intrusions where the executives might be targeted through, you know, a phrase I coined multiple years ago, spearfriending, where they establish a relationship over social media with an executive or a member of the executive's family. I've done that multiple times over the years. Or create a fake LinkedIn profile or a fake Facebook profile and the executive connects to it directly or you get a family member to connect to it. And then once you're friends, you start able to see the photos of where they go and where they have houses. There's a physical risk protection element there too. Where I've worked with a lot of high net worth families and very successful individuals who are under kind of persistent threat of physical attack. And that cyber privacy audit helps you to understand the attack surface there. I'd be surprised working with a family years ago that was under high threat of physical attack where a member of the family had posted all sorts of information that included the residences that the family had, resources like license plates on automobiles, names of waterborne ships where you could go and look up. And then the key was they posted a photo of a wedding invitation that showed the exact location that the entirety of the family would be brought together A at the religious facility and then at the reception location itself and just not understanding the impact that that leakage can have. The second aspect of that is I always recommend just an education and awareness campaign that is targeted towards those executives but is done at a more intimate level. And you see this even in government as well, where I get brought in to brief at an intelligence agency that has people in cyber that are much smarter than I am. And when I say, well, why don't you just have your own people briefed? They say, well, a, they want to hear from the outside person. But then we're doing this executive session and the executives want to be able to ask you questions that they're too embarrassed to ask their own staff. Right. So setting up these kind of safe training environments where they can ask questions, you can do direct observation. And we've done training where we've had the, you know, the executive and the wife and the kids and we've picked up their mobile phones and they've, you know, we've walked them through setting more advanced privacy settings or setting their Facebook privacy profiles to be more private, getting them to understand the risk. Simple tools that they can buy off Amazon to charge mobile phones through accessible USB ports and airplanes and hotels, et cetera, that prevent the data connectivity so somebody can't hijack into their phone because they plugged into an untrusted port. Just simple things like that that tend to stick, particularly when you can, you know, the person who's providing the training can provide these stories around real world impact and things that have happened.

Yeah, I like that because as you say, if you say to the executives, Here are the 50 critical problems, the 100 medium ones and the 2000 small ones. It's so overwhelming, you kind of get paralyzed by it. And so I like what you've said there, Matt, about here are specific recommendations, some of which are just going to buy this plug on Amazon. The baby stepping probably gets you a lot further.

I'm amazed how many of those I've sold. Just a little $3 device that you plug your USB charger cord into and then you plug that into the USB port and it just disconnects the data connection. It lets the power trickle through, but it disconnects the data. Very simple and cheap countermeasure that you can attach to your cable and forget about that people just don't know exists. They don't understand that the risk exists and they don't understand that the mitigation cost them $3 on Amazon and you.

Should buy stock in the company. If you've.

I should have, I should have.

You should. So, so maybe one last question. So if, if, if you're that head of market or competitive intelligence listening to this conversation, you think, you know, you know, look, I, I know kind of the basics about cyber. It's, you know, don't log into random networks and don't, you know, don't click on, you know, what are a couple concrete steps that a CI professional should take to Help their company on whether it's cyber threat intelligence or just more broadly cybersecurity.

Yeah, you know, I'd point them. We have a guide on OODA Loop that we've compiled based on what we call different thresholds of the threat environment that includes a bunch of different factors. And if they just search for, you know, cybersecurity executive guide on OODA Loop, there are common steps there and we kind of take it from hey, I just want to be more robust in my personal privacy profile to I'm operating in what could be considered to be a competitive or dynamic threat environment and then escalating to I'm an executive traveling to a hostile country and expect to be directly targeted. So you can look at that and kind of determine what steps you want to adopt. But there are simple things that can be done with regards to just making sure that your devices are up to date that you're using two factor authentication. And I can't tell you the number of breaches that I've worked over the years, particularly of executives or high net worth individuals where it could have been easily mitigated through, you know, prevented through the use of two factor authentication. Very, very simple step. You know, obviously creates a little bit of friction but not too much in today's environment to get access to those accounts. It provides an incredible amount of protection. So I would look at those and think through which ones do you want to adopt personally? And then which ones do you also want to get the executives to adopt as well?

I like that. I mean it's a reminder that we all play a role in helping our organizations and frankly just our families be more secure. There's the things that we can do ourselves but also influence others, not that. Especially from a counter intel perspective. Switching a little bit over to artificial intelligence for a second, Matt, in our members world they are frankly spending a lot of money on technology. And one of the primary reasons for that is with data explosion, CI professional could spend all their time doing nothing but collecting data. And that's nothing against data collection, but there are really other things they need to be doing. And so they're using more and more third party generally SAS tools out there to assist with collection, analysis and dissemination. As CI and other functions start buying more SAS tools that pull data from some cases, hundreds of locations. Are there risks? We're not thinking about, you know, anything from just data dependencies to, you know, we, we, you know, putting bad data in to train our algorithms. But you know, what, what are some of the downsides or, or you know, maybe benefits of having that, that type of software environment.

Yeah, that's a, it's a complex question. We could probably spend a whole podcast talking about that. But I'll share some high level thoughts for you. One is if this data is going to be used to support decision making, you need to understand maybe the bias that's inherent in the data based on how it was collected or the sources you collected from. That's one aspect when we talk about training data where you've got some inherent bias. I always like to say that bias or mistakes like that as it relates to machine learning in particular. AI. It's a great buzzword, but we're not really there yet. But machine Lear can provide incredible value in these organizations. But it's like money in a bank account where you have compounding interest, you can have compounding impact of the mistakes. And a lot of these machine learning environments are somewhat autonomous in that they deal with so much data and make decisions at such a capacity that the humans can't decode them. So you really are at risk of being downstream and a having to revert your algorithms back to day one and losing two years worth of progress or making decisions based on data that has gotten incrementally more rotten over time. So that's one area. The second is you have to think in particular about adversarial impact to data if it's being used in decision making where an adversary might kind of pollute the data stream intentionally to try and drive a decision in a particular direction. So that's another area. And then we need to think about just kind of adversarial machine learning in of itself. And we've seen where you can develop machine learning capabilities that basically kind of learn in parallel with your system, whose intent is to be somewhat adversarial, to prevent access to data or pollute data, or drive you towards making decisions that are contrary to your best interest. And again, here we're talking about potentially the use of automated decision making through machine learning. So being aware of that and the technology doesn't just get used for positive benefit. The same machine learning techniques that you might use to drive benefit in your business, somebody might be trying to apply against you to drive you in a particular direction. It's a large complex topic. It's something I've spent a lot of time on over the past several years because what I've seen is this natural progression when we think about the Internet. Steve Lukasik, who was the director of DARPA when the Internet was created, was a friend Night over beers he said, geez Matt, had we known this was going to be the backbone of global commerce and communications and society, we would have thought about security. But it wasn't. We were trying to create a resilient network. It was meant to be kind of a node agnostic way to route around damage in a network and maintain communications. I saw the same thing when the emergence of IoT and industrial IoT where we had all this great benefit and people rushed to bring it to market. And now saying the same thing, oh man, we should have thought about security before we hooked that refinery up to be completely controlled by computers. We should have maybe had a security model for the build of those devices. Now I look at it and say every company over the next 10 years is going to be impacted by machine learning or some early phases of artificial intelligence in a way that can be a big impact beneficially to their organization. Maybe you should be thinking about the security model. Maybe this is one of those where we slap ourselves in the forehead and say, oh man, I should have thought about that. Now is the time. Before you build the dependency as you're making these investments, let's think about the integrity of these systems. They can derive great benefit. One aspect of my history that we skipped over is actually built a special purpose investment holding company where we brought machine learning to boring entities to help with their growth trajectory. I saw firsthand, put 15 brilliant data scientists in the room with any type of company and you can greatly impact that business. Let's also think about the security measures of that as well.

Yeah, that's fascinating and a little bit scary. So I'm going to probe a little bit on that. Now this malevolent ML, there's probably a cooler catchphrase than that, so if you have one, I'm all ears. But is this something you've primarily seen on a national security level or is this actually occurred in the corporate environment?

Yeah, you're seeing it. You know, it's primarily the research is being driven at a nation state level. Right. These are national programs. But this, you know, a lot of the emergence of great machine learning capability has come out of academia. Right. So you see that as well. And we see this as well with kind of the adversarial machine learning where a good exemplar. There's been a few stories recently about how it's impossible to solve captchas because there have been adversarial entities that have basically incorrectly solved them. So the training data that I was using for my captcha machine learning capability to understand what is a crosswalk was polluted where they didn't understand what a crosswalk was anymore. Or we see that where people try and take advantage of algorithms around autonomous vehicles driving to impact them by either inputting symbology or changing the profile of a stop sign or something that they might recognize. So you see that happening kind of across the spectrum. And a lot of interesting stuff. I put out a newsletter for free every Sunday called the Global Frequency and I try and track developments in that space of adversarial machine learning. There's at least once a month a really fascinating paper that is originating out of a research entity, at a corporation or within academia that demonstrates some of the cautions that we should have in developing these dependencies on data and machine learning. Another one thing I would recommend. There's a great book that I just read by an author called Jared Thorpe called Living in Data. He's a great data scientist that I like to analogy is the Indiana Jones of data science. Not only has he worked on these big data science issues or machine learning issues, but he's traveled around the world to look at issues of data adjacencies, missing data, our relationship and data we can collect from the environment that helps us better understand humans relationship with the environment and how it is changing in these ecosystems. Fascinating book, but also gives you what is a very accessible narrative around understanding these issues as well.

Yeah, and I love that. And it's a good reminder that there is warranted enthusiasm around digital transformation and all the things that powers it, including ML as we've been talking about here. But like with any trend, there is a downside and there are aspects of that trend that people will seek to exploit. So it's a very well made point Matt, that we need to take a balanced approach towards this because people are people. And whether we have adversarial ML today in your face or whether it's 10 years from now, I think your point's well made that it will come.

And it's that aspect of you can't manage to a risk if you don't acknowledge its existence. Acknowledging the existence of the risk now allows us to build with the benefit of these programs while appropriately managing the risk. I think too often with technology deployments like this, all that we can see are the benefits. We can see the impact on our revenues or our customer retention or access to new markets or whatever it may be that we're hoping to magically enable through data science, but not thinking through the risks. So then downstream, when we've built that dependencies or we have a critical issue based on the pollution of the data that we used in that environment, then we're kind of slapping ourselves and. And you're paying a lot more downstream with regards to impact of the business or just even the cost to mitigate the risks that you've introduced then you would have if you had acknowledged those risks and planned mitigation strategies in parallel as you're moving forward to reap the benefits of machine learning.

Yeah. And it's a eye opening way to look at the challenge. Right. Because if you looked at the concept of security from a corporate perspective 50, 75 years ago, a lot of it was largely physical security. And as we started our conversation talking about cybersecurity, stealing ip, we didn't talk about viruses, but malware. Those are obviously a concern. But where our conversation now really is, Matt, it's talking about far more subtle ways to cause problems, injecting bad data. Adversarial machine learning. These are much more nuanced and sophisticated ways for bad actors to disrupt organizations and nation states. It's scary. It's a little scary, but I guess you get that a lot.

Yeah.

Now, last thing I'd love to get your perspective on is where all this goes. And so one of the things that CI professionals are keenly interested in in is some might call it foresight megatrends. But you know, looking out several years, where do you see emerging in the disruptive technology landscape going over the next few years? Do you see ML just accelerating and proliferating? What are some of the trends and issues that you think our listeners should watch out for?

Yeah, I think definitely the acceleration of machine learning is going to be huge. By all means. Right. And then the autonomy and automation that that introduces into traditional environments. Right. We've seen it in factory floors for a long time now. That's only going to advance. We're going to see it in self driving cars. You know, we're seeing rockets that are able to land themselves now, which was inconceivable years ago. So just kind of that there are all these parallel advancements that will be made possible based on advanced computational power and AB to analyze and act on these huge data sources. So the AI machine learning. Absolutely. A key area, another one that we pay close attention to at UDA is developments in quantum computing technology and the impact that that will have. And it really is kind of understanding the landscape. But if you are an entity, particularly a government or a company that relies on protection of critical IP through the use of encryption, you need to start thinking through what is the lifespan that you want that protection to be enabled for and how does that map up with developments in quantum computing and the potential for quantum computing to crack encryption. You might be using an algorithm now where you are hoping you're going to get 30 years of protection based on current computational assumptions. And the introduction of quantum computing could reduce that to a five year window. And is a five year window protection of those secrets adequate or do you need to develop more robust quantum resistant capabilities? Definitely an area that is of keen interest. There's two more that I would mention. This is also another topic that I'm fascinated by and spend a lot of time, but two that I think that would be great to mention. It's also developments with regards to gene editing and understanding DNA and CRISPR and all kind of that sort of stuff. And I won't pretend to be an expert there, but I've read enough and been brief, briefed by enough people to understand that those are game changers with regards to medical and just kind of our existence as a human species and understanding the potential impact there. And then the fourth one that is a little bit more, it's going to sound a little bit more grounded in Earth, but maybe a little bit more out there in whether predicting it accurately or not is I do feel like we're going to migrate within these systems away from large third party aggregation stores of data about us individually. If you just think about how we operate today, all of your network traffic, what you're buying off Amazon transactions that you're engaging with with your credit card, all of that data gets collected and aggregated and it gets used to market stuff to you and you could determine that that marketing provides some value. I certainly derive value from Amazon's book recommendations as I'm an avid reader, always looking for the next thing. But we don't derive much other benefit from it. One trend that I definitely see emerging is this concept of digital self sovereignty where the individuals start to take more control of that data and start to dictate the relationship and how it's used with these third parties. And I think we're just starting to see the emergence of some pushback on that collection of data. If you just look at Apple's announcements with iOS 15 and the, you know, basically embedding a Tor network private browsing capability to help prevent this aggregation of data. If you look at developments in blockchain, you know, and digital identity and you know, immutable transactions and things of that sort, I could envision a Scenario and I hope that I'm right where we start to control more of that data, but then we start to become empowered to engage in ML for our own benefit. You know, maybe the ML notices that I've stopped at 7:11 and bought Pepto Bismol four times over the past two years and every four times that I did it it was because I ate this particular type of food. And maybe there's an allergy. Right. I mean things like that where we don't get the benefit of our own data that could be used to improve our lives and also has monetary value. Right. Where we can determine the value proposition for what we allow others to know about us. So I certainly feel like that's a trend that's going to be interesting and worth watching at a big macro level.

And Matt, you call that, was it data sovereignty?

Yeah, they're kind of digital self sovereignty. Yeah. Where you start to become in more control of your data. Right. So you would almost have kind of your personal data repository and maybe on a transaction basis, you know, you define the rule sets and it might be that some people just say I want all of the data about me accessible and it gets me at $10, 10% discount at the ice cream store or you know, at the bookstore or whatever it might be. Or others might develop a better protection profile around that and say, well, I'm okay with Amazon knowing what books I read, but I'm not okay with them knowing that, you know, I'm buying diapers because my wife's pregnant, you know, things of that sort. You can think of all sorts of different scenarios. But more importantly that we start to be enabled with mining and using that data for own benefit. Because there's, you know, I wrote about this a decade ago, wrote a quick, quick essay that said I am, you are big data and so am I. Where we are big data right now, I mean we're generating a ton of data just even as we sit here and exist on the zoom line. But we don't get to derive any of the benefit of that data. And I think the world is going to change in a way in such that we'll start to be able to derive benefit from the data that we're generating on ourselves.

Well, I think it's a great place to end because it really, you know, from my perspective, Matt, it kind of is a good reminder and a wake up call that we need to be proactive about our digital footprints, about how we create share store data both as individuals and also as, you know, members of organization. That we can't be passive and just assume someone else has taken care of it. Matt, that was a great conversation. Really appreciate you talking with me today.

Absolutely. It was a lot of fun. Thanks for having me on.

Yeah, definitely. Folks, Matt DeVoe from OODA. We will drop a link to a couple of things he mentioned, including their newsletter and website in the show notes and appreciate everyone listening.

Bye bye Bamboo.