Katharina McFarland on Winning in the Age of Artificial Intelligence

Informing your decisions with intelligence analysis and insight brought to you by the team@ooda loop.com.

The video you're about to see is from a recent OODA salon which featured the honorable Katrina McFarland. Katrina McFarland has led change in a wide array of national security domains, including space, missile defense, nuclear, nuclear posture, and acquisition. She was also named a Commissioner of the recent National Security Commission on Artificial Intelligence. In this OODA salon, we hear directly from her on things we all need to know about the coming age of AI and things that we should consider doing to accelerate the use of AI into our businesses and our nation, and of course, our national security community. I hope you enjoy. Ladies and gentlemen, welcome to today's OODA Salon with the Honorable Katrina McFarland. She was a commissioner on the National Security Commission for Artificial Intelligence. And Katrina, we really appreciate you being here today.

Our honor.

Well, let me give folks a little bit more of your background. In reading your bio, I noticed that academically you have a science and engineering background. And then you applied that in public service, first with the US Marine Corps as an engineer, and then in the acquisition community. You became president of the Defense Acquisition University, which is widely known in the the national security community and DoD circles, and of course the circles of community that feeds the Department of Defense as a critically important university that helps teach people the right process and methods for acquisition. After that, you went on to be the. No, I'm sorry. Before that, you were the head of Acquisitions at Missile Defense Agency.

That's correct.

Then the President of Defense Acquisition University. Then you were the Assistant Secretary of the army for Acquisitions, Logistics and Technology, which was the Army's acquisition executive. You were later appointed to be the Assistant Secretary of Defense for all acquisition. Did I get that about right?

No, but you were close and it really doesn't matter, right? What you said in general is the truth. I have a background, started in 1986 in engineering and ended up in procurement because I wanted to get things done and ultimately ended up as the Assistant Secretary of Defense for Acquisition. I double had it for a period of time, also as the Army's acquisition Executive. So I could check my own homework, but yes. And now I'm out on the street causing trouble. Yeah.

You serve on several corporate boards. SAIC is one of them, so. Absolutely gives you a feel for governance, the importance of that. And you serve on nonprofit boards. I got to serve on one with you. So I got to know you that way with the Defense Intelligence Memorial foundation, but also as a Commissioner of this National Security Commission on Artificial Intelligence. And that's what we hope to get some of your thoughts on today in this salon. And before getting into that, to kind of kick off and frame the discussion, I wanted to read to folks the first paragraph of your report because I think it's a good introduction to why AI is important. It goes like this. It says, no comfortable historical references capture the impact of artificial intelligence on national security. AI is not a single technology breakthrough like a bat wing stealth bomber. The race for AI supremacy is not like going to the moon. AI is not even comparable to a general purpose technology like electricity. However, what Thomas Edison said of electricity encapsulates the AI future. He said it is a field of fields. It holds the secrets which will reorganize the life of the world. Wow, that's pretty impressive. And with that, Katrina would love to get your thoughts on AI and the commission and what we should all know about that.

Well, first of all, thank you, Bob, for inviting me. And hopefully the audience won't get totally bored out of their gourd from listening to me ramble on on artificial intelligence, because obviously I'm quite passionate about it. I started many years ago in electronics, and it's evolved, right? No longer does the form of AI that we're used to, which is computer, be the only place that artificial intelligence exists, exists in biology, chemistry, material science. It's everywhere. Let's start with a foundational baselining of ourselves so that you understand why I say what I do. The concept of artificial intelligence is essentially dating many, many decades ago. And that is essentially allowing for a human to extend or enable themselves further, replace something or do something that they don't have to individually or as a group, do. So for computer geeks, that means that computation, when it first evolved, it was trying to get us. If you recall, some of you probably watched the show that talked about the computers needed to put us on the moon, but processing is one of them. But frankly, a coffee machine that we time and put on a clock to wake up in the morning to having fresh coffee is artificial intelligence. It's essentially moving something, unburdening something, or enhancing something that a human wants to have done. Well, that's foundational to understand because if we think of it in those terms, you can understand the preface to the Commission's report. Now, the Commission report is a public facing document. If you go to the website and you type in the National Security Council's on Artificial Commission on Artificial Intelligence, you'll find the report, and we've enabled that report to be Interactive so you can actually have some fun with it and not have to be bored with 700 and some pages. Read the first 23 and then drill further if you wish to. And we've populated the commission with a lot of congressional language that was developed by writers of congressional language that actually about 75%, maybe even now 80% have been adopted by Congress. So it's had a meaningful impact. It was a joy to be inside and working with folks. And I felt a little bit like where's waldo? Of the 15 commissioners, there was only two of them that were not technocrats. And I was one of those too. But if you think about Andrew Jassy, Afsafra Katz, you know, Eric Horatz, we had some pretty, pretty amazing people to work with to help write this. And the most important thing that I think the report did was baseline everyone in that room by getting them cleared for access to intelligence information to understand what that world looked like that wasn't through our personal biases as a nation. What I mean by that is that it was rather revealing. And within one year, when we had everybody briefed to the threat, several of the companies immediately withdrew all their assets from China. Oracle did it within two weeks, for example. To understand what the world looks like, you have to understand the term national security. I just heard John Hamre say this, so I'm going to repeat it because I like it. There's a small ns national security and then there's a big NS national security. The big NS national security, if you go back to the Eisenhower view post World War II, it is that the strength of the nation is based on its economic power as well as its military power and many others. I'm just going to label those two. The kinetic and non kinetic strength of the nation, the ability, resource and protect its interests. Small national security is what we fondly call Department of Defense and its agents and affiliates. When we think of the small NS people get all wrapped up in war and conflict of a traditionally kinetic way similar to what we're seeing in the Ukraine, sadly for all those folks, God bless them. But what we're really talking about when we talked in the National Security Council or Commission was the big national security. And that is that the engine of our nation has been built on the diversity of the peoples in it. And their innovation has driven an economic strength to our country. And that economic strength is based on the biases that we have. And what has occurred over the last couple decades is we've been kind of asleep at the wheel for the erosion of Our intellectual property, our intellect in terms of innovation or and our thoughts that people wouldn't steal or take or abuse the basic principles. If I built it or I figured it out, I get to reap the rewards or benefit of it. It doesn't exist in other nations. So when you read that report, you'll see some very distinct discussions about how do we reset our clocks and move forward so that our countries biases, which is to tolerate other nationalities, other ideals. This country is based on tolerance. I mean think of the Statue of Liberty bringing your homeless or helpless, whatever. My mom escaped from East Germany to come here. We want to purvey those biases in the technology because otherwise the technology will implement its biases or on us. And when we think of artificial intelligence, the analytical powerhouse that we often think of, dream of is here today. The challenges are how do we do that and implement it in a nation that has learned from its bad and hard lessons and does it with thought? Thoughtfulness is a marker for our nation. Sometimes, however, that slows us down. Some of the commission's work were trying to accelerate and create the confidence of the nation about AI and all of its varying forms of enabling technologies that match that. When I think of artificial intelligence and report we talk about it in the first part and many others have done this in other activities in terms of artificial intelligence. If you you read the Cyber Solarium, there seems to be a resonating theme here. First part is enable the country to get to it and then use it to advance the country's ability to live into the future with its particular metrics of freedom, et cetera, what we base our value system on. I'm going to stop there for a second and ask if there's anybody who wants to poke at what I've said so far. Don't want this not to be interactive if it's at all possible. Otherwise I'll continue and bring about some thoughts to the table about where that needs to go and where it is in industry and government. And what we see has changed even since the Commission ended.

Would love for you to continue Katrina, and would also like to ask, if you're comfortable, would you share more of the story of your mother and her escape from East Germany?

So I'll do this top level because I'd want to ask my mom's permission. She's still alive. She's 94, but I'll give you a daughter's view. My mom was born in a town called Dresden and if any of you are World War II buffs, you know, it was heavily firebombed. In fact, more people died in Dresden and then Hiroshima and Nagasaki combined. It was firebombed. And it was pretty much an end of the war thing. But my mother grew up before the World War began and went through a huge change in Germany, both culturally and in many other dimensions. And when the war ended, she was on the east side. And I might want to note that every male in the family and any male with what looked anywhere older than 12 was immediately killed when the Russians came. Not to say I have anything against Russian people, but the government, I do. And so my mom, at that point, being in her 20s, chose to find a way out. And at the time she left, they had only put up the barbed wire and the dogs. The protection was not with a wall yet. And so she went out through Czechoslovakia, which was 9 km from her home. So she went out of her home through Czechoslovakia and came back in on the western side. And when she arrived there, she had a grandmother in the United States that had. This is strange. The day my great grandmother landed at Ellis island was the day World War II broke out. She was supposedly coming for vacation to visit her son, who lived in Upper New York and ended up, of course, the rest of her life in Upper New York State. My mother had gotten and was able to solicit and obtain a visa to come to the United States, presumably to bring her grandmother home, only to discover that her grandmother was not coming back to Germany and was now a firm United States citizen. And in terms of my mother's approach, she had to take 10 years to get naturalized and would never consider any other country on par with the United States. From her own experience, and let's just say that my mother would be a good person to interview. I've been trying to do that for the purposes of people realizing what history really is at a personal level.

Thanks for sharing that because it is so important and so relevant to our topic today, because one of the issues we all deal with with AI is topics like privacy and human rights. And I think all of you commissioners probably had your own personal stories like that that underscored for all of you why it's so important that our way of life be protected. And one other note on that. There's other people writing a lot like Kai fu Lee, brilliant PhD writer, who talks about the advantage that China has and how essentially they're going to clean our clock, they're going to beat us like a drum because they have so much data. But the fact is that's a Different approach, that's a different way of life. And we don't have to do it their way. We can have AI and its benefits and protect our way of life. And I think that's why I think these human stories are the most important stories in AI.

It is interesting, the history repeats itself in terms of how something happens. And there's actually a country song that I thought was good is if you want to hear God laugh, tell him your plan. But there's little things that everyone can do to assure our future. And to your point, and it's those little things that actually matter in an aggregate. And in terms of artificial intelligence, I'll just refer to the more common understanding of artificial intelligence, which is computers and programming, etc. We can prevent tampering with our software. There's levels of preventing tampering with software. So for example, we shouldn't malign artificial intelligence, we should malign its use. And how it's used is what we care about as a nation. So for example, we get on our concern when somebody talks about facial recognition because we believe in privacy. But yet facial recognition technology in terms of AI was that which provided huge advances in medical science in terms of detection, early detection, dissemination, interpretation. I'm sure all of you realize that, but probably you didn't understand or associate those two technologies are identical. So if I want to use the technology for medical science, I shouldn't say no more facial recognition technology. I have to be thoughtful. And so this is the thing about this type of technology space. Going back to my earlier conversation, we as a nation are risk averse because we've had failures and we're ashamed of them and we want to protect the future and we want to do it right. And in this particular complex problem, it's very hard. It's kind of like a divorce court. Which technology where is good and how do I make sure that I've picked the right one? Right? And how do I stop the ones that should be stopped? But then again, that same science can be used elsewhere. So it's a real tough nut. But if everybody starts to engage in the discussion and everybody talks about how they view ethically and morally and understand as a nation, we will write code and behave in accordance with our biases to our advantage, but also to our detriment, then we can move in a fashion forward that will allow us to have the success and protect our freedoms that we so want to contain.

Great.

Well, Katrina, please let's get back to the report and give us any other context you can on Its recommendations and things we need to know about how to move forward smartly.

Sure. So the first section of hard enabling is things that you might think of in terms of assume away. Right. That should be there. Well, what's interesting is that we discovered that the policies around the use, validation, testing, production, scaling, all of those things are actually very, very nascent. We do have a need as a country to validate before we issue forth whatever it is that we build. And in the terms of artificial intelligence, this is a very challenging area, both from a computing perspective, but also biotech, chem tech, material science, et cetera, those areas. These enabling sciences need tools. So we're very strongly advocating that the world, in our world is in particular the United States, start thinking about how to build tools that give justified confidence in the use of artificial intelligence. And those tools are very important to us. All right. Because that will remove the stigma that many had wrapped their heads around thinking artificial intelligence is terminator coming on. I would like to point out, because I can say in terms of what is in the press, we've been very clear, I think openly to the public that much of our social media has been because we believe in free speech modified and shaped by offshore influences. And so much of our own freedoms were very well studied by China and Russia and Tyrann and others and used against us to change our value systems or to misinterpret systems that give mistruths or moderately modified truths so that you could slowly shift somebody off dead center. How many of you sit down at night and realize how many things from your childhood have changed in terms of view and whether or not they're right, you probably even question yourself. Very artful. My mom talks about this quite a lot. Just I can bring this to the table. She says some of the things that she sees are eerily familiar to what Hitler did in terms of propaganda, but also essentially transforming the population. It's very, very troubling when it comes because it's through the use of the digital era that a lot of our folk A's, mores, value systems have been eroded. And I'm not a bible thumper, as they say. I'm a person who believes there are certain truths and yes, there are gray areas and it's an intellect that makes the decision on the gray area. But as a nation, we should understand value systems. They should be fairly simple or it causes discontent. That is in the report. That discussion is in this report. So again, tools to validate policies, to surround to help people understand how to work ethically. Morally, what are the rule systems around it? How do you provide people relief if something they believe precious has been impacted by it? Those things are in this report. And if the front section starts you thinking about how to bring your own capabilities to the table, good, because then the second piece is then how do we bring it to bear to reap the rewards as a nation, to protect ourselves from it? And we talk quite a bit about specific technologies investments into the National Institute of Standards and Technology investments and also governance. How do you think back to history you didn't experience when most people today that you walk into, except for our older, you know, older folks here, they weren't witness to the Cold War. They didn't have that surrounding concern or pressure that something bad was going to happen and that freedoms were going to be hurt. I think some of that recently has been revisited based on the stability overseas of an individual. But as a nation, we took for granted some of the things that were natural protections and barriers against influence, against our inability to operate with freedom, which is called the global commons, et cetera. So when I looked at the report and others did, the second part helped say, hey, you know, World War II, they established a top down strategic plan that all the cabinet ministers were engaged in and it was essentially chaired by the vice president in situ. And that's what we made a recommendation for, is a technology Competitiveness Council comprised of the members of the cabinet. So that it was a concerted and unified and strategically aligned plan to ensure that our future and our ability to produce the resources needed to protect our countries from freedoms were in place. That has not been adopted as yet. That's still on the table, but that's one example of a governance model to allow for people to have the ability to recognize and align themselves from industry, which is the strongest component of this whole dialogue to make this happen. The good news is most industries have picked upon it, up on this and written their own ethics policy for their population and for their producers, if you would. Many of them have also worked into their efforts things that are strategic in terms of investments for you that don't know about it. There's a, there's groups of venture capitalists and private equity firms that are actually organized with large firms, you name Amratheon, Lockheed, everybody to actually provide seed funds, kind of like angel funds or others to startup innovators to help them get started. And it isn't, well, I can build you and then eat you. It literally is to help breed more innovation, bring more STEM work back into our country and start Fostering and advancing the ability to overcome what we found with COVID which is a global reliance that actually can strangle. Much of the second part, which was conducted during COVID was uniquely helped, if there's a good news story out of COVID by the fact that we started realizing our interdependencies and investments that had been made by China and others at lower level communities which are actually part of our global supply chain. You'll have a whole discussion about how do we protect what is critical for us in terms of the grander foundry of the United States to be able to keep its ability to sustain artificial intelligence was a good umbrella for that discussion. Because as it turns out, as many of you are aware that work in industry as I do now, the supply chain is very, very convoluted. And our infrastructure to communicate with our supply chain is a very handy tool for those who want to, I'm going to call it as it is, steal from industries large and small and be able to develop their own mimicking technology space. So again, the second part or the second half was taking AI into the future. And that was inclusive of things like material science, enabling sciences technologies such as microelectronics, 5G. We really advocate for 6G, leave 5 behind. You know, those type of technology spaces were discussed a lot in the report. And the things that I would offer to each of us as individuals is the theory that each of us contributes something to this grander scheme. And how you do that is whatever way you can. We talked about the fact that the workforce really needs to be readdressed, that we need effectively an education renaissance. And what I mean by an education renaissance, if anybody could go back to the 1950s, we had actually what was called an education renaissance. We're talking about number two, which is we're leaving behind a huge volume of a workforce. A lot of the higher degrees above bachelor's are filled with offshore seats. That's not bad. But if they're going to come, they should stay. We're welcoming everybody, but to thieve away the education system isn't helpful to us as a nation. And we really do have a bunch of people who then can't fill the seats, who are organic to our country and those people have skills that we should adapt and tie into. So we advocated and it actually starts, if you would, with thinking in terms of more of a vocational or technical build out. Because when we had Covid, people started to realize that hey, we could actually depend on online learning and people actually come out with skills. Well let's expand this, let's provide people in job in situ learning where they come out with certificates that build up to a degree. Let's take young people who are looking for employment off the streets and providing meaningful jobs that they can do while we train them, they can produce product for us, they get certificated, we get a higher level of skills in the United States and it helps our economic engine all around. So we talked about that and means of doing it and investing in that with the Board of Education and the the Caucus on Education up on the Hill. So all of those things were tied to taking us to the future. So just to recap, the first part talks about tools starting off small biting crawl walk run, building out areas, talking about how to set the stage with policies and tools for testing and, and efforts that are things that are low risk that you start with to build that confidence out and then building upon it along with some governance models specifically targeting the Department of Defense. And then the second part is taking it to the strategic level. How do you looking at the big ns expand and globalize our ability with our collaboratives. We have a whole section on how to work with like minded nations and build out this technology and preserve the integrity of how they want to. Again, we're a nation of tolerance and we would be preferring to have partners at the table who also want to protect the way they view their freedoms or their way of life. And so we did a whole section on that as well. And then we talked again about investments. Where would those investments go best served? How much would it take to stand up an additional element of the nist, if you would or the National Science Foundation? We asked originally for a new stand up branch of the National Secure Science foundation and said we're going to actually sort of divvy up the National Science foundation into two sides. And I'm kind of getting to the end of the big picture stuff and again I don't want to completely cover just the Commission's work. I'd love to talk to you about how and where we are today because things have actually changed since the Commission ended and we ought to have a conversation about how that could interpret that into value for like for example, the intelligence community or things that go on in the department or commercial world as well, but on the fence world.

Those are exactly the kind of topics we want to get into. That was a fantastic overview and good context and I really enjoyed the report because it, it was thorough and hit on many aspects and you have recommendations for the intelligence community, for example, which were really right on the mark to me and I hope was accepted by the intel community because it was just laid out so logically. AI needs to be applied smartly in collection and processing and analysis and dissemination. And so very good report. And I did. I liked the way that you connected to other technologies. So it is not a narrowly focused, just AI. This connects broadly to microelectronics and as you said, 5G and quantum computing. So I like the connection there. And I also like what you said about everybody should play a part. This is not just for developers who take some extra courses on how to do machine learning. That's important. And I want more of that. Of course. But everybody can play a part. The legal community, the marketing folks, strategists, business leaders. This is important. And I think you guys lay it out very well.

Super. Yeah. I think that intelligence community rose very quickly. The questions now are not so much the intelligence community not realizing that they have a need to do this. It's a question of how. And there's still a cultural impediment. So for example, one of the things that I've always been an advocate for before the commission around was the integration of intelligence into our requirements and acquisition process. The issue for me is that we focus too much on commodity and we don't look at enabling technologies because that's the backbone to commodity. Until we had the, you know, radar absorptive material that we applied to the wings of an aircraft, we were pinging every radar out there. It's not the aircraft in that case, it's the radar absorptive material. Material science. Right. That made this a really sexy thing to have. Right? True throughout every element. If I don't have a good energetics, which is another enabling science that nobody pays enough attention to, I can't get that missile to be hypersonic. So for me, I think the evolving artificial intelligence allows analysts in the intelligence world to make much more data gathering a reality. Natural language translation, the ability to process volumes of data gives me a sense of a predictable plane of technology. And it's very interested in that. As an acquisition or an engineer, I want to know where things are going because I don't want to be behind, I want to be ahead of it. Right. As a nation economically as well. So for me, I'm looking at the integration of intelligence being broader than the commodity based, operationally based, typical intelligence collection. I'd like to. And we set up artificial intelligence sector in the intelligence community to look at that technology and where is it going? Same should be Done with material science, chemistry, biotech, all of those areas and I'm hoping they do.

Yeah, okay, I want to get to that in just a second. But first I want to mention these topics of AI have been around for a while. I guess the first AI conference in academia was in like 1956 or 57 and lots of progress since then, but it's been around for quite a while. And so what, what was really what caused the difference now? Why all of the excitement about AI today?

Well, I, I've always been of the opine personally that it there wasn't a suddenness to AI. I believe the real issue became a problem when suddenly a memo went out by similar to the committee of concerned scientists. It was signed by a bunch of people who said oh my gosh, with machine learning there's an opportunity for machines to make human decisions very badly. Horribly, horribly mispaced. It's kind of like the Russian dead hand philosophy for missile systems. We as a nation won't go there just letting you know, just saying. But other nations might. And that became, I think a strong enough dialogue because computing processing got to the point where it could be that people would make a machine decision without having someone in the loop. Right. A meaningful presence. And so that triggered this commission, the evolution of computing processing, getting to the point where possibility happened. Then suddenly there was an interest. That's my guess, to be honest with you, I have no idea. It could be serendipity, but it got there and here we are looking at it differently. Even though like you say, decades, decades working.

Yeah, I have a feeling part of it is the massive quantities of data. Maybe Moore's Law has a piece of it too. The ubiquitous processors everywhere. But yeah, something was different. Now speaking of history, I was just reading the other day in Wired magazine. Their first issue came out, 1993 and it's talking about artificial intelligence. So they, they've been doing this for a while too. Their current issue is talking about Ukraine and the use of some artificial intelligence technologies there, including they specifically call one out Primer Primer. AI Primer is widely known in intel circles as a company that can understand lots of text and put sense on it. And what Wired was writing about them is that they are using Primer to listen to the voice recordings that are being posted on the Internet made of Russian command control. A lot of that's being intercepted and is then being recorded and post it online. So Primer's not doing the collecting, they're just pulling that off the Internet. And then they put AI on that, and they remove a lot of noise, they figure out what the guys are talking about and where they are, and then they use that to support operational decisions. Now, there's a lot they don't say in the Wired article. I have no idea how tightly controlled or connected they are to weapons systems, for example, but you could use your imagination, and this is getting really close, where AI is going to be taking direct data and then getting it to someone with a weapon to take some action and make a decision. So it's the reality that we've all been talking about for several years is here and being exercised in Ukraine. And I just thought I'd mention that and see if this is the kind of thing you guys were envisioning on the commission.

So to me, that's kind of like a couple decades old. So when we have a Tomahawk missile on a ship, which we've had for three decades, we have a two point state vector in its path before it goes where we want it to go. We collect radar data, we collect intercept information. In other words, we logically, mathematically, if you would determine what the algorithm is, is to send it to a place and we update the system at least twice on its path. Right. We do that today. We've done that for three decades. This technology is here today and it has been here. The attention to it is more important. I think that you said, which is somebody stepping away from the console that hits the go button. That part as a nation, we aren't inclined to do. We will take information in, but we have, as a human, the interest in putting us into the loop of that decision. We do not and will probably never have, in my estimation, although I could be somebody later quoted. I don't believe personally that we have the will or the intent to remove ourselves from a decision when it comes to a kinetic act that is, that's going to harm somebody. Right, Makes sense.

Katrina, I want to bring a few folks online on camera to ask a question or two, if that's all right, please. One is Colin Agee. Do you know Colin?

Yes, I do. Hi, Colin.

Okay, let me see if I can promote him here to panelists.

And.

All right, Colin, you should be able to turn your camera on here in just a second.

Trying to connect.

Oh, you're getting on.

Can you hear me, guys?

Yep, can hear you.

Yeah. So, okay, you know, I'm at nga, where we're, ironically enough, we're not allowed to have cameras. So I'm on. I'm on voice only. So, Katharine, I Want to play a bit of devil's advocate here on the human in the loop. And I'm actually plagiarizing and I suspect you may have heard this because this used to be the. Whenever Drew Cukor did a speech about Maven, he started with this and he talked about self driving vehicles and he talked about that accident, I think it was in Arizona where the pedestrian and got hit and they shut down the whole program. And so he points out that there's 27 people killed in auto accidents, 365 days a year. So my question is, if we know how fallible humans are, why would we impose a standard of perfection on the machines? I mean, if you can empirically prove that the machines are better than the humans, isn't it immoral to keep a human and his propensity for error in the loop?

So humans actually are far less accurate than computers. But humans did the programming. So which human are you referring to?

And that's a great point. Yes, right. But you can incrementally improve. And this is kind of where machine learning comes in. You can refine the process. Process to get beyond what is organically within the capability of humans. I mean, look what's going on over. So a huge concern in the application of lethal force is collateral damage. Well, geez, look what's happening over in Ukraine right now. This isn't collateral damage. This is deliberate. So I don't know, I just think morally, ethically, I don't know, we're a little humanistically arrogant to think that we need to keep humans in the loop.

So English is not my best language. So forgive me if I'm not as clear as I'd like to be. We will not as a nation, I do not believe that we will allow a computer to conduct its own programming to do something that's lethal. We will require somebody to, to check the math. And that somebody could be another computer program that a different human wrote. The issue on the table isn't that we don't already have systems that we have that are autonomous. What the issue is, is the future debate is about machine programming their own language and real time. Okay, the grayness. And humans are nonlineal thinkers. Computers are not. Computer processing has evolved such that we can have the speeds that are approximate to nonlineal thinking. But then there's experiential. One of the reasons why we have a problem with the ability. And maybe this will be overcome in the future. I'm not so naive to think that there's something that won't change this but right now I bring an experience and I have a nuance I can detect things based on experience that computers do simply only through what's been programmed. And if I take a simply programmed computer and I reprogram or allow it to continue programming further beyond what I have done, I can run into problems. So at this point we do this as a risk based approach. I'm hoping this makes sense. But to me when I say meaningful human in the loop I'm not talking that I'm not going to with a pre programmed like I said with the Tomahawk agreement that we have tested and validated does what I ask it to do. I do not think that I will have at least in the current terms of what the technology space is the I don't think as a nation we will allow computers to conduct their own programming.

One step below that and then Colin, let me know if I'm getting off the subject you were asking about but the one step below that I mean is could what about a weapon that is launched programmed by a human and that program says loiter over here. If you see a vehicle on any of these five roads, destroy it. So that's human programmed write its own program and it's launched. But it's AI and it's.

I think those things happen right now I typically have a screen up because I'm going to watch to make sure that it's not severely civilian collateral. But you know, in the terms of war things the system is going to perform to protect but it's a human. So if I launch a system and I've written again humans error and if I write programming that's good for one task doesn't mean it's good for the second task that a computer decides is good. That's where I'm trying to differentiate okay because we are having programming conducted by computers now and the question is to what extent where's that order of what we consider is acceptable? Great. Colin, you have comments?

No, I appreciate that. I'll just share an anecdote and I hope I'm not violating I don't think this was not for attribution or anything but several years ago when we did the AAR for the Army's big Unified Quest exercise which really doesn't exist anymore. And this was when General Milley was the chief of the army before he became the chairman. And so the theme of that particular exercise was AAA Artificial Intelligence Automation and Augmentation. And so he asked a question, he said okay, we've been talking a lot about robots. Can we expect the robots that are involved in warfare in the future to behave ethically? And so I wasn't even part of the ar. I was sitting in the back. But you know, one of the subject matter experts who was sitting in the front row like enthusiastically said yes. And I just couldn't help myself. I came from the back of the room and I inserted myself and I said, you know, when I was a teenager I loved Isaac Asimov too. But you know, the three laws of robotics, that bullshit is fiction and there's no way. So yeah, I don't know. I think Katarina is right. I think that we won't allow this to happen in our country for ethical reasons. And being an intel guy, I always think about the interplay between us and our enemies and I'm just really, really scared that we are going to self impose an asymmetry here, you know, where nanoseconds matter. That that human is going to introduce latency that is going to be fatal to us. So yeah, I think, I think I've said enough and I do appreciate the answer. It was a good. That's a good answer.

Thanks Kong. And I have the same nervous. But I just can't give up my, my ideals yet. Who knows?

Yeah, you know, there was a really bad movie like two years ago about. There was an Android who, who was an 06 in the Army. And then this troublemaker Navy guy, I think he was a lieutenant, got assigned to work for him as punishment. And the Android made a great statement. He said we need the humans to be as humane as the machines. And there had just been a deliberate targeting of civilians in the movie. And I just thought it was a brilliant and in a really nondescript movie that I can't even remember the name because it was so bad, but I just thought that one line was really brilliant.

It's true.

Thanks, Colin. Robert, good seeing you again. And Katrina, have you ever met Robert Stratton?

I don't believe so.

Well, let me just give you some background. This guy is one of the smartest technologists I've ever known. And for decades he did not invent the Internet, but he made it work. And he's also had a career in leadership and investing and venture and is an all around smart guy.

They used to work for Gilman when we were setting up in Qtao.

So.

Thank you very much for doing this. It warms my heart and it's one of these things where you actually brought up certain topics that was very reticent to bring up. Because they're so frustrating to me. I feel like in the policy realm in particular, there's a real challenge. I illustratively, I just discovered something today. I don't know if you heard this, but apparently there turned out to have been a bunch of an actor group, a particular visible one right now, that falsified a bunch of data requests from information services through compromise, like law enforcement agencies. I was actually in the staff meetings for what was called digital telephony before it was called CALEA and warned them about this and nobody wanted to hear it. So I feel like there is this strategic deficit on the part of national leadership that I don't know what to do about. I've tried over the years to do what I can. More recently I found a set of folks that seem to be trying to bite off a big job, and that was in the IEEE where there is a whole new class of policy work, of standards working groups. And this is kind of a new thing for the institute around ethics in design and in particular around AI. And the one that I've been on 7014 is focused on ethics and emulated empathy on autonomous and intelligent systems, which also includes things like nudging, deriving emotional state from biometric data, things like that. One of the things that's become apparent, I have seen at least one US government person participating. But one thing I was wondering is, is how plugged in is from what you know, right now, the US Government actually in terms of work in places like that, because the people I see right now trying to craft to first have the discussion and craft tangible guidance not about what to do so much as how to think about what to do before doing it has been in places like the IEEE lately and it's weird and different and new. But do you see like, does the US Government even have a really good feeling, and I know it's hard to generalize right about where the good work is happening and are they involved or do you feel like there's a much larger education effort that still has to happen?

So I couldn't really speak on whether the government is or isn't anymore because I've been out for five years. I'll just be candid. My work now is pretty much in the front end of the science and technology things. If you had the chance to read my bio, I kind of work like Bob said, on a couple boards. But I've been picky, right? So I can be picky. So I'm on the National Academies of Science. I'm going to Be shifting to the Army Science Board. I'm trying to stay in the front end of things to do exactly that, to find out where the things are. And so do I sign that the government is. I do. In pockets. I don't think it's necessarily. I know this sounds funny, but if I were to reflect backwards, having been in a place of responsibility for what you're talking about, I don't think it's not for the lack of wanting. I think there's so many burdensome prohibitions to having conversations about where things are that you really as a government entity are almost stymied. It's by luck or accident that you get exposed to stuff. I think they're trying to change that. I mean I really do. I've seen certain activities that are very, very interesting. I'll say this. The use of other transaction authority, the use of program intermediary agreements, the stand up of a new element in the Science foundation and in the National Institute of Standards and Technologies. They're revisiting the whole way of creating standards. You probably are aware of this, but standards are created by leadership through what is considered published and China figured that out. And published and published and published but wasn't necessarily of value but by doing so they got a foothold in the standards which is essentially a foundation of where we actually build to. So yeah, there's I would say, you know, again, somewhat fact free analysis. I would say that it isn't necessarily that the government isn't wanting to get access, but the challenge isn't the access. It's the amount of available space on your calendar every day to do this as compared to what you're going to get hung and fried for if you don't do again. I would spend a lot of time trying to get access to information through third parties and was constantly siamyed by the competitive act or by some issue of if I limitations in conversations. If I don't do it with everyone, I can't do it with one. I broke through some of those. But to your point, I would say that there's ample opportunity to improve and I think data collection where it's neutralized and why I'm really interested in the intelligence threat forecasting through STI work and IMD work is an opportunity space because I think that will help us look better.

Katrina, I appreciate that and thank you. Robert, I wanted to ask. We're nearing the end of our time but we have about four more questions. Is your. Would your schedule support go on a little bit longer?

Yeah, yeah, yeah. I Don't see it.

Great, thanks. So next let me promote up Creighton. Creighton Reed. Just a second. Creighton. Let me click your button here and you'll be promoted to panelist. All right, Creighton should be live here.

Hi.

Hi. Very nice to see you. Thank you for the time today. So my question is, where are we along the path of evolution for AI and its adoption across the government? If you want to use the inning are we inning one, inning eight? And what do you see as the next one or two biggest barriers to scale in terms of just things that are ceasing from wider spread adoption?

Hey, Katrina, let me also mention Creighton is with one of these companies that is bringing real AI capabilities to the intelligence community and I'm sure many other places too. But the intel community is one I'm most familiar with. So percipient AI if you're familiar with that company.

Oh, how neat. Very good. Nice to meet you.

Nice to meet you.

So let me characterize this uniquely because there isn't any big whitewash on this. Certain elements are fast adapters. They're fast adapters because they have to. Other areas are so. And again, I don't care if this is for attribution or not, because I'm retired. So what are they going to do? Send me back to Bouquet? We don't need tanks. Sorry, but we're going to build more tanks because Congress tells us to. I would need more AI, I don't need more tanks. But there's only so much budget going around. And as much as the budget looks really big for the US government, the entitlements take almost 90% of that budget. So the real fungible budget is about 10%. And when you consider that and you try to buy Ohio subs, new Space Force Command and next generation combat vehicles, we have a problem. So what we need is more AI then I think foundationally your question is how do we stand up to our competitors globally? I believe we are very much more adept at kinetic AI than any nation on the face of the earth. The use of kinetic AI is extended into the cyber world. So I believe we are very ostensibly leaders in that area. What we're not is the other AI that improves our ability to speak, to answer, improves our ability for logistics, personnel, Systems, Simple Mundane ERPs, Enterprise Something Programs. We are ahead in medical science and the use of AI in biological and pharmaceutical areas is more advanced than most nations. Although we have some very good competition out there. Thankfully most of them are allies at this point. Still, so I can't. We're not, we're not on. Nobody is at home plate. I think we've got a couple infield running. Maybe some people are trying to steal bases and they're, they're maybe able to make so. But until we get the nation understanding and having confidence in it and actually providing for that economic rationale, the incentive, I sort of look at us somewhere between second and third base, sometimes running one way or sometimes running the other way.

Well, just quickly I would echo your thoughts on. We're definitely finding some pockets in the US government that are forward leaning and really want to adopt new technologies like ours and others. And we do run against challenges around budget and trying to find transactional authorities and the ability to move quickly. So thank you. Appreciate it.

You're welcome.

We have a question from Randy Ford. Just a second here. Randy. You probably know Randy well, correctly.

I do. I haven't seen him in a while. Hopefully I get to see him now. Let's see.

All right, here he comes. Hello, Randy.

Hey, good afternoon.

Hi, Randy.

So, Katharine, you, you made a brief reference about the middle of your remarks about cyborgs. And I just wondered if you could go back to that a little bit. Big brains like Elon Musk and others have said. And I heard Elon speaking at a forum where one of the biggest threats of the 21st century is an AI or an Android or however we want to characterize it, exponentially growing to the point where they become a threat to human beings. And so I just wonder, do you think that's overblown? You know, how do we, how should we think about that? I mean, if we're just some crazy tinfoil hat person, but when people like, you know, Bill Gates and Musk and, you know, people of that, of that stature say that, you know, something is a risk, you can't just instantly dismiss it out of hand. So I just wonder, how do you, how do you approach that and how do you think about that?

So I look it back to 10,000 leagues under the Sea. Before submarines were made, somebody imagined a submarine. Now what they imagined we don't have. But we have something like that somewhere that we created in an image that we can live with. I'm not going to discount count a dream. Humans are remarkable. But I will say with a view and having spent some time, I would say I'm pretty much within one year of accuracy. There is a long way to go. We cannot make self driving cars. We have yet to be able to have. Watson, as you probably remember Watson. Or there's a whole slew of these game machines that are doing well, but they don't have the ability to represent what that philosophical discussion is that has been presented. That doesn't mean that we shouldn't think about it. What I like about those folks that send out those big flares is makes people watch it for a while. And why is that important? Well, if there is something that could be as devastating or as horrible as that, that means that minds are going to be thinking about how make sure it doesn't happen. Because many of the things that have happened, have happened in the world, not by choice, but by accident. And let's not have that type of an accident. If it's possible, if it's feasible, it's definitely way off in the future because we have real problems doing what we are doing. I mean, I don't know how many of you know the Institute for Human Machine Cognition down in Florida, but if you want to spend some interesting time learning about people actually trying to do science of this type, go watch to see how far along they are. NASA did that robot, the cyber that was out there. We're a long way from that. Again, I will never, ever doubt that human capacity could do remarkable things. But at this point, I'm glad people are thinking about it because hopefully that prevents the bad side of that from happening and focuses on the good, like the 10,000 leagues under the Sea scenario. But, yeah, not yet. Randy.

All right, thanks. Good to see you.

Next.

Katrina, I want to introduce you to Florian Wolf. Florian. Very smart technologist, business leader. Created a company that actually tracks innovations across multiple industries and then presents that in a way that's useful to decision makers. Did I say that about right, Florian?

Yeah.

Thank you. And thank you for your time for doing this. My question basically is, in my experience, there's often a disconnect between the technical practitioners on the one hand, and decision makers who don't necessarily have a technical background, but who sit on the funding. And how do you mediate between those two worlds? What's your advice there?

So I'm going to separate the real world from the defense world for a moment, because I'm presuming you're talking defense, Right, the federal government.

I haven't even thought about that, but probably both.

Okay. Totally different ones, entirely motivated by profit, and one's entirely motivated by requirements, operational requirements. In the commercial world, you have to bring something that's a technological product or service that can sell. Okay. If it can sell. And you can show how I can sell you'll get support. Okay. Because the commercial world is entirely made or industrial world is entirely made up of people who are trying to pay for employees, make a profit, make their enterprise of value. Right. And if it's a publicly traded company, boy, it's all about those shares. The defense world has a totally different thing. When I was sitting in the chair of the head of acquisition of the Missile Defense Agency, I used to have the saying that if you brought me anti gravity paint and I didn't have a requirement for it, I wouldn't buy it. I'm being serious. Sadly, it's true. I have only so much public funds available to me and so I have to have an operational use for that. Whatever it is that you're bringing to the table that I can either get a requirement written for or have a requirement already on the table for to be able to fund, and then I have to go compete for that money. Totally different world. Totally different world. It's not that the person behind the screen doesn't know or understand the technology. It's frankly that it's going to be another great thing that I can't get because I don't need it. I, as an engineer, would naturally want all the anti gravity paint I could find, but I would never be able to buy it because I didn't have a requirement for it. Make sense? Yes. Thank you.

I think that's important and can inform strategies of companies that build anti gravity paint. Make sure you're talking to the right office and make sure you're talking to, in that case, Congress or some other decision maker who could buy the anti gravity paint.

Right.

Katrina, can we take one more question?

Okay, hold on a minute. Yeah, go ahead.

Okay. And this is from Daniel Pereira, who is part of our team here at uda. Daniel, you're promoted to panelists. Now.

I want to be demoted.

Hey there.

I chew him in a spot where I. Hi.

Hi.

Do we lose him? Looks like we lost him. Daniel?

Daniel?

Let me check our configuration here. Daniel, are you there?

Hi there.

Yeah, we hear you, Daniel.

Hi, everyone. Yeah, I too am in a place where I can't turn my video. Sorry about that, but thank you for. This has been fantastic because we really were on the brink of taking a look at our AR coverage to date. And the question is, how do we follow up from an OODA loop research and analysis perspective? How do we take. You know, we've looked at the AI accidents framework, we've looked at the commission recommendations, we've looked at some of the stuff that STRATTON was talking about, he turned me onto the IEEE framework. And on top of that, there's all kinds of industry verticals that are doing different levels of creations of frameworks on different levels of concern for ethical deployment and some with no concern for national security, because that's just not their consideration. So how best do we cycle through this information, all the different initiatives, and create insights for our membership, particularly in the private sector, in different verticals?

That may be a very hard question, Katrina, but would appreciate any insights at all and how we could do a better job at OODA in informing our network on AI and what they should be doing.

So one of the things that I enjoy about reading your packages, which I do, by the way, is that you're getting access to information and articles that I can consume that I can help shape my thinking about AI. So I would advocate that you're probably already on the right path. What you might want to do is see if there's tendencies, use analytics, see if there's a pathway that you're starting to see that's emerging. What's interesting about nature is it's, it's, you know, entropic, it spreads out, it goes in every direction. And interestingly, one of the things that I like doing when I'm reading these articles is seeing if there's any trending. And I'm not looking for big, big trends, I'm looking for little trends, things that align, the things that are moving faster, something that people are starting to follow, something that is, you know, you know, something that catches your attention, that seems to be catching everybody's attention or is catching traction as a product overseas or here. Those things are very, very meaningful. Artificial intelligence can be used to help us in our ability to track and develop defensive and offensive postures to the furthering of the digital environment. And you guys are very, very capable in doing that and helping your customers see those things or your consumers, if you would.

Well, thank you, Katrina. Thanks for that advice and guidance. Extremely helpful. And thanks for the time today and the insights into artificial intelligence. Really appreciate it.

Yeah, take care of yourselves. Thanks for listening to this OODA loop production. For the latest analysis on cybersecurity technology and global global risks, please visit www.oodaloop.com.