How the U.S. military thinks about AI

Bandwidth for Changelog is provided by Fastly. Learn more@fastly.com we move fast and fix things here at Changelog because of Rollbar. Check them out@rollbar.com and we're hosted on Linode Cloud Servers. Head to linode.com changelog.

This episode is brought to you by DigitalOcean. DigitalOcean provides worry free database hosting with their managed databases. If you need to get data in and out of Postgres, MySQL or Redis, call on the world class support teams at DigitalOcean and stop wasting time on setup, backup and maintenance. Get simple predictable pricing. Get detailed documentation. Get up and running in minutes so you can get on with your business. What are you waiting for? Head to.cochangelogged again. That's.cochangelogged. Foreign.

Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show@changelog.com community follow us on Twitter. We're at Practical AIFM and now onto the show.

Welcome to another episode of the Practical AI Podcast. I am Chris Benson. I am a Principal AI Strategist at Lockheed Martin and with me, as always, is my co host Daniel Whitenack who is a Data Scientist with SIL International. How's it going today Daniel? It's going pretty good.

Got a bit of a cold but not too bad and doing some pretty interesting stuff with text to Speech this week, so that's pretty fun.

Really? Anything worth sharing or are you gonna wait till later?

Well, I think there'll be some things revealed at the Voice conference which we'll both be at in January. Yep, so I'll leave you in anticipation until then. But it involves lots of speech data and some text to speech in local languages, so.

And for any listeners who happen to be in the Chattanooga, Tennessee area north of Atlanta and want to catch Daniel and I, we're going to give a talk at the Project Voice Conference and we're also going to be recording some episodes there. So we would love to see you there. If you happen to listen to us, come up and say hi. Well, I am pretty excited about today. We have a guest that I already have quite a bit of knowledge about. We have some work between our company and his organization. With us today is Greg Allen, who is the Chief of Strategy and communications at the U.S. department of Defense's relatively new Joint Artificial Intelligence Center. Welcome to the show, Greg.

Hey, thanks for having me yeah, I'm.

Pretty excited about it. You guys do some super cool stuff, and so I'm really happy to share that with our listeners. I've been looking forward to doing this for a while. I guess if you could just start us off kind of telling us a bit about yourself and your background, kind of how you got to this point, and then after that, we'll dive into what the Joint Artificial Intelligence center does.

Sure. Well, as you said, I'm the Chief of Strategy and Communications here at the Joint AI center, and I came here from a variety of backgrounds. Really. My primary professional experience has been in corporate strategy work, and I did that both as a management consultant and then also working in the corporate strategy offices of a variety of commercial technology companies. But after a while, I was working on some AI strategy projects and ended up doing a piece of work for the Intelligence Advanced Research Projects Activity. That organization is called iarpa. If you're familiar with darpa, this is the sister agency that covers the intelligence community of darpa. And my report there was titled Artificial Intelligence and National Security. And it was essentially going through the different aspects of AI technology, how it was changing over time, and elaborating on what the likely implications of that would be for different areas of national security.

What year was this around in terms of how things were developing?

Yeah, this report came out in 2017, and I was working on it for the entire preceding year.

Gotcha.

So if you're familiar with the Obama administration, White House's reports on artificial intelligence that came out in October 2016, sort of the origin of this was that there was an additional report that they talked about writing on AI and national security, but ended up punting on it and deciding, you know, oh, somebody will get to that later, we should focus more on the workforce and the economic and the research and development impacts of AI. And so iarpa, the head of IARPA at the time, Jason Matheny, asked me, well, rather than wait, why don't we have you, as sort of an outsider, take a first stab at this report that was not written. And so it was on behalf of iarpa, but it was ultimately published through the Harvard Belfer center for Science and International Affairs. And after that report came out, there was a great deal of interest from the US national security community. And I ended up joining the center for a New American Security, which is a think tank in the Washington, D.C. area, and doing a lot of analysis and sort of pro bono advisory work to the Department of Defense. And so when they ended up standing up The Joint Artificial Intelligence center, the individuals who were standing up that organization, asked me if I would be willing to help out. And of course, I jumped at the chance. It was a very exciting opportunity, very cool.

So I guess. And just as to call out a couple of acronyms, I know you described iarpa. You also referenced that by talking about kind of the intelligence side from darpa. Listeners who are not familiar with it may remember that DARPA is the agency, Defense Advanced Research Projects Agency, if I got that right, that started the Internet. So if you're not familiar with this already, that's probably where you would have heard of DARPA and iarpa, its sister agency for defense. So I guess at this point, if you could kind of, you know, I know the jake, and I should say Jake refers to where the center that you're at, the Joint AI center at the Department of Defense, and we call it the JAKE for short. So I know it is relatively new. You guys just came into existence in the, I think middle to late part of the year in 2018. If you could kind of start off by telling us what is the JAKE and, you know, kind of describe its mission and its budget and, you know, why is it there?

Sure. So the Joint AI center was established as the Department of Defense's center of Excellence for AI Technology. It was officially stood up in parallel with the release of the Department of Defense's AI strategy. So that document was released in the middle of 2018, and it was released publicly in the unclassified summary in February of 2019. So a bit of a gap between when the strategy was finished and when it was released publicly. But ultimately it's out there. You can read it on the Internet. And so the DoD AI strategy that came out in the summer of 2018, what it says is that the Department of Defense recognizes the strategic importance of AI technology. And it also says that the Department of Defense wants to pursue advances in AI technology, basically for military advantage. So the basic reason that we're interested in most technologies, and that document said that we were going to create a new organization called the Joint AI center, which would be the focal point of the implementation and execution of the DoD's AI strategy. So maybe I'll start by talking a bit about the DoD AI strategy, and then I can get into the specifics about what the JAKE is doing to make good on that strategy.

That sounds fine.

The DoD AI strategy has five pillars. These are deliver AI capabilities for mission impact, scale AI's impact through a common foundation, cultivate a leading AI workforce. Engage commercial industry, academia and international allies and partners, and lead in military ethics and AI safety. What I just described, those are the sort of five key pillars of the DoD AI strategy. The Jake is the focal point for each of those pillars. But we're especially interested in, at least in the near term, we have been especially interested in delivering AI capabilities for mission impact. So, and this is a little bit of how we are structured as an organization, it's quite common in commercial industry for AI organizations to separate into who are the people who are actually developing and implementing and executing AI capabilities and then who are the folks who are building the infrastructure, the platforms, the tools to enable that other group. So there's a separation between your data scientists and your data engineers, and similarly a separation between your sort of AI capability developers and your AI infrastructure and platform developers. And that is a distinction that is recognized in the DoD AI strategy and is also recognized in the organization organizational structure of the jake. So we have our mission initiatives, our national mission initiatives. These are specific projects that we're going after because we believe that they have a lot of the features that allow for success in AI development. So some of these projects are, I'll just list off a few, humanitarian assistance and disaster relief, predictive maintenance, cybersecurity, intelligent business automation, warfighter health and joint warfighting. And the sort of common approach to how we identify projects and select them to go after them are what are the features that we believe lend themselves to success in developing AI capabilities? So this is areas such as is there a relevant data set that you can use for training your machine learning algorithms and do you have access to that data? Is it of superior? The second thing that we're looking for is is there mature AI technology in commercial industry or academia that could be used against this problem? The third, and perhaps most importantly is would there be mission impact if you did succeed at building this thing that you are setting out to build, Would anybody actually care? And then the fourth criteria is, do we have access to end user partner organizations to who would be willing to test our capabilities as they're being developed in an iterative basis? User feedback is critical for any software development effort. But we believe that iterative user feedback is especially critical for success in developing AI programs. So if you have those four criteria as a recipe for success, then you have a shot at being one of our mission initiatives. And those are the ones that we're going after right now. The sort of second big chunk of our organization is related to, as I mentioned before, infrastructure and platforms. And so our Infrastructure and platform team is developing what we're calling the Joint Common Foundation. And this is an environment that lowers the barriers to entry to develop machine learning and AI capabilities in the Department of Defense. As you can imagine, the Department of Defense is a pretty significant target for hacking and adversarial intent, just in general. So when we're developing software, it's critical that we do that in a secure environment. But at the same time, a lot of these security processes can sort of slow down the development of software. So that's why we're developing the Joint Common Foundation. This is an infrastructure environment that has pre cyber hardened tools that are the same sort of machine learning development frameworks that you might want to use in commercial industry, but they're adapted to be compatible with DoD cybersecurity policies. They're containerized so that they run in cloud environments at reasonably high performance levels. And that combination allows you to develop software quickly and get it into the hands of users and testers in a reasonable time frame, but also do so in a way that takes into account the very significant cybersecurity security risks that any significant DoD undertaking faces.

Yeah, I have a question there around that sort of flow from what frameworks and software is used in industry to what you have to develop on. Are you involved very closely in, for example, the TensorFlow PyTorch communities and kind of you have your own versions of those things that you run internally, or is it more taking frameworks like that and wrapping them up around like middleware and other things that give the protection that's needed?

Yeah, so I don't want to get into too much of the secret sauce for security reasons, but the basic thinking there is that open source tools are popular for a reason. Right. They work well, they have been tested by users who have pretty significant requirements and needs. And we want to make sure that when you're developing machine learning software in the DoD environment, we don't want you to have to use baby software or software that has been largely disabled because so many of the features were not approved. So the goal very much is to give our communities of developers access to the same types of things that they would be using if they were doing so in a commercial industry environment, but doing so in a way that gives us confidence about the security of that operation. So there are a few in the commercial world. A lot of what we talk about in software development is what's called DevOps development operations and the sort of seamless figure eight loop between those two where you're constantly adding features Deploying those features, gaining feedback from your users on how those features are used, which informs how you might want to modify them or come to with a new batch of features or overhaul the software in more significant ways. In the Department of Defense, we have sort of adapted that paradigm to DevSecOps, which sort of recognizes the different security requirements that you might have at each stage, the development and operations process. And so for the machine learning world, some of this is relatively uncharted territory, which makes it exciting. But we're finding that a lot of the value that we can add is I'm just taking, you know, commercial and academic and open source tools and adapting them to the, to the national security use case.

Gotcha. One of the things I'm kind of curious about is, you know, the very first word is joint in the organization's title and you end up, as you described, already working with all these different partners. Some are from industry, like the company I work for. Some are academic, various universities that are doing great work in AI. Obviously you work with the various branches of the military and they obviously have their own initiatives in AI. Could you talk a little bit about what those different types of interactions look like, both with industry, how you support it, what you're asking from it, as well with the academic, and also what is the division of responsibility that the Jake has with the various service specific laboratories that do work in AI, their own specific missions? What do all those different relationships look like to you?

Sure. So starting with the last part of your question, which is how we interact with the service laboratories, and I'll add to the service laboratories also an organization like darpa. In general, these laboratories, the Naval Research Laboratory, the Air Force Research Laboratory, the Army Research Laboratory and so on, they are primarily focused more on advancing the state of the art in AI and dealing with situations where the sort of existing state of the art is not a good fit for military requirements. So that's a very important job, but it's a bit different from the job of the Jake. We are interested in problems that we can go after where technology that is available in commercial, industry or academia today, so sort of state of the art as it currently exists, is a reasonably good fit for military needs. DARPA and the service laboratories are focused more on areas some additional research and development is needed in order to get the technology to a level of maturity where it can be useful for DoD needs. So the way we sort of view it is if your problem is the kind of problem that can be solved in zero to five years, you're probably A better fit for the Jake. If your problem is the kind of problem where it might require five to 20 years of research and development to be solved, then it's probably a better fit for the service laboratories in darpa. And that's not sort of a perfect summary. Of course, each of the service labs and DARPA also work on some near term projects as needed by necessity, or some specific competencies and skills they might have. But to sort of a first approximation, that kind of characterizes the division of labor between the two organizations. And the fact is, DARPA and the service laboratories have been doing amazing work and continue to do amazing work, but the JAKE was kind of stood up to solve a different problem. So not the problem of advancing the state of the art, but the problem of adopting the state of the art as it exists in commercial industry and academia. Which gets to the second part of your question, right? How do we engage with commercial industry and academia? And the obvious answer is early and often we have, I would say, a sort of unusually aggressive, you know, outreach program. We have one staff member who is co located with the Defense Innovation Unit in the San Francisco Bay Area. And full time his job is to sort of go out, meet with companies, meet with venture capitalists, identify, you know, what is going on in the tech space that might be relevant to the types of problems that the military has and the types of problems that the Jake is either trying to tackle or might be interested in trying to tackle in the future. And the organization that he's co located with that I mentioned, the Defense Innovation Unit or diu, is similarly. That's an organization whose primary priority is improving the Department of Defense's relationship with commercial industry and the commercial technology industry especially. So there's a variety of sort of different contracting mechanisms that are designed to make it a little bit easier to do business with. The Department of Defense folks who have not been paying close attention would probably remember that it's often very lengthy and process intensive and there's a lot of bureaucracy to do business with the Department of Defense. And that's very much something that the DoD has been working on quite intensely to reform. And so DIU in particular has pioneered the use of some not new, but comparatively unfamiliar contracting and acquisition techniques, such as other transaction agreements. And these allow you to sort of get on contract quicker, get money flowing quicker, and actually start doing work quicker. And to that, the JAKE has also been an early adopter of commercial solutions openings, which is sort of another contracting mechanism. And the important feature of these mechanisms is again, it makes it easier to do business with commercial technology companies, and it makes it easier to do business with sort of smaller companies who can't always afford the overhead required to do business with the Department of Defense for our bidding process and our proposal writing process.

I'm coming at this conversation as someone who is not deeply involved in the defense world, but one of the things that was kind of coming to my mind as we entered into the conversation was I was wondering how much of the strategy that that you're putting into this, that the DoD is doing is driven by what other maybe potential adversaries are doing in AI and like, what is the landscape of AI and defense look like around the world? And how has that impacted the priority we put onto it? And then like, how we go about, you know, developing that technology?

Sure. I mean, I would start by saying that the two countries are specifically named by the National Defense Strategy, which came out in 2018. Two countries are specifically named as strategic competitors, and that is Russia and China. And so these are countries who have interests that are identified as being contrary and in contradiction to, in many cases, the interests of the United States and also who have oriented their national security establishments in competition with the United States. And that's not a very surprising statement, I would say, to most. I mean, when the National Defense Strategy came out, it was sort of putting on paper the sorts of things that a lot of United States leaders had been saying. And frankly, that a lot of leaders in the two countries I just mentioned one were also sane. So that's sort of basic backdrop of strategic competition into artificial intelligence. Of course, this is the national security world we're talking about and the military we're talking about. So we remain quite interested in what is going on around the world, and we would be silly not to be paying attention to that, I think, speaking about China and Russia each in turn. China's AI strategy, which came out in 2017, specifically identifies that they see AI as a transformation formative technology in many different areas, including in national security. And it also identifies AI as a leapfrog technology. The term leapfrog is interesting in this use case because it is described elsewhere by Chinese military thinkers and strategists as really sort of describing their belief about what AI technology will enable their military compared to the United States military. So if you can think about the example of cellular telecommunications infrastructure in developing countries, notably many developing countries in Africa, this is the canonical example of a leapfrog technology. There were many developing countries in Africa who did not have well built out landline telephone infrastructure. And yet this was no disadvantage whatsoever in adopting cellular telephone infrastructure infrastructure. They just skipped the development step of landline telephones and went straight to cell phones. And that skipping is referred to as leapfrogging. And in our competition with China in military technology, there are many things that we are quite good at that they have a very hard time with, in a technological sense. Things like jet engines, things like aircraft carriers. These are really tough technologies, really complicated technologies that we are, are, as a country, tend to be quite good at, and that China as a country has historically had a lot of difficulty with. And so when they write about AI technology, they're saying, well, if we could really develop an interesting advantage in AI, perhaps we could leapfrog the United States. Which is to say, perhaps we would not have to catch up to them in aircraft carriers or catch up to them in jet engines, because we will shift the basis of competition. And they write, you know, Chinese military thinkers often write quite optimistically about China's opportunity to compete with the United States technologically in these terms. So, you know, we would be remiss if we were not paying attention to that. The second thing I will say is that as pointed out by the Secretary of Defense at the National Security Commission on AI in his speech at that commission, you know, there are many Chinese weapons manufacturers who are currently selling on international markets weapon systems advertised as being autonomous, meaning they can sort of make their own decisions and act independently, and also having that full combat autonomy, meaning they can actually be responsible for the use of lethal force. And so that's what China is sort of up to today, at least in terms of what they're advertising on the international market. Russia is similarly very interested in AI technology. One quote that everybody really paid attention to was In September of 2017, when Vladimir Putin said that whoever leads in AI technology will be the ruler of the world. And I think Russia does not have a very clear path to leading in AI technology. Whereas the United States and China regularly top the lists of who is publishing the most AI research papers annually and who is publishing the best AI research papers annually and similarly lead in measurements about who is attracting the most venture capital for AI companies, Russia is pretty low on all of the rankings that I just mentioned. So I don't think that Russia has a clear path to leading in AI technology. Unfortunately, they do have a reasonably clear path to leading in the weaponization of AI technology. I think this is a pretty similar story to the Internet. Russia was not a leader in any of the foundational technologies for computer networking or the Internet. And yet nevertheless, Russia developed a very advanced and broad and deep cyber capability, military cyber capability. And so similarly, I think Russia is looking to be a leader in the weaponization of AI, just as they were a leader in the weaponization of the Internet. In terms of what they're doing, it's a lot of what you would expect, the social media disinformation campaigns and, and influence operations that Russia has been in the news a lot for lately. They are also interested in bringing more advanced machine learning and AI capabilities to these operations. And then secondarily, combat robotics is an area that Russia has devoted a lot of investment and has shown a lot of interest. And they're experimenting with a lot of their military robotic systems operationally, literally, you know, using some of these systems systems in Syria. So both Russia and China are moving out aggressively to incorporate AI capabilities into their military. I would say in terms of the United States response, our intent is to lead the world in the military use of AI for the benefit of United States national security. I don't think we're so much obsessing over each individual capability that comes out of China or Russia so much as we are sort of looking at the broader landscape. I think AI is a general purpose technology the way electricity is a general purpose technology. It's useful for things like radio, but it's also useful for light bulbs. Similarly, computers are useful for just about everything. And we similarly think that AI ultimately has a lot of application specific opportunities for us. So we're interested in leading an AI in the broad sense.

And would I be correct in saying then that of course there's certain things that even me not being involved in the defense industry at all, like I'm aware of, for example, China using facial recognition technology and Russia interfering in elections and doing behavioral type of stuff like you talked about. And so would I be correct in saying that there's kind of for every different type of AI model you can think of, whether that's, you know, natural language processing, computer vision, object recognition, trend analysis, forecasting, there's, there's probably all sorts, because this is general purpose, like you say, there's probably a way to weaponize all sorts of those types of models. It's just not that, you know, facial recognition is, is an issue. It's, you know, a whole varied range of things that could be at play.

Yeah, I think machine learning is just a new way to create software. You know, traditionally software required, you know, every line of code to be typed out by human hands, but suddenly there's been a vast improvement in this field of artificial intelligence known as machine learning, from which, as I'm sure most of your listeners already know, you know, from which we can program, or to a certain extent, the system can program itself based on what it learns from data. And so that general truth about the rise in machine learning software, well, that's useful for most of the places where software is useful, which is to say absolutely everything, whether that everything is a missile guidance system or sort of back office applications like word processing. I think over the long term, and I do want to emphasize long term, because while some of this transformation will be quick, some of it won't over the long term, we expect machine learning AI to be useful for just about everything, right from the most backwater parts of the back office to the front lines of the battlefield.

So one of the things that you were talking about a moment ago was kind of our adversary's perspective on they are kind of advertising AI in a lethal context and stuff. I'm assuming that the U.S. department of Defense has a policy out there regarding how we think about AI being incorporated into lethal force scenarios. Can you talk about that? What is the American viewpoint on that question?

Well, sure. I think the first thing I would point out is that the Department of Defense abides by the law of war. And for folks who are not familiar, you know, with the national security context, the law of war might not be familiar to them. These are the, you know, literally legally binding principles that are enshrined in documents such as the Geneva Convention, but also have flow down requirements to DoD policy, DoD training, rules of engagement in different scenarios. So the overarching thing which guides all, you know, U.S. conduct and military, military operations is the principles underlying the law of war. And these principles really get to when is the use of force appropriate, when is it acceptable? And let me just sort of list some of the key principles underlying that military necessity. Right. Was your use of force the only way that you could achieve the objective that you had to achieve? Proportionality? If somebody slapped you in the face, did you respond with a nuclear weapon? Well, they. Then that would not be a proportional response distinction. Right. Did you make every effort to prevent harming civilians and only harming enemy military combatants, humanity? Did you use any means that violate the principle of humanity? And of course, abiding by the military principle of honor. So those ethical principles, the Department of Defense has been guided by those, and these are actually legally binding. You know, you could be court martialed for failing to abide by these principles, but those have been guiding the Department of Defense for decades now. To that, there are two things that are comparatively new. The first is the Department of Defense Directive 3000.09, which relates to the use of autonomy in weapons systems. And that was a policy that was released in 2012 and was widely praised at the time. And that policy was renewed and essentially made permanent in 2017. Basically, neither Russia nor China have any kind of policy comparable to 3,000.09, which I think says a lot, right? And then there's also what came out just recently, which is the Defense Innovation Board Board's principles for the Ethical Use of AI.

This episode is brought to you by Brave. We deserve a better Internet. That's why the team behind Brave reimagined what a browser could be. Brave is like Chrome, the good parts, even your extensions will just work. It has built in add and tracker blocking, easy anonymous optimization with the Tor network, earn tokens while you browse and use them to tip your favorite creators. And did I mention is lightning fast. Turns out the web is super fast when you remove all the cruft. Download Brave today using the link in the show notes and give tipping a try on changelog.com.

So Greg, I know one of the really hot and important topics these days in AI is kind of the ethical concerns, ethical principles of how you apply AI and what that means. And I know both in the defense industry as well as in lots of different commercial industries, that has become a big thing. And you know, a while back we saw Google and Microsoft and other large organizations releasing kind of public principles and frameworks about how they thought about this. And I know in the defense industry there is an organization made up of a lot of experts from outside the defense establishment itself. It's called the Defense Innovation Board. And they had a fairly substantial conversation around AI ethics and principles and recently released a document that covered a lot of that. And I am told that you were pretty involved in that process, as was the Jake at large and that. And I was wondering if you could kind of talk a little bit about your involvement, what it means to you, and actually go through what those principles that you guys worked on are and how that relates back to the Jake and what the DoD will be doing going forward.

Sure. So the Defense Innovation Board is a federally appointed advisory committee. So while they are not actually part of the Department of Defense, the federal government has sort of given them an official relationship whereby they can advise the Department of Defense. And when we released the DoD AI strategy in summer of 2018, it was immediately obvious to us that the Ethical considerations of AI technology was something that we wanted to take very seriously and that we wanted to understand very thoroughly. So the Secretary of Defense asked the Defense Innovation Board to run a study about the ethical implications of AI and recommendations for principles for the Department of Defense's use of AI technology. They ran a very robust process. They were working on this for 15 months. They held three public forums at leading universities in the United States. They ran multiple rounds of open calls for comment on the subject from expert communities and also just the general public. And they ended up talking to hundreds of not just leading AI researchers and leading AI industry types, but also ethicists, lawyers, and folks with perspectives from a lot of different industries and walks of life. So this was a lengthy, robust process. And the report that they released on October 31, 2019, reflects their best judgment at the intersection of ethical obligations, the requirements of the domain of national security and the mission of the Department of Defense, and then the nuances of AI technology. And so they are an independent organization helping out the Department of Defense on these issues. So I was not involved in the actual writing of these principles. My only involvement was giving advice on how they could structure this to maximize the benefit from the Department of Defense's perspective as a consumer. So helping them craft actionable recommendations and thinking through, you know, how to write their recommendations in a way that would actually be compatible with the DoD bureaucracy and the DoD process. But what came back ended up being really quite substantive and something that we take seriously here in the Department, as now it falls to us to implement them. And also something that our allies, whether that be in Europe or in Asia or elsewhere, our allies have shown a great deal of interest in these principles as helping think through how do we recognize the legitimate ethical concerns that the rise of AI technology raises while also being able to move forward, forward and use this technology in the mission of national security. So maybe I'll just sort of go through each of the principles that the DIB is recommending that the Department of Defense adopt, and we can sort of think about how the DoD thinks about these issues.

Yeah, if you could kind of hit the six and then kind of dive in however you want to the dozen recommendations afterwards and just kind of name what the six principles are and then talk about what those recommendations that came out of that, if that's okay with with you.

Absolutely. So the principles start with responsible. So this is that human beings should exercise appropriate levels of judgment and remain responsible for the development, deployment, and outcomes of AI systems. Second is equitable that DoD should take deliberate steps to avoid unintended bias in development and deployment of combat or non combat AI systems that would inadvertently cause harm to persons. Third, is traceable that DoD's AI engineering discipline should be sufficiently advanced that technical experts possess an appropriate understanding of the technology, the development process and operational methods of AI systems, including transparent and auditable methodologies, data sources, and design, procedure and documentation. Fourth is reliable DoD AI systems should have an explicit, well defined domain of use and the safety, security and robust robustness of such systems should be tested and assured across their entire life cycle within that domain of use. And fifth, and finally, governable DoD AI systems should be designed and engineered to fulfill their intended function while possessing the ability to detect and avoid unintended harm or disruption, and for human or automated disengagement or deactivation of deployed systems that demonstrate unintended escalatory or other behavior.

Yeah, there's a lot of jargon there. I would be interested in hearing kind of how those play out to real sorts of scenarios or maybe the sorts of scenarios that were in people's mind when they were thinking of those.

Sure. I think the overarching two things that I would say about these principles are one, that the Department of Defense's ethical principles related to the use of force are those enshrined in the law of war, military necessity, proportionality, distinction, humanity and honor. That really gets to the ethical question of what is the ethical use of force? So the Dibbs principles are answering a different question, which is, you know, assuming that you are abiding by the ethical principles that govern the use of force, how do you have confidence that your technology will be able to implement your desires? Right. You know, today it is ethical to operate a commercial airline. In 1903, it would not be ethical to operate a commercial airline for passengers because aircraft technology, I'm thinking specifically of, you know, fixed wing propeller aircraft, was not ready, it was not safe enough for you to responsibly offer that on the commercial market. So part of the sort of, you know, the ethics of the technology relate to matching its current use to the maturity of that technology and your processes for understanding that technology and ensuring that it is both robust and safe. And so that that sort of gets to the dibs approach to ethics and how it complements the ethical principles that are enshrined in the law of war. If you say that you're going to have AI systems that abide by the law of war, how do you know that and how do you prove that and how do you have the relevant Processes to ensure that that's sort of what the DIB AI principles are going after, is not what is the right or wrong way to use force. That question is best answered by the principles enshrined in the law of war. But if you are trying to do the right thing, how do you have confidence that your technology is going to allow you to do the right thing? And so that's about having the kinds of testing, evaluation, verification and validation procedures so that we can rigorously test our systems to show that they are going to perform as intended. That gets to the training of your operators so that they know that this is an appropriate time to use this technology. And this is an inappropriate time to use the technology. Keeping with the aircraft analogy before, if your aircraft is only rated to fly at 40,000ft, do not fly it at 80,000ft, right? And so that gets to sort of what is the appropriate domain or mission use case for a technology and only using technologies for applications for which they have been appropriately and rigorously tested. And I think this is really valuable. We see AI ethics and AI safety as intimately interconnected, and that's reflected in our approach. I think one of the other things that the DIBS study very helpfully pointed out is that the Department of Defense is not starting from scratch when it comes to reliability of incredibly complicated technologies operating in circumstances with life and death concepts. The Department of Defense stuff that uses technology in applications that are incredibly complicated, incredibly difficult. And not only that, there's potentially an adversary out there who is trying to make you fail in addition to the sort of inherent difficulties of the technology.

That's a good point.

And we have a lot of experience doing really extraordinary work in that regard. One example that I like to point out is a system called gcaas. This is something that exists in the fighter aircraft that the United States uses. And if you know anything about what it's like to be a fighter pilot, you know that they undergo some really intense acceleration periods. And those acceleration periods can cause the pilot to blackout or to red out. And part of the autopilot of this system, which is, you know, a software. A software system mixed in with hardware developed here in the United States, can automatically detect if the. The pilot has blacked out, and then fly the plane level and straight so they don't crash into the ground, fly the plane level until the pilot regains consciousness, and then transfer control back to the human. I mean, so it's not as though the Department of Defense is starting from scratch here. A lot of the techniques and processes and policies that we have rolled out related to traditional software will serve us very well in an era of AI powered software. And I think with the dibs sort of long form report, this is a 70 page report, goes into quite some depth. A lot of what it's helpful for is pointing out here is where your existing processes kind of already do what you need to be done and here are some areas where AI really does change the game and you need to go through the hard work of improving upon these policies and processes.

Absolutely. Just as a comment to finish that up, I want to note that for the F16, that system you described is actually a Lockheed Martin system. I end up talking about that with folks quite a lot. I really appreciate you kind of diving into that because that's a question that so many people have in terms of the ethical and talking about lethal force policy and how you guys think about it, explaining how it fits into the law of war is very helpful as a framework, I guess as we kind of close out the show, I was wanting to just kind of understand as the Jake is going forward over the next few years and we are in this, this industry that is moving so fast with all of the, the research and the new algorithms and new techniques and the compute that's racing along. How does the Jake look at AI related technologies going forward? What kinds of things draw your attention and make you sit up and go? That's something that has use cases that we particularly care about. What does that look like as you're seeing new things come across your screen?

Sure. I think the sort of first answer is it's much of what you would expect. There have been some really exciting developments in computer vision and natural language processing. These are two areas in which the performance of machine learning systems today is just orders of magnitude better than where we were a decade ago. And I think we still have not actually finished harvesting those gains. There are many parts of the Department of Defense where a lot of people's time and energy is devoted to applications, where machine learning technology is already ready to start helping them do their jobs. And so I think the sort of next couple of years will continue to be focused on adopting the technology that already exists and is already pretty capable and powerful. I think over the longer term there are some really exciting things coming down the pike. Of course, I think the first folks to get to play around with these technologies will be in DARPA and the service laboratories who, as I said, their mandate is a bit more focused on advancing the state of the art in AI technology. But these are things like the increasing relevance of transfer learning. So ML systems, if they learn something in one domain, can that help them not start from scratch as they look at related problems in perhaps different data sets? So, for example, I think all of your listeners are probably familiar with the AlphaGo example. So the AlphaGo system was literally better than the world's best Go player. But that system not only could not play chess, it could not play Go on a different sized Go base board. So some go boards are 16 by 16, some are 32 by 32, et cetera. And so it really had no ability to apply what it had learned about Go and make that a better player for playing chess. And so that transfer learning capability was not present when AlphaGo. There have been some really interesting research results that indicate that researchers are making more and more progress at tackling the transfer learning problem. And so that could be really useful to DoD if we're thinking about analyzing satellite images from a desert environment, you know, might that actually make our algorithms perform better if they are also looking at images, you know, in a seaborne environment, for example? And so that ability to sort of combine what the machine learning system learns from different domains is something that I think is just going to have a ton of benefit for us.

All right, well, Greg, this has been a truly fascinating conversation. Thank you so much for coming onto the practical AI podcast and kind of taking us through how AI integrates into the world of the DoD and defense at large. Really appreciate it. Thank you for coming on.

Well, thanks very much for the opportunity. It was great to talk with you.

All right, thank you for tuning in to this episode of Practically I. If you enjoyed this show, do us a favor. Go on itunes, give us a rating. Go in your podcast app and favorite it. If you are on Twitter or social network, share a link with a friend. Whatever you gotta do, share the show with a friend if you enjoyed it. And bandwidth for changelog is provided by Fastly. Learn more@fastly.com and we catch our errors before our users do here at Changelog because of Rollbar. Check them out@robot.com changelog log and we're hosted on Linode cloud servers and@lino.com changelog check them out. Support this show. This episode is hosted by Daniel White Knack and Chris Benson. The music is by Breakmaster Cylinder and you can find more shows just like this@change law.com when you go there, pop in your email address. Get our weekly email keeping you up to date with the news and podcast for developers in your inbox every single week.

Thanks for.

For tuning in. We'll see you next week.