Dr. Melissa Flagg of the Center for Security and Emerging Technology on the Future of Technology

Informing your decisions with intelligence analysis and insight brought to you by the team@ooda.

Loop.Com hello, I'm Bob Gourley, the Chief Technology officer of OODA LLC. And today on the Oodacast, Dr. Melissa Flagg. Melissa, welcome.

Great to be here. Thanks, Bob.

Hey, I really appreciate the time. And you have such an interesting background. I want to tell our viewers some of it. And starting at the beginning, you have a undergraduate degree in pharmacy. You were going to be a pharmacist.

I do. I have an undergraduate degree from the University of Mississippi. Hotty Toddy. For those of you who know that I grew up in the country. I grew up in Arkansas and Missouri. And for my parents. Right. College was really about a job. So I was very focused on a career that I knew I'd be able to do, really in the places where I was from. But I was lucky because it opened up this whole door, this whole world that I didn't know existed in research. I had never met a Ph.D. i think, growing up.

And your Ph.D. is in chemistry?

It is. I am a natural products chemist. So that is drugs from plants. That's the eight second or the two second elevator speech. I don't know how many of your viewers are old enough to remember the Sean Connery movie Medicine man, but that was sort of my vision of what my life was going to be. I was going to be trekking around in the jungles of Brazil and curing tuberculosis, you know.

I need to ask you how you made the transition from chemistry to government service. But another question I want to ask before that is, do you think that your chemistry training and your pharmacy training had an impact on your mental models and your ability to think through complex things and.

Absolutely, yeah. It's funny, when people ask me about, does having a PhD matter or not, I always say yes, but not because I memorized things, not because people give me a chemistry quiz in the halls of the Pentagon, but because it gave me a scientific method, a way of analyzing information that comes in, a way of thinking about how to question it, and more importantly, a way to understand when I need to change and update my own model, when I. When I need to change my mind based on new information coming in, when I have to say, the experiment didn't work, the hypothesis wasn't proven. Now I need to go back and come up with a new one.

Cool. Yeah. Cause some new data or something changes the hypothesis. So you got your Ph.D. and then it was from there to the Department of State. Is that correct?

Yeah. So I got my PhD and I actually did a brief postdoc at the University of Mississippi because I was really struggling with, okay, I have my PhD, but a lot of things happened that were really challenging in academia and that wasn't going to be my path. So I did a brief postdoc with an old friend of mine, Dr. Alice Clark, who was just an incredible mentor. And she encouraged me to actually think about how to apply science outside of the lab. So I applied for a AAAS Science and Technology Policy Fellowship in dc. I got it. I got a position at the State Department. I was one of the first AAAS Fellows for the science advisor at the State Department under first Madeline Albright and then Colin Powell. And I was going to work on sustainable development in Africa and in South America. And I had all of these visions of greatness. And I arrive in D.C. and in my first week of orientation, 911 happens. Nobody cared about development after 9 11. And so it was this moment where I'm at the State Department and I have this view of international policy, but everyone is retrenching. And so I was working on how do we keep the Department of Defense from locking down visas on all international scientists. Sound familiar? I was working on how to actually work with scientists to understand how to really think about their research and think about necessary collaboration and to think about how to talk to policymakers about their research and explain it. So I spent all my time working on security when I literally had never even heard of that, I think before I arrived in Washington.

Right. And from there, what did you do after that period at the State Department?

I had an incredible opportunity. I think that maybe the Navy both wanted me. They kind of would rather have me on their team than against me because I spent a lot of time saying no in meetings at the State Department. But also I helped out a technical director for the Office of Naval Research Global in London on some of their European efforts when I was at the State Department. And so when he heard that I was looking around and thinking about where I would go next, he offered me the opportunity to go to London and work for the Navy. And that just seemed like the most exciting, incredible opportunity for a 30 year old ever. So I literally, without really a clear understanding of my job description, got on a plane and moved to London in January of 2004.

Man. Fascinating. So there's the Office of Naval Research and a Navy Research Lab. Is that a different thing?

So the Office of Naval Research Global was actually created just before the Office of Naval Research. That was like 1948, 1949. And it was created post World War II when we realized we needed a better understanding of global st. So we created these outposts in London and in Tokyo first. And we now have them in Singapore, I think in Chile as well still. And we did that so that we could spur collaboration with international scientists, try to pair them up with Americans, but really keep a deep understanding of what that global state of science was. And I had worked on international S and T as a grad student. I'd spent time in Chile and Brazil. I'd worked in global ST at the State Department. So it was a really natural fit. And the Navy was an incredible teacher for me. They sent me out on a ship for five days and really like gave me as a scientist at sea and really gave me a firsthand understanding of what the operational Navy was. I met these 18 year old kids that were about to be deployed to the Gulf. It gave me such an incredible sense of mission and what my work was really about. And I've really never forgotten it.

Yeah, that's great. You know, one of my favorite authors, Nassim Nicholas Taleb, talks about leaders needing skin in the game. And it sounds like that gave you the kind of skin in the game because yes, you have the technical background, but now you're talking and working with the real operators and applying your technical talents there.

Absolutely. And operators, really at the beginning of a long war. Right. This was a moment where I knew all of those kids probably won't come home. Right. In 10 years, many of those kids won't be back and how many times will they deploy and how many wars will they see? And I really viewed my work as twofold. And one is to make sure that I give them the best capability to go out, accomplish the mission and come home in one piece. And second, to create a military that is we're the worst case scenario guys in the Department of Defense. We shouldn't be the source of power you use first. But I want to have a military powerful enough that nobody really wants to go that last step with the U.S.

Yeah, see, and Melissa, I want to get back to that point when we start talking about CSET in a minute because you just hit on something I think we need to explore in much more detail. But before we get there, can I say I think your Navy background also positioned you perfectly for Department of Defense working in OSD atl, because that there you influence broad programs, Correct?

Yes. I was super lucky. Admiral Bill Landay was the Admiral who was the Chief of Naval Research who brought me back from London and gave me this incredibly impossible task to think about global tech awareness. How would we know through the lens of intelligence, counterintelligence, open science, programmatics what was going on in any area of value to the Navy anywhere in the world at any time? He's like, go do that, Melissa. And what an incredible, impossible task to work on. And it's really created the frame for my entire career, that one task from Admiral Landay, and it really opened my first opportunity to go to the Pentagon, where I was the director of a technical intelligence office. And this was in 2009, and big data was still starting, and we were trying to use it, and it was hard and it was exciting, and we sort of failed, and we did some good work and whatever. So I leave government and I go to Chicago to the MacArthur Foundation. I'm like, I'm leaving government. I'm out. I'm going to go do philanthropy. What that gave me was suddenly this domestic focus. I was looking at the best st going on in the United States. I had been internationally focused so long. I'd been focused on threat for so long. Now I'm just submerged in optimism and domestic capability. And it really just gave me this recharge. So when I was given the opportunity to go back as the DASD for research in the end of the Obama administration, it was such a moment of just let me apply all of this vision that I have for harnessing our domestic capability, for really valuing our international partners and really taking absolutely a hard line on our adversaries. To me, these things are not. They are not in conflict. And I felt like this weird sort of conglomerations of jobs that I'd had really led me to that moment, and it was incredibly powerful and incredibly exciting. I think one thing I really want to hammer on here is that we commonly talk about how terrible civil servants are or how incompetent our government is. There are 40,000 scientists and engineers in the Department of Defense, and they do things as crazy as hypersonics to high energy lasers to quantum sensing to synthetic biology. And I love that capability so much, and I value it so much. And I see the same thing in industry and I see the same thing in American academia. And I don't know when we lost our vision for America through a positive lens, because that job, it just filled me with this realization that leaders owe this country a positive vision for America.

Right, man. Fascinating. It really is. And after DoD, you eventually went to army, then?

I did. I went walkabout for a year. So I literally after the election I put all my stuff in storage and I went on a ten month road trip. I crossed the country twice in my Subaru. Plug for Subaru never let me down. I lived in friends houses, I talked to people across the country. I stayed in Utah for about four months and I really talked to people who didn't think like me. In Utah I sat down with a couple of conservative think tanks and said talk to me about science and what you think about federal funding of science. I talked to the Cato Institute, I talked to random people in the La Quinta in the middle of a snowstorm in Billings, Montana. And I really tried to challenge all of the things I'd been taught. And I spent 10 months really just out on the road on my own. And when I came back to the Army Research Lab it was an opportunity to open an open campus site for the Army Research Lab up in Boston and to try out many of the ideas that I had. It turned out that in that 18 month period there was a lot of change happening in the country, there was a lot of change happening in army and it became clear that I had some ideas that I needed to get in writing. So when I was given the opportunity by Jason Matheny and Dewey Murdoch, old friends of mine who had started CSET to come back and be a part of was hard to turn down because it was a chance to get some of these ideas on paper.

I'm tracking CSET and I think a lot of other people are. But would you tell me in your words, what is CSET and what do you guys hope to accomplish?

I think there are two unique aspects about CSAT that draw me to it. And one is it is fully philanthropically funded. So it is an independent voice. They don't look for money. I'm not out getting somebody to pay me to write an idea down, which means my ideas really are my own and my voice is my own. And that's I think incredibly powerful in Washington and really inspired by this kind of rand of the 50s, right where they weren't looking for money, they were really trying to dig into the hard big problems. The second part about CSAT that I find incredibly attractive is they have committed to actual Data. We have 90% of the world's scientific literature, we have patent data that's curated for artificial intelligence. CSET does focus on artificial intelligence, at least in this initial couple of years. I, I should have said that.

Another thing to please. CSET stands for the center for Security and Emerging Technology.

Yes, sorry, it's the D.C. disease. Right. Of acronyms. So we're part of Georgetown University and we are a think tank. But I think these defining characteristics of having this real independence and having this incredible amount of data, we have financial data, we have a survey capability to generate data when we don't have it, we have marketing and sort of news data, scientific literature, patents. It's really incredible the breadth and depth of the data that we have. And we have a really great data team to help us use it and to help us use it with rigor. So for me, it was such just. It's like a playground. Right. I can really dig into the things that I love, which are this sort of intersection of R and D and policy and helping the mission for national security. And for me, national security is securing the nation, not just spending the money that the sort of agencies are given and not controlling the nation and not just beating someone else. It's really about making sure that we come out of this period as a nation with values that we recognize and that we want to secure. And so for me, so much of it is about ensuring, again, back to Those days in 2004 when I stood on that ship deck, this is what the mission is. The mission now is sustaining America and the values that we hold dear, not just America as a set of borders.

Great. And so for CSET people who want more information, I know you guys run a mailing list and I'll capture in our show notes the URL to that so people can sign up for it.

Policy AI. It's very easy. Go to Policy AI. And we're also at CSET georgetown. Edu and all of the research that we've done, I think we've produced in the first year about 47 data briefs and policy briefs and issue briefs, and they're all on the website. And everything we do is openly available and all based on open source data. Great.

So can I ask you some questions about things that you think are important when it comes to security and technology?

Absolutely.

I noticed you guys write a lot about artificial intelligence, and clearly we need more focused rigor on what that is. And one particular aspect that I'm really attracted to is ideas around how we secure our AI. And when I use the term security and AI, I use it in its broadest possible sense. You want AI that's ethical and free from bias, but also transparent and. And understandable. And on top of that, you want stuff that's resistant to adversary attack. Either adversaries attacking your training data or adversaries attacking your models. Or adversaries attempting to deceive your AI. And it seems like there's a need for far more research into that. Can you tell me your views on that particular piece?

Yeah, so I do. I think it's like so many questions, right? Because AI, first of all, is not a widget. It's something that's going to be integrated in like cyber. It's going to be integrated into so many legacy systems. Right. So I think there are aspects of like, first of all, understanding what AI is real, understanding how robust it actually is, and understanding where it's being applied and what potential fragility or sort of backdoors are you putting into those systems when you try to move to be a first adopter. Right. I think that's critical test and evaluation and V and V is a big conversation in this area and I think really legitimate CSET just hosted a conversation this week, actually focusing on that. And one of the comments that came up in the panel session was really just making sure that we understand the difference between there are systems that are machine learning that we sort of lock into place and then we deploy them, but they're not actually learning in the field. And then there are these systems that are going to be coming online that are actual learning systems. And you have to think about certifying those differently. You have to think about sort of T and E test and evaluation and verification and validation, sorry, T and E and V and V differently in those types of systems. So we do have to have nuance in our technical base, but also in our program management base where we really understand what is the actual algorithm for that we're integrating in. The other aspect of this is I don't necessarily think we just need AI research on security for AI, right. We need systems engineering research. Because I don't validate an algorithm necessarily. I validate a guidance system or I validate some other aspect of the overall military system. I'm thinking about this very much from a Department of Defense, see. So I apologize, but it's where most of my examples come from. We actually validate the system and so how do we actually make sure that the testers and the evaluators understand the type of AI that's being added to that system? And how does that change the traditional approach that they normally use for validating that system? And I think there's a lot of nuance in this that goes way beyond the research community. And what happens is you see the research community like, glaze over. They want to talk about, oh, I hacked this thing Or I spoofed the computer vision by putting weird things on my cheeks or whatever. Right, okay, great. But what we care about is does the system work reliably every time? Do I know what it's going to do when I hit the button and turn it on? And do I know that it's going to stop when I tell it to stop? And do I understand the decisions that are being made and that it is doing what I have commanded it to do? Right. The military does not want unpredictable things on the battle space. Right, that's the word. We don't even want our people to be unpredictable. We have a hierarchy for a reason, much less wanting our machines to be unpredictable. And so I think it's just that we have to get past just the tech conversation and we actually have to have this systems engineering conversation.

Right, Great, thanks. That's good context. And I want to try another thesis out on you. At ooda, we run a monthly video session for our members. It's private, it's Chatham House rules. So I don't want to talk about who said this, but we do things like I review what I think the issues are and people give us feedback and they kind of steer our research. And this last month I mentioned that I think one of the top stories this year is the DARPA research into AI for fighter pilots. They called this most recent test Alpha Dogfight. And it was really wonderful. They took AI against AI and found the very best AI fighter pilot controlling a simulated fighter. And then they put it against one of the Air Force's best, a Air Force fighter pilot who flying a simulator. And he lost five out of five times to this AI. And as I discussed this in our monthly meeting as a watershed event, one of our members pushed back on me. Now he happens to have been a former Deputy Under Secretary of Defense and really tracks AI. And he just simply said that, you know, if you are in a dog fight with a peer competitor, you've already screwed up so great having AI there. Please continue to push it. But we need AI to help in other ways. And I think he made a really good point and that's what I wanted to ask you about. Do you see the application of AI in other ways? Like decision making support? Like people who go into these deputies meetings and they're making the big decisions, they come in with just reams of paper and old briefings and read books and is there any way AI can help them make better decisions?

I am in violent agreement with him. This is not. We keep trying to use AI for Things that are really sexy and crazy and right out on the edge. That is not where we need AI. We need AI. Okay, so let's, let's, let's step back a second. AI that we are talking about right now is actually neural net based deep learning, right? There are lots of kinds of AI, there are lots of approaches. We've studied AI for decades. It's not new. We had some breakthroughs that allowed us and some sort of technological infrastructure developments around cloud computing, around the fact that we have companies that have been able to push consumers to produce massive volumes of highly structured data that they could use this sort of neural net based deep learning approach on. And so we were able to make these breakthroughs in, in deep learning also because we were making breakthroughs in compute power and the generation of massive quantities of highly structured data and applying them to the types of decisions that are repetitive. Right? So AI right now, which is actually deep learning is really good for places where you have large amounts of highly structured data making repetitive decisions. Where do we do this? We do this in hr, we do this in contracts, we do this in logistics. We do this quite frankly in frickin budget meetings at the dod, Right? Because we know the exact structure, we know where all the data comes from. It's spread across a known quantity of actors. So of course AI could help us there, right? It could say here's 20 years worth of this data, here's what's changed this year. Why don't we just focus on that? Right? Or here's where the guidance said X and the entire budget plan that you've developed is completely opposed to that. I mean that would be great for leadership to actually say go back and try again. Right. I think there are tons of ways that AI can help. But let's just remember that AI is deep right now is largely neural net based deep learning. It has strengths in a specific area that can be applied today. And we're not using it for those things because we're busy talking about robot dog fights. Like why would we do that?

Great. Melissa would love to get your ideas on other topic areas. Let me just ask. Another hot one these days is quantum based solutions, either quantum computers or quantum encryption. Quantum key distribution. There's talk about what's the net assessment? When will the quantum computer be able to run Shor's algorithm and break any asymmetric encryption? Or when will the, the quantum key distribution be so widespread that no quantum computer can break it? And what's your net assessment on all this quantum stuff?

So I mean, so my, my assessment on like quantum key distribution is I'm a little less worried about it I guess than maybe what people want me to be. For some reasons that I won't really talk about here and for other reasons of just that encryption works while the information is encrypted, but somebody has to put the information in and somebody has to get the information out. So there's always an opportunity to take it. So I don't know, I mean, I'm worried ish, but less than people want me to be. But on quantum computing and specifically on small qubit quantum computing, which is coming much, much more quickly in my opinion than like you know, a quantum computer at your house, which is 50 years away. But small qubit computing is here. And what's interesting is we have a couple of companies out there now and I don't know if I should plug one shamelessly, but we have some small American startups like mit. The Engine is an incubator up at MIT and a venture capital group and they have a company called Zapata Computing that I've worked with when I was at army. And there are a couple of other startups as well that are actually looking at operating systems and algorithm development for these quantum computers. That to me is the money. Because now you're actually saying I don't just have a computer, nice, can't use it if I don't have any actual way to use it. Right. But now if I have an operating system and I have algorithms that are tuned, I can develop special purpose compute in levels of complexity for questions that I've never been able to compute before. I think the opportunity to radically improve and change AI or other things like that by bringing into play this type of special purpose compute power through quantum is fascinating to me. Think about design redesigning outside of what we can imagine because now I can take million parameters and design with them because quantum just allows us just an immense space for compute power. I'm really bullish on quantum and excited about it, but I don't necessarily think about it as like a big general purpose quantum computer. I really think about a few areas of military relevance in my universe that are so complex that could be radically altered with just this almost unfettered compute capability.

Great. You know, and I've also heard, I mean going back to your PhD in chemistry, that the quantum ability to model individual atoms. Right now Google just apparently proved that they could do a tiny one atom, but eventually to model entire molecular processes which will help us better engineer and design manufacturing capability to more efficiently produce the chemicals we need.

Absolutely. And in fact, the chief scientific officer and founder of this quantum computing group, I was talking about, Subhada, he's a chemist and that was really where he started. Right. Is this ability to actually model. People don't appreciate like how little we actually can compute and model at that sort of atomic and then molecular level because it's so complex and the ability to do that, it would fundamentally change materials research and chemical research radically. Right.

Melissa, are there any other technologies we should talk about? What do any. We've talked about AI, we talked about quantum, we talked about security. What is really going to dramatically transform the defense establishment or the American economy that we haven't talked about?

So I'm really excited about synthetic biology and largely because I think so few people in our national security structure understand biology. I think a lot of the people that run these organizations, they're electrical engineers. Right. They're chemists, they're not biologists. And molecular biology has changed and exploded so much in the last 20 years that very few people who are in leadership levels were learning these things in school. Right. They weren't getting their PhDs in these areas. And I think it has such an interesting opportunity for supply chain resilience. I think it has such an interesting opportunity for fundamentally new types of living materials or sort of hybrid materials. I think it has interesting opportunities for optics and other things like that. But I think that the reason it's so exciting to me is that it's also the largest opportunity for surprise because our enterprise is the least well equipped to deal with it. We have people who love space and they love hypersonics and they love these big hard mechanical things that you can hit with a hammer. Biology, they think about it through the lens of threat, but they don't really think about it through the lens of capability. And that. That excites me.

Yeah, that's cool. I'm glad you mentioned that. Now last year we published a report on our OODA loop site, the Executive's Guide to the Biotech Revolution, where we went through just everything we could about what does a business leader need to know about it? With the thought that no matter what business you're in, the biotech revolution is going to impact you in some way or another. You know, it's going to change your employees lives and it may very well change your markets and how you do work.

Absolutely. There are a couple of people. There's a Dr. Melissa Rhodes is at Lockheed Martin. I love talking to her because she really thinks about it through this lens of she's an electrical engineer and a bioengineer, so she thinks about this intersection in super creative ways. And then Michelle Rosso, who's the technical director or I don't know what their titles are anymore, the Pentagon changed everybody's titles. But at the Pentagon in R and E for biology, she's also really thought about this around supply chain resiliency and other things. I think we have some really creative people in this space, but we don't really have a critical mass yet. So I'm with you. I think it's really exciting. I think it could change everything. I also think it could really surprise us.

Awesome. Melissa. I'd like to transition now, if it's okay with you, and ask ways that you keep yourself current. Now I know you stay current because you're engaged with a lot of other people in discussions like this, but what books are you reading?

So I am a little bit of a non traditional reader. I've been known to offend some people. I try not to read things that are just going to give me back more of what I've already learned from my sort of networks and mentors and education. I try to read books that give me just 10% of the book maybe that says, ah, you hadn't thought about the problem that way. And so I've been reading some books like Peter Zehan's Disunited States, that for me gave me this context of geography as an unchanging dynamic in power. And I've really been thinking about that through all of the changes in the global order during COVID Geography is this very unchanging thing and it dictates a lot of the choices that we are allowed to make as nations. And I think you really see the US having different strategies and different opportunities. And you see Australia and New Zealand similarly having these because of their geography. Whereas Europe is sort of required to take different approaches. So that has really. I don't agree with all of the book and you know, a lot of the writing is a little. It's polemical and whatever, but I love the idea that it gave me a new frame that I now bring to things. I'm reading Tyranny of the Experts right now. It's a real challenge to the sort of nation level approach to international development. And it thinks about how would you think about development if you were thinking about individuals as the ones that needed to benefit rather than states? And how would you think about development if those same actions were being applied to the United States, not just to a third world country. And again, it's not something where I would say, oh, I agree with the whole thing. You should read it. No, I think you should read it to say, hey, maybe you weren't taught this in school. How does that make you feel? How do you think about it? How is that new information challenge the hypothesis and the experiment you would run? So those are just a couple of books that I think I've been reading recently to try to shake up my frame a little.

Well, I'm going to put links to both of those in our show notes for anyone who's interested. I'm going to check them out myself.

Well, I apologize if they offend anyone. Although I'm sort of sorry. Not sorry. Right. Because everybody needs to be challenged a little bit in their life.

That's right. Melissa, thanks for all this. I really appreciate your time. It's very gracious of you to give us the this all your thoughts on these issues.

It's really exciting to engage with you and your listeners. It's important to me to be available as well. So please, if people want to talk to me, all they need to do is find me on LinkedIn, shoot me a note. I really feel that we have a responsibility to help everybody be in service in their own way. And that doesn't always mean working for the government. So however, I can help people do that. That's my goal.

Awesome. Thanks, Melissa.

Thank you. Thanks for listening to this. OODA Loop production For the latest analysis on cybersecurity technology and global risks, please visit www.ooda loop.com.