Courtney Bowman: Privacy and Civil Liberties Engineer, Palantir Technologies

Informing your decisions with intelligence analysis and insight brought to you by the team@ooda loop.com hello.

I'm Bob Gourley, the Chief Technology Officer of OODA, llc, and you're watching the oodacast today with Courtney Bowman, the Director of Privacy and Civil Liberties Engineering for Palantir. Hello, Courtney.

Hi, Bob. It's a pleasure to join you.

Yeah, it's good talking with you again. Courtney, a little over a year ago in March 2019, you allowed me to interview you for my CTO Vision publication. You did a nice piece on artificial intelligence, including ethics, security and compliance, and your views on that and approaches that we could all put to use. And I would love to continue the discussion from that. But, you know, over the last year, the world has changed with this pandemic and even more need for an understanding of ethics and compliance and data. And I noticed you've been writing about that. I'd like to talk about some of your recent writing, and I also thought our community would really find your background interesting. You're a very unique individual, and I could talk a little bit about that.

Great. That sounds exciting. I'm happy to delve into whatever details, biographical or my specific work that might be of interest to you.

Good. Let's start with the beginning. You pursued a degree in physics and philosophy at Stanford, is that correct?

Actually, dual degrees in physics and philosophy as an undergraduate at Stanford, yeah.

That's fascinating. Now, what made you want to pursue that?

Really what it came down to. It's a longer story, but what it came down to is I couldn't choose between the two, and I decided to do both. One was sort of. Both were passions. One had more practical ramifications, namely physics. Clear to me and probably to my parents that there was probably a lot more utility that I would get out of a physics degree and less utility out of a philosophy degree. But I was pretty adamant that I wanted to do both. And in some sense, that maybe wasn't clear to me then, or it wasn't entirely clear to me then that those two pieces of my head would come together later in my life into some meaningful ways of comporting myself to the words, towards the world.

Yeah, it's really fascinating. In physics, I think we all know is, you know, the study of reality, philosophy, just as a layman, the study of how we think and why we're here and what is life. But I wanted to ask you, what is philosophy? Is that how you define it?

On some level, I think I reduce my understanding of philosophy to its literal meaning. In Greek, philosophy means love of wisdom. And that's really what philosophy means to me at a most fundamental level. And the thing that I've taken from studying philosophy over many years and being very passionate about the subject is that this acknowledgment that love of wisdom is really a recognition of humility. It's as much understanding what we know as understanding what we don't know. And constantly that point of humility and calling attention to the limits of understanding the limits of what can be known in the world has helped situate my thinking on a lot of topics in my personal life, but also in my professional life.

Great. And speaking of professional life, so now you're at Palantir, where, as I mentioned, you're the director of Privacy and civil Liberties Engineering. And I thought that title was interesting because that says a lot. Number one, it says you guys take privacy and civil liberties very seriously. And I've known that. But you also put the title engineering in there. And I know you have a big impact on policy and process and other leadership. But you said engineering, which to me implies getting things done and building real solutions. Is that the right implication there?

Yeah, I think that's quite right. So when I joined Palantir Close to 10 years ago, the colleagues that I was working with and the directors and the executive team at Palantir really had a vision for integrating privacy as a core function, as a reflection of commitments that the company had made in the space of work that it contributes to, which is sensitive data, powerful technology for powerful institutions. And the thing that we were trying to build upon was this thesis that in order to create technology platforms that enable institutions to do powerful tasks and to enable their use of new data sources, there was a recognized risk that attaches to that work. And we didn't want the outcomes that were enabling to come at the expense of values that we think are central to societies that we live in and personally value. So, namely, things like privacy and civil liberties were challenges that were identified within the information sciences faces. But we were looking at ways of delivering on that commitment. And we did a market assessment to understand the existing landscape of how other institutions deal with privacy. And largely what we found was that most companies at the time really thought about privacy as a sort of compliance oriented, almost a legal formalistic concept where you have a set rubric of requirements that you have to meet and you go through and check the boxes and then you consider your privacy obligations to be fully exhausted. And the insight that we have was that that's really an inadequate approach to thinking about privacy. To really get at the crux of privacy, you have to acknowledge that privacy is much more than legal formalistic qualities. It's a confluence of social sector considerations. It brings in aspects of philosophy of social sciences, but also core engineering practices. And so we really built up this notion of privacy as an engineering concept that melds together these different concepts and focuses on engineering solutions that anticipate changing environments. And that anticipation component is particularly critical when you think about the fact that the current status of privacy law, privacy regulation, policies and procedures really only reflect the technology of today and more realistically of 10 years ago. If you want to get ahead of the issues that are going to be on the horizon today and tomorrow and five years from now, you have to use a bit of an imaginative exercise to anticipate what that landscape is going to look like, to see around corners and then build against that future vision. So a lot of what we focus on intentionally was to create this engineering practice and exercise of thinking ahead and designing solutions respective of this broad set of considerations and built into a comprehensive.

Practice that makes a lot of sense. And everything you're saying resonates with me as a security practitioner. Very similar ideas where a lot of folks will say I'm compliant and therefore I'm secure, I'm following the rules and therefore I'm secure when you're not. And the same kind of thing with privacy. I know the way you explained it just makes it very clear and there's just no way that any compliance regime can keep up with the continuous innovation underway in startups around the world or large companies around the world are innovating so fast. So I like the way that you're being proactive and taking a longer term view. Another way to underscore it is the fact that decisions you make today could end up 10, 15, 20 years from now violating somebody's privacy. Maybe the easiest example might be if someone gets access to my DNA and my genome, they're violating the privacy of all of my descendants now and well into the future. So these are complex things that I'm glad you guys are thinking through.

Yeah, I think that's, that's very much right. They're complex subjects and as you, as you started to allude to, they're things that go well beyond the scope of immediate public policy. They kind of implicate a future vision of the world. But there's also, if you scratch the surface, you start to realize there's some, some underlying social theory and philosophical concepts that need to be reflected in how you approach those questions. So that's, that's really getting at the intent behind this, this concept. And over the years what we did is build this out as a, as a comprehensive practice within Palantir, where today we have about 20 privacy and civil liberties engineers focus on these issues, running the gamut of the policy considerations in the spaces of work that we operate in, but also building privacy enhancing technologies with a view towards those policy and non technical components. So we're both examining this landscape of risk and opportunity and then building solutions to address that within the product landscape that we as a company focus on.

Great. So let's talk a little bit about the crisis. And you know, in any crisis there's a tendency for good people to want to do good things and let's just do the right thing and solve the problem. But too frequently people will do that in a way that might trample on privacy issues and never good to do that. Whatever the crisis is, whether it's a criminal event or a natural disaster or now we're all finding out this big pandemic. There was an early rush, people wanted solutions and people like me would think, whatever it takes, let's just beat this, let's get all the data together. When really you guys are counseling that you do that in a way that does not violate privacy. There's no reason to violate our ethics or privacy or data security rules when we analyze the data to solve these kind of problems. And, and that's one thing I wanted to underscore, that it's clear to me that we do not have to violate our privacy or push our ethical boundaries as we analyze data.

Would you agree? I think that's right. And you alluded to some writing that I've done in my role at Palantir is something that we focused on in the early days of the pandemic in mid March. This was before we started to work with any public health agencies or any public or private sector organizations on pandemic response efforts. We saw kind of a rush towards technology to help address the challenges of the current crisis. And we know from experience, having built the business in the wake of 9, 11, that in the wake of a crisis, the tendency is to throw all available resources at a challenge and oftentimes kind of give short shrift to the normative considerations that usually define the landscape of opportunity and types of solutions that we throw at those problems. We view that as a risky approach. And we set out a set of guiding principles and thoughts on how institutions, not just Palantir, but for our customers. But any institutions that are looking to use information and information science and information technology to address the COVID crisis, how they should be thinking about the pandemic and the best use of information to solve the challenges at hand. So we set out a set of ideas and they turned out to be guiding principles that directed our work in the space. There are things like thinking about the end life cycle of the technology. So the extent to which you may need to collect new information sources and apply those to analysis to drive supply chain understanding or to look at the distribution of health resources across different health service providers. Those information data sources need to be time bound and focused on a limited scope of applications so that you're collecting sensitive information at the end of the crisis, you have a mechanism to delete it, ensure that it's not retained and used for purposes beyond the immediate crisis. We also called attention to the fact that oftentimes in a moment of crisis or exigent need, there's a tendency to go towards shiny objects. The belief that things like, and this is something that we talked about the last time we we you interviewed me, this sort of tendency to look at novel forms of technology like artificial intelligence and machine learning as a, as a panacea to the world's problems, that may not be the case. It may be the case that there are techniques that can be applied in the machine learning landscape that are effective for specific and discrete applications. But the belief that you can just throw data into a black box and have it tell you how to resolve this inordinately complex challenge of solving a pandemic, we view that as folly and we thought it was more important to start with the immediate challenges. Use those challenges to define the landscape of information that's available, that needs to be captured and also the technological components that need to be brought to bear in the analysis of that, of that data and then use that to gradually guide your approach to new technology or new data data source acquisition. So really what that gets to is a reinforcement of long standing practices in the privacy landscape. There's a concept of fair information practice principles that are well enshrined within existing privacy regimes and policy and regulations. And in particular a couple of concepts or a few concepts stand out to us and really help direct my thinking in a lot of the work that Palantir has done, which are the concepts of necessity defining what's necessary in a minimum capacity to perform a given function. The further concept of proportionality, what's proportionate to the need at hand and what's legitimately justified for a specific outcome and limiting yourself to only that information for the purposes that are defined at hand. And then as you're building out these processes and using the data in a necessary and proportionate way, ensuring a third concept of transparency to the stakeholders that are involved, making sure that the public is brought along in the journey and understands the application of data science and why it's intended to perform certain outcomes that have been mandated by the public role of institutions that serve us. So it's these sort of concepts that we tried to bring to bear and to paint a picture of how institutions in the public and private sector should be using technology to address the current crisis.

All of that, Courtney, underscores to me the fact that you need to have a vision for what you're going to do, especially in a crisis. And to make that vision real, you need a data strategy. And that to me, everything you just said underscores those points. So would you agree you need a good vision and data strategy for what you're going to do?

Yeah, I think if you start off with the idea that you just take the data and the data will provide a pathway, that's another vision of folly. It's a way to end up with a tangle of raw data that doesn't really tell you what to do, doesn't really give you insight into how to solve the challenge.

It may come with large risk.

It will certainly create risk that you maybe didn't anticipate and will have to deal with later on, because there's a tendency to rightful tendency to be concerned about the over collection of data. Particularly in a society in which we have so many gadgets that are creating data exhaust and trails of information that may not be necessary, but starting with a strategic approach, thinking through what it is that you're trying to do, what's necessary to direct yourself towards those aims, and then gradually building out towards proximate outcomes where each step justifies the next step in a way that you can demonstrably point to that chain and say, well, I've achieved this objective. The objective comports with the principles that I laid out initially. Here's why. For the furtherance of public health needs, it's necessary to go an additional step, all of that trajectory. And laying out and curating those details gets you to a point where you can demonstrably and faithfully represent the value of applying information science and information technology to these types of crises situations.

Fascinating. And after laying out this, then, I mean, there's also, I guess, the way I want to Ask it. There's foundational technological things that can be provided that make it easy to do a lot of these things. And maybe the simplest example I can think of is just always having your data encrypted and managing the keys smartly and having a good data fabric like that that has built in these security foundations. Are there other engineering things like that that you just deliver as a foundational privacy capability?

Yeah, you'll have to forgive me for the shameless plug, but I and a few colleagues a number of years ago authored a book called the Architecture of Privacy, which was really meant to serve as a manual for technologists and entrepreneurs who are building information systems to think through the fundamental components of a privacy protective system. And our argument in that text was that there are a few, a handful of critical considerations that go into responsible data architectures. One of those things is an access control regime that allows you to selectively reveal and expose information to. So consistent with this principle of proportionality and use limitation that I talked about a few moments ago, that's kind of a fundamental component. But even before that, there's a sort of threshold question of what are the problems that you're trying to solve and are those problems well suited to information technology? There's certain classes of problems that may not actually lend themselves to technological interventions. They may be so deeply socially grounded that the real solution comes in the form of a community engagement. But for the class of problems where it does make sense to apply information science and information technology, you want to start with a solid basis for security. And the principles that you talked about, ensuring encryption in rest and in transit, but operational controls over how information is used through access limitations. And then building on that, thinking about justifications for further use of the data and other principles of ensuring accountability and oversight. So ensuring that you have a good audit regime in place to make sure that the user interactions with the system and the sensitive data are faithfully represented in logs that can then be interrogated by oversight authorities for forensic purposes or for immediate identification of malfeasance or misuse of the data. Those are critical principles. And then the kind of final tranche of considerations are thinking about the full lifecycle of data management. So it's not just one thing to collect data responsibly, but and to use the data responsibly. But you have to think about the end of the life cycle, what happens when the data no longer serves the purposes for which it was generated. You need to think about how you sunset that data, either archive it or delete it in some strong sense so that the data isn't sitting out there in some persistent form that then can be exploited for ill gain. So all those principles I view as being kind of central ideas to the appropriate and responsible use of data systems, and those are just sort of scratching the surface of things that need to be in place. You get into more complex considerations when you start to really refine the ideas of what you're doing with the data. Then you can get into more involved questions about different modes of anonymization practices like differential privacy, which provide you with ways of partially obscuring and providing measurable guarantees of whether you sufficiently de identify data. All those things are additional techniques. But the key point is that there's some fundamental things that you can use to secure the use of information and systems and set your enterprise up for a solid foundation to move forward from.

Fascinating. And again, you make me think through so many other follow on questions I want to ask you. But first let me bring it back to this pandemic. Because just two days ago I was listening in the car to a talk show commentator who was talking about contact tracing and he was railing on about how this is a violation of our civil liberties and he does not want contact tracing. He was afraid that that database is going to be put right next to the FISA database and connected to NSA and it's just going to pull all this data together for big government to spy on us. Now that made me mad that someone so smart as that guy was making these fallacious arguments. But I know some people believe that and contact tracing could be done in a bad way that reveals too much information about people, or it could be done in a smart way where information is protected. And you mentioned differential privacy. There could be ways where names are never used, but there's hashes that can get you back to the right data or other smart data strategies. Because I really believe to get through this pandemic we need good contact tracing, but we also need to protect privacy and people's rights to, you know, keeping their, their own life private to the greatest extent possible. So I mentioned that because everything you're talking about is so on point and topical. And I wanted to say one other thing too. As you're talking and I'm thinking it's just perfect that you have this philosophical background to deal with these kind of issues. And it reminded me of the old question we used to ask ourselves in the national security domain. Do we want to be like Athens or Sparta? Do we want the freedom and liberties and free thought of Athens, where there's so much creativity and storytelling and peace and harmony and love? Or do you want the strict regime of Sparta where there is zero freedom but you are solidly defended? And in a national security context, the right answer is always, you don't want to be Athens or Sparta. You want to have freedom and security. And now I hear you talk about data and I think it's a similar thing. We want data that can be used and is really functional. It's got to get the job done. But you don't violate security or privacy or civil liberties in order to do that.

Yeah, I mean, this is, I think you touched Bob on one of the foundational premises of the company. When we were founding the company, the idea was that there's sort of this false dichotomy that's, that's been established since time immemorial. And you use the example of Sparta and Athens. It really is this false choice between security on the one hand or privacy on the other. And granted, there are instances where you do need to make explicit trade offs and there the game is really methodically explaining why you've made specific trade offs for specific purposes and as much as possible time bounding the trade offs that you've made. But by and large, what we've discovered as a company and built the enterprise around is this notion that oftentimes you don't have to trade security for privacy or data utility for privacy in the commercial context. Oftentimes there's ways of sensibly arriving at or optimizing against both outcomes. And the way that you do that is by smartly engineering systems that enable mission objectives, but do so in a way that's responsible and methodical. So when you talk about things like contact tracing, contact tracing, I should say Palantir. As a company, we're not building contact tracing technologies, but there's a prospect that our information systems is platforms that are designed to deal with comprehensive sets of data for our customers could be used for some version of contact tracing application. Then that raises the question of, well, how do you do this responsibly? And the starting point there is, well, what's the reasonable use case of contact tracing? Where does it make sense? Those threshold questions raise further questions about whether you have a firm testing regime in place and whether you've significantly reduced the volume of cases so that you, you have a sensible load of cases that you can actually contact trace versus dealing with full community spread where contact tracing doesn't make as much sense as just broad public health messaging. But Assuming contact tracing makes sense in a given environment, I think there's a lot of sensible proposals that have been put out there for digital versions of contact tracing. It may very well be the case that in certain jurisdictions, the digital exposure notification systems, so the example is the Apple Google protocol we've seen a lot of writing about, may not make sense, and it may make more sense to focus on a manual version of contact tracing, which is the traditional version of contact tracing. In either case, there are some real concerns because contact tracing involves some very sensitive personal information and, and saying you want to make sure that the extent to which that data is being collected by digital devices or by manual auditors, that information is being very selectively collected, controlled, and used for specific purposes so that you don't run the risk of the data being repurposed or being inadvertently or advertently shared across institutional boundaries. And all of those things are standard practices for public health agencies that are focused on these challenges. And I take it as a matter of good faith that most of these institutions really are focused on achieving the right outcomes for public health initiatives. But the way that they do that and the way that they earn the public trust is by explaining how the systems that they use really are intended. The systems and the programs that they're building around these types of regimes are put in place to protect the interests of the constituents, to deliver the public health outcomes, and, and also to ensure that the values and rights and liberties of individuals affected by these systems are being respected at every step along the way.

Right. And Courtney, that's also very consistent with the way you wrapped up your Medium article. You talked about the fact that history will have its eyes upon us. Us being all of humanity, of course, but also our large corporations, our governments from state, local, federal, other international governments. History will have its eye on us. And we have to respond to this pandemic. We have to crush it. But we also have to protect privacy and civil liberties. And I think by saying it the way you did, that history will have its eyes on us. It's a good reminder that we got to do the right thing and return to normalcy as soon as we can. And when we do, make sure we have not violated privacy or our ethics.

Yeah, I think that the other way of framing that is, will we recognize ourselves when we exit the pandemic, or will we be in a world in which the landscape has so dramatically shifted and the normative expectations around the use of information and information technology has so dramatically shifted towards the. The concerns of a surveillance model or Panopticon such that it doesn't feel like the society that we were in when we entered the crisis. That's the thing that we constantly have to be vigilant of and guard against. And that was really one of the motivations for writing that piece and disseminating it, that I wanted to share that as a set of guiding thoughts for institutions focus on these challenges and not knowing at the time that Palantir was going to become heavily involved in public health response efforts. But when we did get involved, it really was a guiding precept for us as a company. We continually ask ourselves the question of is the work that we're doing really lending itself to outcomes that we're going to feel comfortable with at the end of the pandemic? But five years, 10 years out, when we write the story of Palantir in 2019, 2020, 2021, will it be a story that we, as engineers and practitioners in this space, feel comfortable having played a small part in contributing to?

Great. Thanks, Courtney. And Courtney, I also want to ask another question about how you keep current. Now, I know you stay current because you're directly engaged, not just with your team there at Palantir, but your customers. And so you know the issues. But what kind of reading do you do?

Yeah, I so I A lot of my reading is kind of runs the gamut because of the work that I'm focused on, is fairly heavily involved in regulatory circles. I have to keep up on developments and data protection landscapes, data protection legislation that's being proposed, that's near term going to be implemented, that constitutes a big share of my reading. But I also have a few kind of passion projects that I think fit squarely within the space of things that Palantir thinks about as a company and is concerned about as a company. So I do a lot of reading on the prospects of artificial intelligence, but more critically with an eye towards underlying philosophical concepts. And I'm really focused on this idea of understanding the limits of technology. And I have some specific thoughts on this that I continue to explore, and a thread of philosophy that I study very closely as a student, but it continues to drive a lot of my interest. So a lot of the time that I'm doing leisurely reading, it's reading about philosophy of artificial intelligence, the ethics of applied data science, and then kind of channeling that into current themes and trends in the legislative and policy landscape with an eye towards taking these kind of more abstract philosophical concepts and translating them into practical insights that can be reflected in my day to day job.

Wonderful.

Great.

Courtney, thanks for this. I really appreciate the time.

My pleasure, Bob. It's always fun chatting with you. Thanks so much for the interview.

Yeah, let's do this again sometime.

Great.

Thanks, Courtney.

Thanks for listening to this. OODA Loop Production. For the latest analysis on cybersecurity technology and global risks, please visit www.oodaloop.com.