Privacy Preserving AI (Andrew Trask) | MIT Deep Learning Series

Today we're happy, very happy to have Andrew Trask. He's a brilliant writer, researcher, tweeter, that's a word in the world of machine learning and artificial intelligence. He is the author of Grokking Deep Learning, the book that I highly recommended in the lecture on Monday. He's the leader and creator of Open Mind, which is an open source community that strives to make our algorithms, our data and our world in general more privacy preserving. He is coming to us by way of Oxford, but without that rich, complex, beautiful, sophisticated British accent. Unfortunately, he is one of the best educators and truly one of the nicest people I know. So please give him a warm welcome. Thank you. Thanks. That was a very generous introduction. So, yeah, today we're going to be talking about privacy preserving AI. This talk's going to kind of come in two parts. So the first is going to be looking at sort of privacy tools from the context of a data scientist or a researcher, like how their actual UX might change, because I think that's sort of the best way to communicate some of the new technologies that are coming about in that context. And then we're going to zoom out and look at under the assumption that these kinds of technologies become mature. What is that going to do to kind of society? What sort of consequences or side effects could these kinds of tools have, both positive and negative? So first let's ask the question, is it possible to answer questions using data that we cannot see? This is going to be the key question that we look at today. And let's start with an example. So first, if we wanted to answer the question, what do tumors look like in humans? Well, this is a pretty complex question. Tumors are pretty complicated things. So we might train an AI classifier. If we wanted to do that, we would first need to download a data set of tumor related images. So we'd be able to statistically study these and be able to recognize what tumors look like in humans. But this data is not very easy to come by. So it's very rarely that it's collected, it's difficult to move around, it's highly regulated. So we're probably going to have to buy it from a relatively small number of sources that are able to actually collect and manage this kind of information. The scarcity and sort of constraints around this likely to make this a relatively expensive purchase. And if it's going to be an expensive purchase for us to answer this question, well, then we're going to have to find someone to sort of finance our project. And if we need someone to finance our project, we have to come up with a way of how we're going to pay them back. If we're going to create a business plan, then we have to find a business partner. We're going to find a business partner. We have to span all our classmates in LinkedIn looking for someone to start a business with us, right? No, it's because we wanted to answer the question, what do tumors look like in humans? What if we wanted to answer a different question? What if we wanted to answer the question what do handwritten digits look like? Well, this would be a totally different story, right? We download a data set, we download a state of the art training script from GitHub, we'd run it and a few minutes later we'd have a ability to classify handwritten digits with potentially superhuman ability, if such a thing exists. Why is this so different between these two questions? The reason is that getting access to private data, data about people is really, really hard. And as a result we spend most of our time working on problems and tasks like this. So Imagenet, mnist. If R10 anybody who's trained a classifier on MNIST before, raise your hand. I expect pretty much everybody. Instead of working on problems like this, does anyone train a classifier to predict dementia, diabetes, Alzheimer's? Looks like we should go in depression, anxiety? No one? So why is it that we spend all our time on tests like this when these tasks, these represent our, our friends and loved ones and problems in society that really, really matter. Not to say that there aren't people working on this, there are whole fields dedicated to it. But the machine learning community at large, these tasks are pretty inaccessible. In fact, in order to work on one of these, just getting access to the data, you'd have to dedicate a portion of your life just to getting access to it, whether it's doing a startup or joining a hospital or, or what have you. Whereas for other kinds of data sets, they're just simply readily accessible. This brings us back to our question, is it possible to answer questions using data that we cannot see? So in this talk we're going to walk through a few different techniques and if the answer to this question is yes, the combination of these techniques is going to try to make it so that we can actually PIP install access to data sets like these in the same way that we PIP install access to other deep learning tools. And the idea here is to lower the barrier to entry, to increase the accessibility to some of the most important problems that we would like to address. So as Lex mentioned, I lead a community called OpenMind, which is an open source community of a little over 6,000 people who are focused on sort of lowering the barrier to entry to privacy, observing, AI and machine learning. Specifically, one of the tools they're working on we're talking about today is called PySift. PySift extends the major deep learning frameworks with the ability to do privacy preserving machine learning. So specifically today we're going to be looking at the extensions into Pytorch. So Pytorch people generally familiar with Pytorch, quite a few users, and it's my hope that by walking through a few of these tools it'll become sort of clear how we can start to be able to do sort of data science, the act of answering questions using data, using data that we don't actually have direct access to. And then in the second half of the talk, we're going to generalize this to answering questions, even if you're not necessarily a data scientist. So first, first tool is remote execution. Okay, so let's just walk me through this. So we're going to jump into code for a minute, but hopefully this is sort of line by line and relatively simple. And even if you aren't familiar with Pytorch, I think it's relatively intuitive looking at like lists of numbers and these kinds of things. So up at the top we import TORCH as a deep learning framework. Sift extends Torch with this thing called Torch Hook. All it's doing is just iterating through the library and basically monkey patching in lots of new functionality. And most deep learning frameworks are built around one core primitive, and that core primitive is the tensor. Right. So for those of you who don't know what tensors are, just think of them as nested list of numbers for now and that'll be good enough for this talk. But for us, we introduce a second core primitive, which is the worker. And a worker is a location within which computation is going to be occurring. So in this case we have a virtualized worker that is pointing to say a hospital data center. Right. And the assumption that we have is that this worker will allow us to run computation inside of the data center without us actually having direct access to that worker itself. It gives us a limited sort of whitelisted set of methods that we can use on this remote machine. So just to give you an example, so there's that core payment we talked about a minute ago, we have the torch tensor. So 1, 3, 4, 5. And the first method that we added is called just send, right? And this does exactly what you might expect. Takes the tensor, serializes it, sends it into the hospital data center and returns back to me a pointer. Now, this pointer is really, really special. And for those of you who are actually familiar with deep learning frameworks, I hope that this will really resonate with you because it has the full Pytorch API as a part of it. But whenever you execute something using this pointer instead of it running locally, even though it looks like and feels like it's running locally, it actually executes on the remote machine and returns back to you another pointer to the result. The idea here being that I can now coordinate remote executions, remote computations, without necessarily having to have direct access to the machine. And of course I can get a get request. And we'll see that this is actually really, really important, getting permissions around when you can dot get requests and actually ask for data from a remote machine back to you. So just remember that. Cool. So this is where we start. So in the kind of like the Pareto principle, you know, 80% for 20%, this is like the first big cut, right? So pros data remains on a remote machine. We can now in theory do data science on a machine that we don't have access to, that we don't own. Right. But the problem is, the first column we want to address is how can we actually do good data science without physically seeing the data? So it's all well and good to say, oh, I'm going to train a deep learning classifier, but the process of answering questions is inherently iterative. It's inherently sort of, sort of give and take. And I learn a little bit and I ask a little bit, I learn a little bit and I ask a little bit. This brings me to the second tool, so search and example data. Again, we're starting really simple. It will get more complex here in a minute. So in this case, let's say we have what's called a grid. So PY grid, if PYSIFT is a library, PY grid is sort of the platform version. So it's sort of, again, this is all open source Apache 2 stuff. This is that we have what's called a grid client. So this could be a interface to a large number of data sets inside of a big hospital. Right. And so let's say I wanted to train a classifier to do something with diabetes, right. So it would mean to predict diabetes or predict certain kind of diabetes or a certain attribute of diabetes. Right. I should be able to perform remote search. I Get back pointers to throw the remote information. I can get back sort of detailed descriptions of what the information is without me actually looking at it, right? So how it was collected, what the rows and columns are, what the types of different information is, what the various ranges of the values can take on. Things that allow me to do sort of remote normalization, these kinds of things, and then in some cases even look at samples of this data. So these samples could be sort of human curated, they could be generated from a gan, they could be, they could be actually short snippets from the actual data set. Maybe it's okay to release small amounts, but not large amounts. And the reason to highlight this, this isn't like crazy complex stuff. So prior to going back to school, I used to work for a company called Digital Reasoning. We did sort of on prem data science, right? So we did delivered sort of AI services to corporations behind the firewall. So we did classified information. We worked with investment banks, helping prevent insider trading. And doing data science on data that your home team back in Nashville, in our case is not able to see is really, really challenging. But there are some things that can give you the first big jump before you jump into the more complex tools to handle some of the more challenging use cases. Cool. So basic remote execution, so remote procedure calls, basic sort of private search and the ability to kind of look at sample data gives us enough sort of general context to be able to start doing things like feature engineering and evaluating quality. Okay. So now the data remains in the remote machine. We can do some basic feature engineering. And here's where things get a little more complicated. Okay, so if you remember in the very first slide where I show you some code at the bottom, I called get on the Tensor, right? And what that did was it took the pointer to some road information and said, hey, send that information to me. That is an incredibly important bottleneck. Right. And unfortunately, despite the fact that I'm doing all my remote execution, if that's just naively implemented, well, I can just steal all the data that I want to, right? I just call get and whatever pointers I want and there's sort of no additional added real security. So what are we going to do about this? This brings us to tool number three called differential privacy. Differential privacy. Anyone come across a little higher? Okay, cool. Awesome. Good. So I'm going to do a quick high level overview of the intuition of differential privacy. And then we're going to jump into how it could, can and is being looking sort of in the code. And I'll give you resources for kind of a deeper dive into differential priv. See at the end of the talk, should you be interested. So differential privacy, loosely stated, is a field that allows you to do statistical analysis without compromising the privacy of the data set. So more specifically it allows you to query a database while making certain guarantees about the privacy of the records contained within the database. So let me show you what I mean. Let's say we have an example database. And so this is kind of the canonical db. If you look in the literature for differential privacy, it'll have sort of one row for person, one row per person, and one column of zeros and ones which corresponds to true and false. We don't actually really care what those zeros and ones are indicating. You know, it could be presence of a disease, could be male, female, could be. It's just some sensitive attribute, something that's worth protecting, right? What we're going to do is we're going to, our goal is to ensure statistical analysis doesn't compromise privacy. What we're going to do is query this database, right? So we're going to run some function over the entire database and we're going to look at the result and then we're going to ask a very important question. We're going to ask, if I were to remove someone from this database, say John, would the output of my function change? Okay. And if the answer to that is no, then intuitively we can say that, well, this output is not conditioned on John's private information. Now if we could say that about everyone in the database, right? Well then, okay, it would be a perfectly privacy preserving query. Right? But it might not be that useful. But this intuitive definition I think is quite powerful, right? The notion of how can we construct queries that are invariant to removing someone or replacing them with someone else? Okay. And the notion of the maximum amount that the output of a function can change as a result of removing or replacing one of the individuals is known as the sensitivity. So important. So if you're reading literature, you find come across sensitivity. That's what we're talking about. So what do we do when we have a really sensitive function? We're going to take a bit of a sidestep for a minute. I have a twin sister who's finishing a PhD in political science and political science often they need to answer questions about very taboo behavior, something that people are likely to lie about. So let's say I wanted to survey everyone in this room and I wanted to answer the question, what percentage of you are secretly Serial killers, right? And not because, like, yeah, not because I think any one of you are, but because I genuinely want to understand this trend, right? I'm not trying to arrest people. I'm not trying to sort of be an instrument of the criminal justice system. I'm trying to be a sociologist or political scientist and understand this actual trend. The problem is, if I sit down with each one of you in a private room and I say, I promise, I promise, I promise I won't tell anybody, I'm still going to get a skewed distribution. There are going to be some people who are just going to be like, why would I risk telling you this private information? And so what sociologists can do is this technique called randomized response, where I should have brought a coin. You take a coin and you give it to each person before you survey them, right? And you ask them to flip it twice somewhere that you cannot see. So I would ask each one of you to flip a coin twice somewhere that I cannot see. Then I would instruct you to, if the first coin flip is a heads answer honestly, but if the first coin flip is a tails answer, yes or no based on the second coin flip. So roughly half the time you'll be honest, and the other half the time you'll be giving me a perfect 50, 50 coin flip. And the cool thing is that what this is actually doing is taking whatever the true mean of the distribution is and averaging it with a 5050 coin flip, right? So if, say, 55% of you answered yes that you are a serial killer, then I know that the true center of the distribution is actually 60% because it was 60% averaged with a 50, 50 coin flip. Does that make sense? However, despite the fact that I can recover the center of the distribution, given enough samples, each individual person has plausible deniability. If you said yes, it could have been because you actually are, or it could have been because you just happened to flip a certain sequence of coin flips, okay? Now, this concept of adding noise to data to give plausible deniability is sort of the secret weapon of differential privacy, right? And the field itself is a set of mathematical proofs for trying to do this as efficiently as possible, to give sort of the smallest amount of noise to get the most accurate results, right? With the best possible privacy protections, right? There is a meaningful sort of base trade off that, you know, there's kind of a Pareto trade off, right? And we're trying to push that, push that trade off down. But so the field of research that is differential privacy is looking at how to add noise to data and resulting queries to give plausal deniability to the members of a database or a training data set. Does that make sense? Now, a few terms that you should be familiar with. So there's local and there's global differential privacy. So local differential privacy adds noise to data before it's sent to the statistician. So in this case, the one with the coin flip, this was local differential privacy. It affords you the best amount of protection because you never actually reveal sort of in the clear, your information to someone. And then there's global differential privacy, which says, okay, we're going to put everything in the database, perform a query, and then before the output of the query gets published, we're going to add a little bit of noise to the output of the query. This tends to have a much better privacy trade off, but you have to trust the database owner to not compromise the results. And we'll see. There's some other things we can do there. But with me so far, this is a good point for questions, if you had any questions. Got it. So the question is, is this verifiable, Any of this process of differential privacy verifiable? So that is a fantastic question and one that actually absolutely comes up in practice. So first, local differential privacy. The nice thing is everyone's doing it for themselves, right? So in that sense, if you're flipping your own coins and answering your own questions, that's your verification, right? You're kind of trusting yourself for global differential privacy. Stay tuned for the next tool and we'll come back to that. All right, so what does this look like in code? So first we have a pointer to a remote private data set we call git. Whoa. We get big fat error, right? You just asked to sort of see the raw value of some private data point, which you cannot do, right? Instead pass in, get epsilon to add the appropriate amount of noise. So one thing I haven't mentioned yet, differential privacy. So I mentioned sensitivity, right? So sensitivity was related to the type of query, the type of function that we wanted to do, and it's in variance to removing or replacing individual entries in the database. So epsilon is a measure of what we call our privacy budget. Right. And what our privacy budget is is saying, okay, what's the amount of statistical uniqueness that I'm going to sort of limit? What's the upper bound for the amount of statistical uniqueness that I'm going to allow to come out of this, out of this database? And actually I'm going to take one more sidetrack here because I think it's really worth mentioning data anonymization. Anyone familiar with data anonymization come across this term before taking a document like redacting the Social Security numbers and all this kind of stuff? By and large it does not work. If you don't remember anything else from this talk, it is very dangerous to do just data set anonymization. And differential privacy in some respects is the formal version of data anonymization where instead of just saying, okay, I'm just going to redact out these pieces and I'll be fine, this is saying, okay, we can do a lot better. So for example, Netflix prize, Netflix machine learning prize. Have you remember this big million dollar prize? Maybe some people in here competed in it. So in this prize, Netflix published an anonymized data set. And that was movies and users. And they took all the movies and replaced them with numbers and they took all the users and replaced them with numbers. And then we just had sparsely populated movie ratings in this matrix, seemingly anonymous. There's no names of any kind of. But the problem is, is that each row is statistically unique, meaning it kind of is its own fingerprint. And so two months after the data set was published, some researchers at UT Austin, I think it was, I think it was UT Austin, were able to go and scrape IMDb and basically create the same matrix in IMDb and then just compare the two. And it turns out people that were into movie rating, were into movie rating and were watching movies at similar times and similar patterns and similar tastes, right? And they will de anonymize this first data set with a high degree of accuracy. Happened again with there's a famous case of like medical records for like, I think might even been a Massachusetts senator. I think it was someone in the Northeast being de anonymized through very similar techniques. So one person goes and buys an anonymized medical record data set over here that has birth date and zip code and this one does zip code and gender, and this one does zip code, gender and whether or not you have cancer, right? And when you get all these together, you can start to sort of use the uniqueness in each one to re link it all back together. I mean, this is so doable to the extreme that I unfortunately know of companies whose business model is to buy anonymized data sets, de anonymize them and sell market intelligence to insurance companies. But it can be done. And the reason that it can be done is that just because the data set that you are publishing, the one that you are physically looking at, doesn't seem like it has Social Security numbers and stuff in it, does not mean that there's enough unique statistical signal for it to be linked to something else. And so when I say maximum amount of epsilon, epsilon is an upper bound on the statistical uniqueness that you're publishing in a data set. Right? And so what this tool represents is saying, okay, apply however much noise you need to, given whatever computational graph led back to private data for this tensor. Right? To ensure that, to put an upper bound on the potential for linkage attacks. Right? Now if you said Epsilon 0, okay, then that's saying effectively, like there's the. I'm only going to allow out patterns that have occurred at least twice. Right? Okay, so meaning, meaning two different people had this pattern and thus it's not unique to either one. Yes. So what happens if you perform the query twice? So the random noise would be re randomized and sent again. And you're absolutely, absolutely correct. So this epsilon, this is how much I'm spending with this query. So if I ran this three times, I would spend Epsilon of 0.3. Does that make sense? So this is a 0.1 query. If I did this multiple times, the epsilons would sum up. And so for any given data science project. Right. I should I. What we're advocating is that you're given an epsilon budget that you're not allowed to exceed. Right. No matter how many queries that you participate. Now there's another sort of subfield of differential privacy that's looking at sort of single query approaches, which is all around synthetic data sets. So how can I perform sort of one query against the whole data set and create a synthetic data set that has certain invariances that are desirable. Right. So I can do good statistics on it, but then I can query this as many times as I want because basically you can't. Yeah. Anyway, but we don't have to get into that now. Does that answer your question? Cool. Awesome. So now you might think, okay, this is like a lossless cause. Like how can we be answering questions while protecting, while keeping statistical signal gone? But like, it's the difference between, it's the difference between if I have a data set and I want to know what causes cancer, right? I could query a data set and learn that smoking causes cancer without learning that individuals are or are not smokers. Does that make sense? Right. And the reason for that is, is that I'm specifically looking for patterns that are occurring Multiple times across different people. And this actually happens to really closely mirror the type of generalization that we want in machine learning statistics. Anyways, does that make sense? Like as machine learning practitioners, we're actually not really interested in the one offs. I mean, sometimes our models memorize things, this happens, but we're actually more interested in the things that are not specific to you. I want the things that are going to work, the heart treatments that are going to work for everyone in this room. Not just obviously if you need a heart treatment, that would be cool for you to have one. But what we're chiefly interested in are the things that generalize, which is why this is realistic and why with continued effort on both tooling and the theory side, we can have a much better reality than today. Cool. So pros, just to review. So first, remote execution allows data to remain in the remote machine. Search and sampling. We can feature engineer using toy data differential privacy. We can have a formal, rigorous privacy budgeting mechanism. Right? How is the privacy budget set? Is it defined by the user or is it defined by the data set owner or someone else? This is a really, really interesting question actually. So first, it's definitely not set by the data scientist because that would be a bit of a conflict of interest. And at first you might say it should be the data owner. Okay, so the hospital that's trying to cover their butt, right. And make sure that their assets are protected both legally and commercially. Right. So they're trying to make money off this. So there's, there's sort of proper incentives there. But the interesting thing, and this gets back to your question, is what happens if I have say a radiology scan in two different hospitals, right. And they both spend one epsilon worth of my privacy in each of these hospitals, right. That means that actually two epsilon of my private information is out there, right? And it just means that one person has to be clever enough to go to both places to get the join. This is actually the exact same mechanism we were talking about a second ago when someone went from Netflix to IMDb, right? And so the true answer of who should be setting epsilon budgets, although logistically it's going to be challenging. We'll talk about a little bit in part two of the talk, but I'm going a little bit slow, but okay, is it should be us. It should be people around their own information. You should be setting your personal epsilon budget. That makes sense. That's an aspirational goal. We've got a long way before we can get to that level of infrastructure around these Kinds of things. And we can talk about that, and we can definitely talk about more of that in the kind of question answer session as well. But I think in theory that's what we would want. Okay. The two cons that we still. Two weaknesses of this approach that we still lack are. Someone asked this question. I think it was you. Yeah, you asked the question. So first, the data is safe, but the model is put at risk. And what if we need to do a join? Actually, actually, yours is a third one, which I should totally add to the slide. So first, if I'm sending my computations, my model into the hospital to learn how to be a better cancer classifier, my model's put at risk. It's kind of a bummer if, like this is a $10 million health care model. I'm just sending it to 1,000 different hospitals to get learned to learn. So that's potentially risky. Second, what if I need to do a joined computation across multiple different data owners who don't trust each other? Right. Who sends whose data to whom? And thirdly, as you pointed out, how do I trust how these computations are actually happening the way that I am telling the remote machine that they should happen? This brings me to my absolute favorite tool, secure multi party computation. Come across this before. Raise them high. Okay, cool. Little bit above average. Most machine learning people have not heard about this yet and I absolutely. This is the coolest. This is the coolest thing I've learned about since learning about like AI machine learning. This is a really, really cool technique, encrypted computation. How about Home Morph encryption? You come across Home Morph encryption. Okay, a few more. Yeah, this is related to that. So first, the kind of textbook definition is like this. So if you went on Wikipedia, you'd see secure MPC allows multiple people to combine their private inputs to compute a function without revealing their inputs to each other. Okay, but in the context of machine learning, the implication of this is multiple different individuals can share ownership of a number. Okay. Share ownership of a number. Show you what I mean. So let's say I have the number five, my happy smiling face, and I split this into two shares, two and a three. Okay, I've got two friends, Marianne and Bobby, and I give them these shares. They are now the shareholders of this number. Okay, and now I'm going to go away and this number is shared between them. Okay? And this, this gives us several desirable properties. First, it's encrypted from the standpoint that neither Bob nor Marianne can tell what number is encrypted between them by looking at their own share by itself. Now I've. For those of you who are familiar with kind of cryptographic math, I'm hand waving over this a little bit. This would typically be so in, in decryption would be adding the shares together, modulus a large prime. So these would typically look like sort of large pseudo random numbers. Right. But for the sake of making it sort of intuitive, I've picked pseudo random numbers that are convenient to the eyes. So first, these two values are encrypted, and second, we get shared governance, meaning that we cannot decrypt these numbers or do anything with these numbers unless all of the shareholders agree. Okay. But the truly extraordinary part is that while this number is encrypted between these individuals, we can actually perform computation. Right. So in this case, let's say we wanted to multiply these shares times the encrypted number times two. Each person can multiply their share times two and now they have a encrypted number ten. Right. And there's a whole variety of protocols allowing you to do different functions, such as the functions needed for machine learning while numbers are in this encrypted state. Okay. And I'll give some more resources for you if you're interested in kind of learning more about this at the end as well. Now, the big tie in models and data sets are just large collections of numbers which we can individually encrypt, which we can individually share governance over. Now, specifically to reference your question, there's two configurations of secure mpc, active and passive security. In the active security model, you can tell if anyone does computation that you did not sort of independently authorized, which is great. So what does this look like in practice when you go back to the code? So in this case, we don't need just one worker, it's not just one hospital, because we're looking to have shared governance, shared ownership amongst multiple different individuals. So let's say we have Bob, Alice and Teo and a crypto provider, which we won't go into now, and I can take a tensor instead of calling send and sending that tensor to someone else. Now I call share and that splits each value into multiple different shares and distributes those amongst the shareholders. Right. So in this case Bob, Alice and Teo, however, in the frameworks that we're working on, you still get kind of the same Pytorch like interface. And all the cryptographic protocol happens under the hood. And the idea here is to make it so that we can sort of do encrypted machine learning without you necessarily having to be a cryptographer. Right. And vice versa. Cryptographers can improve the algorithms and machine learning people can automatically inherit them. So kind of classic open source machine learning library, making complex intelligence more accessible to people, if that makes sense. And what we can do on tensors, we can also do on models. So we can do encrypted training and encrypted prediction. And we're going to get into what kind of awesome use cases this opens up in a bit. And this is a nice set of features. Right. In my opinion, this is sort of the MVP of doing privacy preserving data science. Right. The idea being that I could have remote access to a remote data set. I can learn high level latent patterns like what causes cancer without learning whether individuals have cancer. I can pull back just that sort of high level information with formal mathematical guarantees over, over the filter that's coming back through here. I can work with data sets from multiple different data owners while making sure that each individual data owners are protected. Now what's the catch? So first is computational complexity. So encrypted computation, secure mpc. This involves sending lots of information over the network. I think the state of the art for deep learning prediction is that this is a 13x slowdown over plaintext, which is inconvenient but not deadly. Right. But you do kind of have to understand that that assumes it's like two AWS machines sort of like talking to each other. They're relatively fast. But we also haven't had any hardware optimization to the extent that Nvidia did a lot for deep learning. There'll be probably some sort of Cisco player that's similar for doing kind of encrypted or secure NPC based deep learning. Right. Let's see. So this brings us back to kind of the fundamental question. Is it possible to answer questions using data we cannot see? The theory is absolutely there. That's something that I feel reasonably confident saying the sort of the theoretical frameworks that we have. And actually the other thing that's really worth mentioning here is that these come from totally different fields, which is why they kind of haven't been necessarily combined that much yet. I'll get more into that in a second. But it's my hope that by sort of, by considering what these tools can do, that'll open up your eyes to the potential that in general we can have this new ability to answer questions using information that we don't actually own ourselves. Because from a sociological standpoint, that's net new for like us as A species, if that makes sense. If ever previously we had to have like a trusted third party who would then take all the information in themselves and make some sort of neutral decision. Decision. Right. So we'll come to that in a second. And so one of the big sort of long term goals of our community is to make infrastructure for this secure enough and robust enough. And of course in like a free Apache 2 open source license kind of way that, you know, information on the world's most important problems will be this accessible. Right. And we can spend sort of less time working on tasks like that and more time working on tasks like this. So this is going to be kind of the breaking point between sort of part one and part two. Part two will be a bit shorter. But if you're interested in sort of diving deeper on the technicals of this, here's like a six or seven hour course that I taught just on these concepts and on the tools. It's free. On Udacity, feel free to check it out. So the question was, he's asking about how I specified that a model can be encrypted during training. Is that same as homework encryption or that something else? So a couple years ago there was a big burst in literature around training on encrypted data where you would homorphically encrypt the data set. And it turned out that some of the statistical regularities, homomorphic encryption, allowed you to actually train on that data set without, without decrypting it. So this is similar to that, except the one downside to that is that in order to use that model in the future, you have to still be able to encrypt data with the same key, which often is sort of constraining in practice. And also there's a pretty big hit to privacy because you're training on data that inherently has a lot of noise added to it. What I'm advocating for here is instead we actually encrypt both the model and the data set during training, but inside the encryption, inside the box, it's actually performing the same computations that it would be doing in plaintext. So you don't get any degradation in accuracy and you don't get tied to one particular public private key pair. Yeah, yeah. So the question was kind of comment on federated learning, specifically Google's implementation. So I think Google's implementation is great. And obviously the fact that they've shown that this can be done, hundreds of millions of users is incredibly powerful. I mean, even inventing the term and creating momentum in that Direction. I think that there's one thing that's worth mentioning is that there are two forms of federated learning. One is sort of the one where your model is federated learning. Sorry. Ooh, gotta talk about what that is. Okay, yes, I'll do that quickly. So federated learning is basically the first thing I talked about. So remote execution. So if everyone has a smartphone, when you plug your phone in at night, if you've got Android or iOS, you plug your phone in at night and attach to WI Fi, you know, when you text and it recommends the next word, next word, prediction. That model is trained using federated learning, meaning that it learns on your device to do that better. And then that model gets uploaded to the cloud, as opposed to uploading all of your tweets to the cloud and training one global model. Does that make sense? So you plug your phone at night, model comes down, trains locally, goes back up. It's federated, Right? That's basically what federated learning is, a nutshell. And it was pioneered by the Corp team at Google and they do really fantastic work. They've paid down a lot of the technical debt, a lot of the risk or technical risk around it, and they published really great papers outlining sort of how they do it, which is fantastic. What I outlined here is actually a slightly different style of federated learning, because there's federated learning with a fixed data set and a fixed model and lots of users where the data is very ephemeral, like phones are constantly logging in and logging off. You're plugging your phone at night and then you're taking it out. This is sort of the one style of federated learning. It's really useful for product development. So it's useful for. If you want to do a smartphone app that has a piece of intelligence in it, but training that intelligence is going to be prohibitively difficult for you to get access to the data for. Or you want to just have a value prop of protecting privacy. That's what federated learning, that style of federated learning is good for. What I've outlined here is a bit more exploratory federated learning, where it's saying, okay, instead of the model being hosted in the cloud and data owners showing up and making it a bit smarter every once in a while now the data is going to be hosted at a variety of different private clouds, right? And data scientists are going to show up and say, hmm, I want to do something with diabetes today, or I want to do something with studying dementia today. Something like that, right? This is much more difficult because the attack vectors for this are much larger. Right. I'm trying to be able to answer arbitrary questions about arbitrary data sets in a protected environment. Right. So I think, yeah, that's kind of my general thoughts on. Does federated learning leak in information? So federated learning by itself is not a secure protocol. Right, to the extent that. And that's why sort of this ensemble of techniques that I've. So the question was, does federated learning leak information? So it is perfectly possible for a federated learning model to simply memorize a data set and then spit that back out later. You have to combine it with something like differential privacy in order to be able to prevent that from happening. Does that make sense? So just because the training's happening on my device does not mean it's not memorizing my data. Does that make sense? Okay, so now I want to zoom out and go a little less from kind of the data science practitioner perspective, and now take more the perspective of like an economist or political scientist or someone looking kind of globally at like, okay, if this becomes mature, what happens? Right. And this is where it gets really exciting. Anyone? Entrepreneurial? Anyone? Everyone? I don't know. No one? Okay, cool. Well, this is the part for you. So the big difference is this ability to answer questions using data you can't see, because as it turns out, most people spend a great deal of their life just answering questions, and a lot of it is involving sort of personal data. I mean, whether it's minute things like where's my water? Where are my keys? Or what movie should I watch tonight? Or what kind of diet should I have to be able to sleep? Well, I mean, a wide variety of different questions. Right. And we're limited in our answering ability to the information that we have. Right. So this ability to answer questions using data we don't have sociologically, I think is quite important. And there's four different areas that I want to highlight as like, big groups of. Of use cases for this kind of technology to help kind of inspire you to see where this infrastructure can go. And actually, before I jump into that, has anyone been to Edinburgh? Edinburgh, cool. Just tour like the castle and stuff like that. So my wife and I, it's my wife Amber, we went to Edinburgh for the first time six months ago. September. September. And we did the underground. Was it the. We did a ghost tour. Yeah, we did the ghost tour. And it was really cool. There was one thing that took away from it. There was this point we were standing. We just walked out of the Tunnels. And she was pointing up at some of the architecture. And then she started talking about basically the cobblestone streets and why the cobblestone streets are there. Cobblestone streets. One of the main purposes of them was to sort of lift you out of the muck. And the reason there was muck there is that they didn't have any internal plumbing. And so the sewage was just poured out into the street, right? Because you live in a big city and this was the norm everywhere, right? And actually, I think she even sort of implied that, like, the invention or popularization of the umbrella had less to do with actual rain, a bit more to do with buckets of stuff coming down from on high, which is. It's a whole different world, like, when you think about what that is. But the reason that I bring this up is that, you know, however many hundred years ago, people were walking through, you know, sludge, sewage was just everywhere, right? It was all over the place, and people were walking through it everywhere they go, and they were wondering why they got sick, right? And in many cases, and it wasn't because they wanted it to be that way. It was just because it was a natural consequence of the technology they had at the time, right? This is not malice. This is not anyone being good or bad or evil or whatever. It's just the way things were. And I think that there's a strong analogy to be made with kind of how our data is handled as a society at the moment, right? We've just sort of walked into a society. We've had new inventions come up and new things that are practical, new uses for it. And now everywhere we go, oh, we're constantly spreading and spewing our data all over the place, right? I mean, every camera that sees me walking down the street, good news, there's a company that takes a whole picture of the earth by satellite every day. Like, how the hell am I supposed to do anything without everyone following around all the time, right? And I imagine that whoever it was, and I'm not a historian, so I don't really know, but whoever it was that said what if. What if we ran plumbing from every single apartment, business, school, maybe even some public toilets underground under our city, all to one location, and then processed it, used chemical treatments and then turned that into usable drinking water. Like, how laughable would that have been? Would have been just the most massive logistical infrastructure problem ever to take a working city, dig up the whole thing, to take already constructed buildings and run pipes through all of them. I mean, so Oxford, gosh, There's a building there that's so old they don't have showers because they didn't want to run the plumbing for the head. You have to ladle water over yourself. It's in Merton College. It's quite famous. Right. I mean, the infrastructure, anyway, the infrastructure challenges, it just must have seemed absolutely massive. And so as I'm about to walk through kind of like four broad areas where things could be different theoretically based on this technology, and I think it's probably going to hit you like, whoa, that's a lot of code. We're like, whoa, that's a lot of change. But I think that the need is sufficiently great. I think that. I mean, if you view our lives as just one long process of answering important questions, whether it's where we're going to get food or what causes cancer, making sure that the right people can answer questions without. Without data just getting spewed everywhere so that the wrong people can answer their questions is important. And. Yeah, anyway, so I know this is going to sound like there's a certain ridiculousness to maybe what some of this will be, but I hope that you at least see that theoretically the basic blocks are there and that really what stands between us and a world that's fundamentally different is adoption, maturing of the technology and good engineering. Because I think once Sir Thomas Crapper invented the toilet. Right, I do remember that one. At that point, the basics were there and what stood between them was implementation, adoption in engineering. And I think that that's where we are. And the best part is we have companies like Google that have already paved the way with some very, very large rollouts of the early pieces of this technology. Cool. So what are the big categories? One, I've already talked about open data for science. Okay. So this one is a really big deal. And the reason it's a really big deal is mostly because everyone gets excited about making AI progress. Right? Everyone gets super excited about superhuman ability in X, Y or Z. When I started my PhD at Oxford, I work for, my professor's name is Phil Blunsom. The first thing he told me when I sat my butt down in his office on my first day as a student, he said, andrew, everyone's going to want to work on models. But if you look historically, the biggest jumps in progress have happened when we had new big data sets or the ability to process new big data sets. And just to give a few anecdotes, Imagenet. Right? Imagenet GPUs allowing us to process larger data sets, even things like Alphago this is synthetically generated infinite data sets. Or if I don't know, did you guys, Anyone watch the Alphastar live stream on YouTube? It talked about how it had trained on like 200 years of StarCraft, right? Or if you look at Watson playing Jeopardy, right, this was on the heels of a new large structured data set based on Wikipedia. Or if you look at Garry Kasparov and IBM's Deep Blue, this was on the heels of the largest open data set of chess matches haven't been published online. There's this echo where like, big new data set, big new breakthrough, big new data set, big new breakthrough, right? And what we're talking about here is, is potentially several orders of magnitude more data relatively quickly. And the reason for that is that I'm not saying we're going to invent a new machine and that machine is going to collect this and then it's going to go online. I'm saying there's thousands and thousands of enterprises, millions of smartphones, there's hundreds of governments that all already have this data sitting inside of data warehouses, right? Largely untapped for two reasons, One, legal risk and two, commercial viability. Right? If I give you a data set, all of a sudden I just doubled the supply, right? What does that do to my billing ability? And there's the legal risk that you might do something bad with it. That comes back to hurt me with this category. I know it's like just one phrase, but this is like imagenet, but for every data task that's already been established, right? This is us, like, I mean, we're working with a professor at Oxford in a psychology department who wants to study dementia, right? The problem with dementia is every hospital has like five cases, right? It's not like a very centralized disease. Not like all the cancer patients go to one big center and it's where all the technology is like dementia. It's sprinkled everywhere. And so the big thing that's blocking him as a dementia researcher is access to data. And so he's investing in private data science platforms. And I didn't persuade him to. I found him after he was already looking to do that. But pick any challenge where data is already being collected. And this can unlock not larger amounts of data that exists, but larger amounts of data that can be used together. Does that make sense? This is like 1000 startups right here. Whereas instead of going out and trying to buy as many data sets as you can, which is a really hard and really expensive task, talk to anyone who's in Silicon Valley Right now trying to do a data science startup, right? Instead you go to each individual person that has a data set and you say, hey, let me create a gateway between you and the rest of the world that's going to keep your data safe and allow people to leverage it, right? That's like repeatable business model. Pick a use case, right? Be the radiology network gatekeeper, right? Okay, so enough on that one. But does it make sense how on a huge variety of tasks, just the ability to have a data box silo that you can do data science against is going to increase the accuracy of a huge variety of models really, really, really quickly. Cool. All right, second one. Oh, that's not right. Single use accountability. This one's a little bit tricky. Get to the airport and you get your bag checked, right? Everyone's familiar with this process, I assume what happens, someone's sitting at a monitor and they see all the objects in your bag so that occasionally they can spot objects that are dangerous or illicit, right? There's a lot of extra information leakage, virtue of the fact that they have to sit and look at thousands of all the objects, basically searching every single person's bag totally and completely just so that occasionally they can find that one. The question they actually wanted to answer is, is there anything dangerous in this bag? But in order to answer it, they have to basically acquire access to the whole bag, right? So let's think about the same approach of answering questions using data we can't see. The best example of this in the analog world is a sniffing dog. Familiar with like sniffing dogs. So give your bag a whiff at the airport, right? This is actually a really privacy preserving thing because dogs don't speak English or any other language. And so the benefit is dog comes by, nope, everything's fine, moves on. The dog has the ability to only reveal one bit of information without you having to search every single bag. That is what I mean when I say a single use accountability system. It means I am looking at some data stream because I'm holding someone accountable and we want to make it so that I can only answer the question that I claim to be looking into. So if this is a video feed, for example, instead of getting access to the raw video feed and the millions of bits of information, every single person in the frame of view walking around doing whatever which I could use for even if I'm a good person, I technically could use for other purposes, but instead build a system where I build say a machine learning classifier that is an auditable piece of technology. That looks for whatever I'm supposed to be looking for. And I only see frames, I only open up bags that I actually have to. This does two things. One, it makes all of our accountability systems more privacy preserving, which is great, mitigates any potential dual or multi use. Right. And two, it means that some things that were simply too off limits for us to properly hold people accountable might be possible. Right. One of the things that was really challenging so we used to do email surveillance, digital reasoning, and it was basically help investment banks find insider traders because they want to help enforce the loss. They get fined billion dollar fines if, if anyone caused an infraction. But one of the things that was really difficult about developing these kinds of systems was that it's so sensitive. We're talking about hundreds of millions of emails at some massive investment bank. There's so much private information in there that say none of our data scientists, barely any of them were able to actually work with the data and try to make it better. And this, yeah, this makes it really, really difficult. Anyway, cool. So enough on that third one, and this is the one I think is just incredibly exciting. End to encrypted services. WhatsApp, everyone familiar with WhatsApp, Telegram, any of these, these are messaging apps, right? Where a message is encrypted on your phone and it's sent directly to someone else's phone and only that person's phone can decrypt it. Right. Which means that someone can provide a service, messaging, without the service provider seeing any information that they're actually providing the service over. Right? Very powerful idea. What if the intuition here is that with a combination of machine learning, encrypted computation and differential privacy that we could do the same thing for entire services. So imagine going to the doctor, okay, so you go to the doctor. This is really a computation between two different data sets. On the one hand you have data set that the doctor has, which is their, you know, medical background, their knowledge of different procedures and diseases and tests and all this kind of stuff. And then you have your data set, which is your symptoms, your medical history, your recent things that you've eaten, your genes, your genetic predisposition, your heritage, those kinds of things. And you're bringing these two data sets together to compute a function. And that function is what treatment should you have, if any. And the idea here is that, so there's this new, new field called structured transparency. I guess I should probably mention, I'm not sure, I'm not even sure you can call it a new field yet because it's not in the literature, but it's been bouncing around a few different circles. And the, and it's, it's F xy. I'm not very good with chalk, sorry. And then this is Z. Okay? So this two different people providing their data together, computing a function and an output. So differential privacy protects the output. Encrypted computation. So like mpc, which we talked about earlier, protects the input, right? So it allows them to compute F of X of Y, right, Without revealing their inputs. Remember this? So basically encrypt Y, encrypt X, compute the function while it's encrypted. Do we remember, do we remember this? Right? And so there's three processes here, right? There's input privacy, which is npc, there's logic, and then there's output privacy. And this is what you need to be able to do end to end encrypted services. Okay? So imagine, imagine. So there are machine learning models that can now do skin cancer prediction, right? So I can take a picture of my, of my arm and send it to a machine learning model and it'll predict whether or not I have melanoma on my arm, right? Okay. So in this case, machine learning model, perhaps owned by a hospital or startup image of my arm. Okay? Encrypt both. The logic is done by the machine learning model. The prediction, if it's gonna be published to the output, to the rest of the world, use differential privacy. But in this case, the prediction can come back to me and only I see the decrypted result. Okay? The implication being that the, the, the doctor role facilitated by machine learning can classify whether or not I have cancer, can provide this service without anyone seeing my medical information. I can go to the doctor and get a prognosis without ever revealing my medical records to anyone, including the doctor. Right? Does that make sense? And if you believe, if you believe that sort of the services that are repeatable that we do for millions and millions of people, right, can create a training data set that we can then train a classifier to do, then we should be able to upgrade it to be end to end encrypted. Does that make sense? So again, it's kind of big. It assumes that AI is smart enough to do it. There's lots of questions around quality and quality assurance and all these kinds of things that have to be addressed. There's very likely to be different institutions that we need. But I hope that at least these three sort of big categories, this is by no means comprehensive, but I hope at least these three big Categories will be sort of sufficient for helping, sort of lay the groundwork for how sort of each person could be empowered with sole control over the only copies of their information while still receiving the same goods and services they've become accustomed to. Cool. Thanks. Questions? Let's do it first, please give Andrew a big hand. Andrew. It was fascinating, really, really fascinating. Amazing, amazing set of ideas and hopefully this can really get rid of the sewage of data on this vision of end to end encrypted services. If I understand correctly, the algorithm would also run on two or more services and the skin image would go to them and then you would get the diagnosis. But the diagnosis itself is not private though, because the output of that is being revealed to the service provider. So it could optionally be revealed to the service provider. So in this case. Oh yeah, something I didn't say for secure NPC for encrypted computation, except for with some exceptions. But for secure mpc, when you perform computation between two encrypted numbers, the result is encrypted between the same shareholders, if that makes sense. Meaning that by default Z is still encrypted with the same keys as X and Y, and then it's up to the key holders to decide who they want to decrypt it for. So they could decrypt it for the general public, in which case they should apply differential privacy. They could decrypt it for the input owner, in which case the input owner is not going to hurt anybody else by him knowing whether he has a certain diagnosis. Or it could be decrypted for the model owner, perhaps to allow them to do more training or some other arbitrary use case. So it can be, but not as a strict requirement, just to be sure. I mean, if Z is being computed by say two parties to send Z back to Y, in this case the machine knows what Z is. So in that sense, even if you encrypt Z with the key of Y, there's no way to protect the output itself. I haven't described this correctly. So. When we perform the encrypted computation, we split this into shares. So we'll say, you know, Y, Y1 and Y2, right? Y2 goes up here, right? And then this populates, what actually happens is this creates Z1 and Z2 at the end, right? Which is still, which is still owned by, you know, person Y. So we'll say this is Alice and this is Bob, right? So we have Bob's share and Alice's share. What gets populated is shares of Z. So if Alice or If Bob sends his share of Z down to Alice, only Alice can decrypt the result. Does that make more sense? Okay, cool. Even the answer is. Even the answer is protected. Yep. And you would only need to use differential privacy in the case that you're planning to decrypt the result for some unknown audience to be able to see privacy. Some models are biased based on real data biases and society tries to make unbiased models like on gender, race and so on. How does it work with privacy, especially when everything is encrypted. So how can you unbiased models when you do not see biases in the data and so on. That's a great question. So the first, the first gimme for that is that people don't ever really debias a model by physically reading the weights, Right? So the fact that the weights are encrypted doesn't necessarily help or hurt you. So really what it's about is just making sure that you provision enough of your privacy budget to allow you to do the introspection that you need to be able to measure and adjust for bias. So I think that's sufficient. Does that make sense? Cool. Awesome. Great question then. How far away do you think we are from organizations like the FDA requiring differential privacy to be used in regulating medical algorithms? So I think the best answer I can give to that, so one, I don't know. And even laws in the UK regarding privacy like GDPR are not prescriptive about things like differential privacy. But I think the best and most relevant data point I have for you on that is that the U.S. census this year is going to be protecting the Census data, the 2020 Census data, using differential privacy. And some of the leading work on actually applying differential privacy in the world is going on at the U.S. census. And I'm sure they'd be interested in more helpers if anyone was interested in joining them. So I guess her question was kind of one of my questions, but it was more just like how much buy in are you getting in terms of adoption for open mind or any of like do you have like any hospitals that are like participating or. Yeah, so actually there's a few things I probably should have mentioned. So the. So open mind is about two and a half years old. In the very beginning we had very little buy in because it was just so early. It was kind of like, who cares about privacy? No one's ever going to sort of really care about that post gdpr. Total, total change. Right. Everyone's scrambling to protect the data, but the Truth is, it's not just privacy, it's also commercial usability. Right now, if you're selling data, every time you sell it, you lower the price because you increase the supply and you increase the number of people that are also selling it. So I think that there's people also waking up to kind of the commercial reasons for protecting their own data sets and protecting the unique statistical signal that they have. It's also worth mentioning. So the Pytorch team recently sponsored $250,000 in source grants to fund people to work on our PISIF library, which is really good. And we're hoping to announce more grants of similar size later in the year. So if you guys like working on open source code and like to get paid to do so to that extent, that's sort of a big vote and buy in as far as our community is concerned. So this year is when I hope to see kind of the first pilots rolling out. There are some that are sort of in the works, but they're not public yet. But yeah. So I think basically this is the year for like pilots. I think that's about as far as we are. And then I have another question that's kind of on the opposite end of the spectrum. It's a little more technical. Weeds. Cool. So when you do the separate. When you're doing the encryption where you separate everything into each of the different owners, how does that work for nonlinear functions? Because I would just. You need that linearity to add it back and for it to maintain the totally. So the nonlinear functions are the most complicated, most performance intensive. You get the biggest performance hit when you have to do them. For deep learning specifically, there's kind of two trends. So one line of research is around using polynomial approximations and then the other line is around doing sort of discrete comparison functions. So which is good for relus and it's good for lopping off the ends of your polynomials so that your unstable tails can be flat. And I would say that's about that. And then like the science of kind of like trying to relax your security assumptions strategically here and there to get more performance is about where we're at. But as far as. The one thing that's worth mentioning though is that there are kind of what I described was secure MPC sort of on integers and fixed precision numbers. You can also do it on sort of binary. But in that sense you get a huge performance here for doing it with binary. But you also get the ability to do things more classically with computing encrypted Computation. It's sort of like doing computing in the 70s. Like you get a lot of the same kind of constraints. Thank you very much for your talk, Andrew. Yeah, I'm wondering about your objective to ultimately allow every individual to assign a privacy budget. You think? You also mentioned, you mentioned that it would take a lot of work to provide the infrastructure for that to be possible. Do you have an idea for what kind of infrastructure is necessary? And also when people are reluctant, even perhaps lazy, and they don't really care and they don't want their data to be protected, I guess it takes some training. But what are your thoughts on building that infrastructure? I think it's going to come in waves. It's the kind of thing where people don't usually invest money and time and resources into things that aren't like a straight shot to value. So I think there's probably going to be multiple individual discrete jumps. The first one is going to be just enterprise adoption. Enterprises are the ones that already have all the data. So they're the ones who are most natural to start adopting privacy preserving technologies. And I think that that adoption is going to be driven primarily by commercial reasons. For commercial reasons, meaning my data is inherently more valuable if I can keep it scarce while allowing people to answer questions with it, if that makes sense. So it's more profitable for me to not send copies of my data to people if I can actually have them bring their question answering mechanisms to me and just get their questions answered. Does that make sense? That's not a privacy narrative, but I think that that narrative is going to mature privacy technology quite quickly post sort of enterprise adoption. I think that that's when that's end to end. Encrypted services are still really hard at this point. And the reason for that is that they require lots of compute and lots of network overhead. Which means that you probably want to have something in the cloud. So some sort of machine that you can own and control in the cloud or have the Internet get a lot faster. But there's this question of how do we actually get to a world where each individual person sort of knows or has notional control over their own personal privacy budget? And let's just say you had perfect enterprise adoption and everyone's tracking their stuff with differential privacy. The piece that you're actually missing here is just some sort of communication between all the different enterprises that are joining up and making just an accounting mechanism. It's a lot like the irs. It's just someone to be there to make sure that you're not double spending in different places. Your epsilon budget that's over here versus over here versus over here is all coming from the same place. It's not totally clear who this actor would be. Right. Maybe there's an app that just does it for you. Maybe there has to be an institution around it. Maybe it won't happen at all. Maybe it'll just be decentralized, but whatever. Another option is that there'll actually be data banks. So there's been some literature in the last couple years around saying okay, maybe institutions that they currently handle your money might also be the bank where all of your information lives. Right. And that becomes the gateway to your data or something like that. So there's different things that are. Because that would obviously make the accounting much easier and also that would give you kind of that cloud to cloud sort of performance increase. I think it's clear we wouldn't go to data banks or these kinds of centralized accounting registries directly because you kind of have to have the initial adoption first. But if I had to guess it's something like that, we won't see that for a while and it's not even clear what that would look like. But I think it is possible. We just have to get through sort of non trivial adoption first. Yeah. So it's kind of hazy, but it's predicting the future. So I guess that's how that goes. I was just wondering if you can comment briefly on what you think is the biggest mistake being made with respect to recommendation systems transparency and if you can comment briefly on what you think might be the most or the best solution. So I don't know if this is a mistake. I would say the biggest opportunity for recommendation systems is that they have the potential to be more holistic. So for example, if you recommended a movie to me based on whether or not it's most likely to keep me engaged, keep me watching movies, it's not really a holistic recommendation. It's not like saying, hey, you should do this because it's going to make your life more fulfilling, more satisfied, whatever. It's just going to glue me to my television more. Right. So I think the biggest opportunity, and particularly with privacy preserving machine learning is that if a recommender system could have the ability to access private data without actually seeing it. Right. And answer the question, how do I give the best recommendation so that this person gets a good night's sleep or has more meaningful friendships or whatever like these attributes that are actually particularly sensitive. But there are things that we actually want to optimize for that, we could have vastly more beneficial recommendation systems than we do now just by virtue of having better infrastructure for dealing with private data. So they actually, as far as like, the biggest limitation recommender systems right now is just that they don't have access to enough information to have good targets. Does that make sense? We would like for them to have better targets, but in order to do that, they have to have access to information about those targets. And I think that's what sort of privacy preserving technologies could bring to bear on recommendation systems. Thanks. Yeah. Great question, by the way. One more time, please give Andrew a big hand. Thank you so much. Thanks for having me.