#27 Data Security, Data Privacy and the GDPR

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com.

In this episode of Data Framed, a Data Camp podcast, I'll be speaking with Kathryn Jarmul, a data scientist, consultant educator and co founder of KI Protect, a company that provides real time protection for your data infrastructure. Data Science and Artificial Intelligence. Catherine and I spoke on 25 May 2018 about data privacy, data security and the GDPR, or general data Protection Regulation, which went into a effect the week prior. What are the biggest challenges currently facing data security and privacy? What does the GDPR mean for civilians, working data scientists and businesses around the world? Is data anonymization actually possible or a pipe dream? Stick around to find out. I'm Hugo Baun Anderson, a Data Scientist at DataCamp and this is Data Framed. Welcome to Data Framed, a weekly Data Camp podcast exploring what data science looks like on the ground for working data scientists and what problems it can solve. I'm your host, Hugo Bown Anderson. You can follow me on Twitter Hugo Bown and Data Camp at Data Camp. You can find all our episodes and show notes@datacamp.com community podcast. Hi there Catherine, and welcome to Data Framed.

Hi Hugo, thanks for having me.

It's such a pleasure to have you on the show. So before we dive in to everything we're talking about, I just want to let you know I've been receiving far too many emails the past couple of weeks.

Oh no.

Do you have any indication why?

Oh, okay. Yes. Yeah, yeah. Emails that ask you, ask you for consent. This is consensual email at its best. It's emails asking you if you want to receive more emails, you know, and.

Why is this happening? What's been happening recently? That means I'm getting. My inbox is really full.

Yeah, yeah. So probably if you deal with data, you've already heard of GDPR or the General Data Protection Regulation, but it went into effect on May 25 and everybody got a lot of emails just talking about privacy. It was really fantastic. You know, it felt like you know, finally, consensual data collection. We're having lots of conversations, but yeah, I think most people just kind of deleted them all, which is for better or worse, maybe what what they all expected when it was sent as a mad rush on the final day for sure.

And it seems like a lot of it's opt in as well, isn't it?

Yeah. So the idea behind GDPR is that it's essentially, as I said, like this, consensual driven. So consent driven in the sense that you have a right to say, okay, I'm fine with you using my data in these ways, or I'm fine with you collecting my data in these ways, or I'm fine with you reaching me out to me in these ways. I think that's a really great step. I mean, I think consent for everything is really cool concept and consensual data collection is something that I think we are hopefully starting to realize is really center of doing ethical data science. And so I think GDPR is a nice step towards that. It gives a lot of rights to European residents and I wonder how it will also affect data collection for the rest of the world. It appears that people have different approaches for this, as in some people are essentially creating EU only versions of their platform. For example, like the USA Today EU site and a few of the other papers and publications published an EU only package. But what I am hoping is that it also allows for a little bit more consensual data collection of people even outside of the eu.

Great. And I think this provides a really nice teaser of the conversation we'll be getting into with respect to data security, privacy, the gdpr, whether it's enough what we might see outside the eu. But before all of that, I want to find out a bit about you and maybe you could start and tell us how you got into data science initially.

Yeah, so it was definitely by accident, as with kind of, I guess I would say a lot of people in my era of computing, which essentially was that I was a data journalist. I was working at the Washington Post. I had had some history and background in some computer science and statistics, but I didn't enter it directly after school and that I found myself in data journalism. After that I got recruited to work at a few startups doing some sort of larger scale data collection and data engineering essentially back in the initial Hadoop days. And from there I went kind of into some ops and security roles, automating deployment and kind of leading teams on those types of things, and then fell back into data doing some Data wrangling after my book was published with Jackie Casel, data wrangling with Python, and since then have been focused more on natural language processing and machine learning. And lately been thinking a lot about data privacy and data security. Kind of, I guess after 10 years in this business you start to like, think about the intersections of things you care about. And for me, that's definitely an important intersection that I think ties in a lot of the passions I have and experience I have in data science as a whole.

And where has that led you now? What are you working on currently?

Yeah, so I'm currently building a new startup called KI Protect. It's Ki after Kunstlich Intelligence, which is essentially the German translation of AI. And our idea and our goal, our solution really is to bring about kind of a data science, a data science compatible data security layer. So the idea is, from my experience and from our experience, Andreas Devis and I, we have seen that the security community and the data science community are not necessarily like overlapping in meaningful ways right now. And we're trying to think about how we can bring more data security and data privacy concepts to the data science community. That makes them really easy to use, really. I would say, like consumer friendly in a sense of being easily integratable into systems that you might use to process your data, like Apache, Kafka and Spark and so forth. And to make it so that you don't have to have data privacy or data security as a core concept of your data science team. You can just do normal data science and you can use our service to help you enforce privacy and security.

Great. So it sounds like you're essentially trying to help people keep doing as they're doing and you'll fill in this particular gap for them with respect to data security and data privacy.

Yeah, that's the goal, is to make it the plugin for data security or data privacy. Of course, this is a delicate and complex topic. So we're exploring kind of what we can guarantee and what integrations make sense for different types of companies. So we don't have like a full product spread available yet, but this is something that we're actively experimenting with, researching and working on. We're fairly confident that we can come up with a few different methods that allow you to use simple APIs for pseudonymization and anonymization of your data sets.

Awesome. So I want to now get into data security and data privacy. But before that you mentioned your love for nlp and I just wanted to let everyone know that you've also got a Great data camp course on fundamentals of NLP and Python, which I had the great pleasure of working on with you.

Yeah, it was super fun and I love all the great feedback and so forth from folks. And yeah, if you're starting to get into natural language processing or you're curious what it's all about, I can definitely recommend taking that as well as the follow up courses which allow for some fun experimentation with some of the common and best libraries in natural language processing.

Exactly.

So let's jump in. What are the biggest challenges currently facing data security and data privacy in your mind?

Yeah, so I think one thing that I've noticed over time is the core competency of most data scientists is not necessarily focused on security and privacy. Now we're starting to see perhaps with for example, the Apple Differential Privacy team and the Google Brain research that has been focused on security and machine learning, more overlap. But the average person who has studies statistics or machine learning and who's doing this in the field, they don't necessarily have a strong background in computer security or in data security or infosec as we might call it. Right. And this is not their, you know, I don't see this as a fault of theirs. It's nothing lacking. Right. They have a lot of their own specialized training. But the unfortunate circumstance of that is that a lot of the way that we manage and handle data is not necessarily the most secure way. And it definitely doesn't always take the ideas of privacy or even user sensitivity in the sense of do I actually need access to full user data? It doesn't really take these into account very often. And therefore as data scientists, we have access to potentially millions of people's personal data, their messages, their emails or chats, their purchase history. We have access to all of these things. And you know, kind of my question is, do we actually need access to all of these to do our job properly? And I think this is perhaps a big oversight in terms of how we've built up the data management and data science and BI platforms that we use today.

You spoke to the lack of focus or knowledge with respect to data security. Do you think this is related to the lack of focus on kind of building ethical systems in general for data science?

Well, yeah, I mean, one of the conversations I've found myself having recently in lieu of the GDPR is that it's been really painful for people to implement consensual data collection in their data science. And why is that? It's because the software is not designed with the user in mind. Right. So the software is maybe designed with the end user, the internal team in mind, but it's often not designed with the actual customer or the client in mind. If we had software that was slightly more driven by the client's desires or demands, like this kind of touches upon design thinking, right? Then it should be cognizant that when we collect user data that we have marked, when they consented, that we have marked, what is the provenance of the data that we have marked, how was the data processed? And of these things, and the fact that data provenance has been more of an aspect of research than actually an aspect of every type of data collection software that you can imagine, this is really problematic, right, because we have accumulated all this data and for some larger corporations, sometimes they have purchased data or they have aggregated data from data marketplaces and so forth, and this means that they now have all of this data, some of which was given directly and consensually, and some of it which was just collected by purchasing power or by buying another company and so forth. And so, you know, this is a nightmare, of course, when it comes to gpr and you have to figure out and sort out what data was given by whom and under what circumstances. But why, why do we have a, why might we have this problem in the first place? Like, why can't we just have perhaps data marketplaces where consumers directly sell their data if they're going to do that? Or also why isn't data provenance and essentially like where this data come from and when does it expire? How long is it good for? Why aren't these like a normal part of how we do data management from the beginning?

I'm interested in how you feel the average, if this is even a well formed question, how the average data scientist responds to, you know, this type of legislation being passed. And if you can't speak to the average, maybe you could give a variety of responses that you think are paradigms of how the community's responding.

Yeah, I mean, I think that, I guess I would say that I have a feeling people are inherently good and want to build ethical systems. This is like where the viewpoint that I'm coming from and I think that a lot of people are like, okay, this is painful, but I want to be able to do the right thing. I want to be able to do ethical data science. What does this mean? How might I have to change the ways that I currently process this data? So I think it's sparking a lot of conversations that are thinking, okay, well perhaps in the Past we haven't done this very well. How might we start again? Or how might we better do this in the future? But then I do think that there are some people that are just like, I don't see it as a nuisance. And, you know, there's been this big rash of variety of software and other platform vendors that are simply saying, oh, well, we're not going to sell to EE residents anymore. And I see this as terrifying, right, because why would I want to use a service that can't guarantee that they're going to ask me if they can use my data? Right. This is, I think, you know, shows that there's essentially, I would, I would argue that there's a big divide between those that see privacy as a burden and those that see privacy as maybe something that we can strive for, that we need to think about and perhaps change the systems and processes that we use in the meantime.

And how do you think data scientists generally feel about the idea of sacrificing some model performance for having more ethical models?

Yeah, I mean, I think that this is difficult. So I've spoken on the topic of ethical machine learning a few times now, and a few times the reaction was very negative. And people were like, well, I don't really see why this is my problem. And I think that unfortunately there is some of that idea like, well, if black folks are treated differently by cops, why should I have to essentially change the distribution of my data set to compensate for this? Right. They say, well, the data is there and that's what the data says. And so I'm just going to build exactly what the data says. And I would say that that is a choice and an action in and of itself. And if you're making that choice in action, you're essentially autom inequalities and you're automating biases, societal biases. And when you choose to do that, you're making a statement. And I would say that the statement is that you say that those biases are valid. You say that it's valid that people are treated differently based on their skin color from police, or that women earn less than men. This is something that you're validating if you just say, well, that's what the underlying statistics of my sample say. So that's what I'm going to do. And so I've definitely had those conversations numerous times. And then I've also had conversations with people like, oh, wow, this, this is really cool. This makes sense. Like, it's so nice to know that there's Quite a lot of different performance metrics you can use to analyze the ethics or the treatment of different groups from your model. And I think that there is also, like new energy, specifically around fat ML and everything that's happening in academia around finding real ways to build ethical models that don't necessarily sacrifice much performance at all.

Yeah. And I think something you mentioned there is that some people have responded. It's not my job to think about these things. One thing that data science doesn't have as a profession yet is standards of practice, codes of conduct, necessarily. And if we think back what's happened historically in other professions, in ancient Greece, the Hippocratic oath was developed to deal with these types of things for people practicing medicine, right?

Yeah, yeah, yeah. And I think that, you know, if you're building some system that maybe controls in some IoT factory device where no humans are affected at all by what you're doing, or if you're making some sort of academic model. Yeah, okay, maybe your impact is very small. Right. But when we're building these systems that interact with humans and now quite a lot that interact directly with like we would say the consumer. Right. Or a person and affects maybe what the person sees, what they click on, what they think about, what price they pay, and then of course, the massive systems like finance and justice and so forth, this is like the impacts we have a growing footprint of the things that data science touches and affects. And because of this, I think that we need to start thinking about if we don't have a Hippocratic oath, both, like, what do we have? Right.

And I do think there are so many more, increasingly more and more such examples emerging. I think one of the ones that I've mentioned a few times on, on the podcast is judges using the output of a black box model that tells recidivism rate for. For incarcerated people, using the output of that model as input for the parole hearing.

Right, yeah.

And actually Kathy o' Neill's book Weapons of Math Destruction, which I recommend to everyone who wants to think about these. These things and actually probably recommend it for more. More to people who don't want to think about these things to check out that book.

Yeah. There's a new one also called Automating Inequality, which is quite good, so I can recommend that one as well.

Yeah, we'll link to those in. In the show notes. We'll jump right back into our interview with Catherine after a short segment. Let's now jump into a segment called Rich, Famous and Popular with Greg Wilson, who wrangles instructor Training here at Datacamp. Hey, Greg.

G'. Day.

What do you have for us today, Greg?

Well, I'd like to start with a story. I was on the streetcar a couple of weeks ago, headed downtown here in Toronto, and sitting beside me, there was a woman wearing a hospital staff badge typing away on her laptop. I wasn't really paying attention, but I suddenly realized that she was typing up a psych evaluation for one of her patients. I could see the person's name, their address, and a couple of paragraphs of very personal information. And when I looked up, I realized that a couple of other people were reading over the doctor's shoulder as well.

Wow, that's awful. What did you do?

I nudged her and said, maybe you shouldn't be working on that here. She got pretty upset, slammed the lid of her laptop closed and told me that I shouldn't have been reading what was on her screen. Now, she was right. I shouldn't. But on the other hand, I don't think she should have been doing that work in a public place where passersby could pick up the most private things about her patients imaginable.

So what does this have to do with data science?

Well, there's been a lot of discussion recently about ethics in data science, about the ways that our work can be misused, either deliberately or unthinkingly, and about what our responsibilities are as data scientists to make sure bad things don't happen. Most of the discussion has been about the big stuff, but I'm starting to think that there are a lot of little things we can and should do to keep ourselves and our data safe, and that we ought to start with them.

Can you give me an example?

Sure. Violet Blue wrote a book a couple or three years ago called the Smart Girl's Guide to Privacy. Its aim is to teach teenage girls what they can do to keep their private lives private without going offline entirely. Simple things like using two factor authentication for their accounts, checking the permissions on their Facebook periodically to make sure the latest updates haven't put things out in the open that shouldn't be, and so on. The Electronic Frontier foundation also has some great materials on surveillance, self defense and PagerDuty have made their security training materials for staff available to the public.

This sounds like Hacking 101.

Well, not really. None of this stuff is aimed at programmers. It's all the equivalent of don't sneeze on people or use a condom, or clean out a scrape with peroxide. And I think that if data scientists use a little bit of data hygiene in their own lives, they'll be more likely to practice safe data at work. Cool.

So where should people start?

Violet Blue's book is a quick read and the EFF and pagerduty stuff is even shorter. I'd really like there to be a data camp course on this. You know, something like personal data safety for data scientists. But our platform is really aimed at people who want to write and run code. And, you know, if we can check or tweak the settings on your phone from datacamp.com, something's probably gone wrong somewhere. I think a webinar or two that walk people through the basics without fear mongering would be easy to set up. And if anyone listening is interested, I'm Greg@datacamp.com and I'd enjoy hearing from you.

Thanks very much, Greg. If anyone in the audience is interested in this, please get in touch. We'd love to hear from you. Thanks, Greg, and look forward to speaking with you again.

Thanks, Hugo.

Time to get straight back into our chat with Catherine. I'd love to hear your take on gdpr, and we've kind of moved around, but I'd love to know exactly what it is and what it means for civilians for users to start off with.

Yeah, so it means that you have a lot more rights than ever before. If you're a European resident, definitely. If you're another person, then at least perhaps some of these rights kind of essentially it's like trickle down economics of rights in that I hope that you have taken some time. Some of the cool things about GDPR that you may or may not know about is you have the right to delete your data. So you have the right to request that a company delete all of your data. For data science, of course, we're starting to think about this and be like, what does this mean? What does it mean for my models? What does it mean for my training sets and so forth. So this is definitely something to start thinking about and discussing with your team. How do we create processes that adequately delete or remove a user's data? There's another right to know how your data is used and how your data is processed, and also to opt out of that processing if you want. So this is again, something we need to think about as data scientists. How we build our pipelines, how we treat data, and how we allow people to opt in and out of probably certain tasks and jobs that we run on data sets over time. So you can think of this almost as like a nice flag in a database or as something that you store in a separate queryable database that allows you to say, okay, this person has opted in or out of processing. And then there's also write. One of my favorite ones is actually the write of data portability. And this is the ability to port your data from your current service, whatever it might be, to another service. And the idea is that the data has to be transmitted in a machine readable way. So this is also this idea that you have your data, perhaps for some app that you use, you would like to try to use a new and different app and you want to make a request to port your data to that different app. So this again, for data science means that you need to create outbound and inbound workflows or streams or something like this that allows people to transmit their data. And I think that this, the data portability is a real boon also to startups in general because it's this idea that, you know, it's kind of like phone number portability, right? It used to be that once you had a phone number and everybody knew it, you were stuck with your service provider until you really wanted to take the big jump and tell everybody you have a new phone number. I think with data, we've seen these entrenched leaders of data science and data collection, essentially, they've been there for now, decades, essentially, and they've had the advantage of the data that they sit upon. And with data portability, this will hopefully start to shake some things up, up and create some more competition. Because the idea that I can take my data with me and move it to another provider is pretty powerful, I think. And also something that I think is a long time coming.

Yeah, me too. And I think this is definitely a step in the right direction. I want to pick your brain in a minute about whether you think this is enough or what next steps would look like. But there's a term that both you and I and the GDPR and everything around it uses constantly. And the term is your data. Now, when I use Facebook or I use Twitter, I use whatever. What is it? What is mine? What do I own in that relationship and in that service?

Yeah, yeah. This is actually a subject of scholarly debate, I would say right now. And we're going to have to wait and see exactly how the regulators put this into effect. Now, I'm not a lawyer by any means, but from some of the law articles I've read around this, the intention of the working group that created that article was that it not simply be just the signup form. So their Working group notes specifically state that this should be any interactions and data that the user provides the company. And so we can think of this as like, well, maybe that's even goes down to your click stream of data. Maybe that even goes down to every post you have viewed. Probably it won't be enforced like that. But we need to think about how we, when we're collecting all of this extra data, when we're collecting like and tracking users, what does this mean in terms of the users that have said, no, please, I don't want to be a part of this and how can we respect things like do not track and how can we, you know, make very clear and evident what we are using data for and have the, have the user want to opt into that, hey, if you provide me this, not that I'm going to give you more targeted ads, but I'm going to be able to offer you this extra feature or something like this. So I think, you know, it makes us start thinking about data not just as something that we can, you know, use however we wish without really asking about it and ask for every single permission on the phone or track people across devices and all these things like that. Maybe we should ask first and maybe we should think what data we actually really need and provide a compelling product that means that people want us to use their data. Data.

And could this kind of force a bunch of companies to change business models in essence? Because I suppose the, you know, there's the old trope, if you're not paying for the product, you are the product, right? So you've literally got companies that are trying to take as much, much as possible because of just the value of, or let's say the assumed value of the data.

Yeah, this is this like also assumed value, right? So like, like one common thing I hear when I go to data conferences and I'm hanging out in the data science tracks or so forth is I hear people say like, oh yeah, just, just collect all the data and we'll just save it in case we need it. And some companies have been doing this for decades. They essentially have, you know, data from the early thousands and so forth on users still. And it's, you're sitting there wondering like, when are you going to actually use this data and how much of this data do you need? Now I think like for somebody that does ad retargeting or something like this, this is of course, this is the bread and butter. But for the average website or the average, you know, app, how much do you think that people would be willing to pay, to not be tracked, to not be targeted. And maybe you should start offering similar to some of the products that were launched last week and targeted free or an advertising experience. And I'm hoping that the consumer, consumer models also start to change around this. Of course, I have no idea what this will mean in the market in, you know, five years from now or ten years from now. Particularly because most of the offerings so far that have been targeting free or ad free are primarily targeted at EU residents and even sometimes not available to US residents.

So we've seen and heard what the GDPR will look like and what it will mean for civilians. What about on the other side of the equation, what does it mean for organizations and for working data scientists?

Yeah, so it means a lot more documentation and a lot more understanding and sharing of exactly how data is processed, how it's where it comes from. Right. So this idea of tracking data of provenance and what it is used for, and I think this is fantastic because I have been, I don't think alone, but kind of feeling like I'm sitting here screaming into the void about like documentation testing, like version control, like normal software practices for data science. And I think that this is finally, you know, the moment where clearing the technical debt, debt that a lot of data science teams have accumulated over time of not versioning their models or not having reproducible systems, not having deterministic trainings and so forth, that this will hopefully kind of like be a turning point where we can get rid of some of this technical debt. We can properly document systems that we're using, can have of course, everything under version control and automated testing. And all of this is going to benefit you because when you document all of this and you share it, you're essentially fulfilling quite a lot of your duties within gdpr, which is this ability a for people to opt out of that processing. So having a process that allows data to be marked as opt out and then also documenting exactly what processing is used, who are downstream consumers of that data, and where does the data originate from and under what consent was it given. I think that this covers quite a lot of what GDPR requirements are for data scientists. The only thing really that's left out is deletion of old data or anonymization of old data, which I think is going to spark hopefully a conversation around how do we expire data or how do we treat data that that is old or from previous consent contracts or was purchased and we're not sure exactly how it was collected and under what circumstances? And I think that this idea, if you're in doubt, if you don't know where the data comes from, if you've gone through and you've documented all your systems and nobody has any recollection where a particular data set or series of data comes from, then you should either delete it or you should go through methods to anonymize it, if it's personal data at all. And I think that this is essentially a spring cleaning for data science, both in terms of our processing and our data sets.

So I want to come back to this idea of data anonymization. First, I'd like to know what's been the general response to the GDPR from.

Organizations, of course, like, so I'm based in Germany and so of course the opinion here is that you know, this from a consumer standpoint and I think from the media standpoint has been very much that this is like a good step from the businesses here. I think there's, you know, that it has been costly both here and I think everywhere to enforce, to bring yourself within compliance before the due date. Now I must remind everyone that everybody had two years to prepare for this. So I mean, it was not a surprise that it was going into effect. But I think, yeah, I think for a lot of folks, unfortunately this has been costly. I'm hoping that the standards that have now been put in place were not a rush job and perhaps have created better processing that actually allows for this type of compliance in the long term. Right. And I think that there's also been a boon within Germany and Europe of startups thinking about these problems and starting to offer things. For example, like myself and Andreas with KI Protect starting to think about, like, what does GDPR mean in the long run? So in the long term, how do we guarantee better security and privacy and make this just a commonplace thing, not a compliance science thing?

And am I right in, in thinking that this doesn't only apply to data from people in the eu, but to data from users that is processed in the eu?

I don't know all of the specifics around this, but there's also, you know, it is, I can say a winner of GDPR is European Data Centers. And this is because there are, there's a provision within there that talks about moving data outside of the eu. So if data originates from the EU and you want to go process it outside of the eu, you need to explicitly tell people and they need to opt in saying that it's okay for their data to be processed outside of the EU from what I understand. And so there has definitely been a little bit of a pickup in the data, data center action here. And of course quite a lot of the large companies that process most of their data, let's say in AWS and so forth, this means that like finally I have some instances in AWS Frankfurt. It was always hard to get like the GPUs and other things available and now we're starting to see some parity which is nice. But yeah, I think this is something to think about is when we're moving data all around and we're moving it to different locations and in the cloud and so forth, these are real computers in real data centers somewhere. And this means that we also need to think about. Yeah. What implications that has A, within the security of our data, but also B, within compliance.

Yeah. And I think I was reading a number of tech companies that are processing data and have offices in Ireland for tax reasons. Among, among other things, they, they may be moving the data processing out in order to perhaps not have to comply with GDPR for the time being.

Yeah, that, that makes sense. And yeah, of course in Dublin there's like some very large offices for Apple and, and Amazon, as I understand it, and Google and, and so forth. And so that's usually like the, the EU based tech hub essentially for the large corporations. And yeah, I think this is probably changing the dynamics there and perhaps also changing the dynamics for a lot of the data processing that happens in Luxembourg as well.

And what happens if companies don't comply?

So it's, the fine is. So I think the process goes something like this. You're contacted, you're supposed to have the data protection officer that's essentially kind of the named person to handle any types of requests and compliance issues. I think that first you get some sort of warning and they ask you to become compliant. Right. You have some short period to respond to that and if not then you can get a fine of 20 million euros or 4% of global revenues. So it's not, it's not a small fine, it's meant to hurt. And for this reason a lot of people have been wondering like, well, will they go after small companies that, you know, small businesses where this might essentially bankrupt them. And this is of course we will wait and see how the regulators plan on enforcing this.

And this Data Protection officer or dpo, I think they're also responsible for if there are any data breaches. Breaches. Right. In informing the people affected and whatever the governing body is within even 72 hours or something. Like that and personally, not just via a press, like a press statement.

Yeah, yeah. So there needs to be information sent out to potentially any affected users as well as of course to the regulation authorities for any data breaches. And this can also, I believe that this also covers data processor breaches. So this is where it comes into effect where if, let's say you're reselling data or you're moving data to partners and your partner has a breach, then this is also your responsibility to essentially like they should inform you. Right. And then you need to inform the end users. And this is hopefully, you know, avoids some instances like Equifax and so forth. In terms of you can't just sit on the fact that there's a security breach for two, three months and sell your stocks or whatever you want to do and eventually like, oh yeah, yeah, you may, you may be a victim of identity theft or something like this.

We'll jump right back into our interview with Catherine after a short segment. It's now time for a segment called Data Science Pitfall. As we're talking about data security, data privacy and the GDPR today, I want to jump in and talk about the problems of data security and privacy inherent in the burgeoning space of machine learning APIs such as AT&T speech, Google prediction and IBM Watson to name a few. We need to talk about certain aspects of machine learning and the ways in which it can endanger data security and privacy. So let's say that you have a machine learning model that is exposed via a public API. Whoa, hold on there and let's explain those terms. Recall that a machine learning model can be trained on some training data to then make predictions on some new data. For example, we could give a machine learning model images of people labeled with their names. This would be the training data and would train the model to recognize people from unlabeled images with certain degrees of confidence. Now we are seeing more and more such models being exposed via public APIs. What this means is that although I don't have the original training data or all the details of the model, I can send an image to the model and get back the prediction. This seems fine, right? Well, it isn't because there are increasingly more and more sophisticated extraction attacks which with no prior knowledge of a machine learning model's parameters or training data, are able to duplicate the functionality of, or steal the model model and steal the training data. Now this is especially the case if the API exposes confidence scores even in a multi class setting. Now, stealing models and training data can be a huge problem, particularly when they're trained on sensitive training data, have commercial value or are used in security applications. We'll include some references in the show notes, including a paper called Stealing Machine Learning Models via Prediction APIs by a team of computer scientists at Cornell Tech, the Swiss Institute EPFL in Lausanne and the University of North Carolina. This is sure to be an area of data science and artificial intelligence to keep your eye on. After that interlude, it's time to jump back into our chat with Katherine Jarmy. So can you give me the rundown of what data privacy looks like currently? Just the current landscape of how everyone seems to think about about it.

Yeah. So currently data privacy, as we've been thinking about this at KI Protect, we've been of course investigating kind of where people are coming at this from a variety of markets and so forth. I think currently I see data privacy either as pay to play essentially. So it's often an add on that you can buy for large scale enterprise based systems where you say, ah, yeah, okay, I have all this other things and I'd like you to implement this privacy layer here. This is for a lot of the enterprise databases and so forth, something that they've been working on for some time, which is great. I think that that's fantastic that that's available. And other than that it's primarily focused, you know, on compliance only solutions. So this is this idea of like you have HIPAA or you have financial compliance regulations and so forth, forth. And these are focused around, okay, we are a database or we are a data processor that only focuses making sure that your hospital data or your bank data or something is treated in a compliant way. And this is mainly for these like data storage database solutions, which again is fantastic. But what does it mean if you actually want to use your own database and then you would like to use the data in a compliant way. And I think that there's been some interesting startups within the space that are trying to perhaps like allow you to use a special connector to your database that does something similar to differential privacy, not quite differential privacy because this is of course very difficult, but similar to differential privacy or that employs K anonymity or that employs something else like this. So there's a few companies in the space of essentially trying to be the query layer and then using your data sources below that and providing some sort of guarantees whether it be differential privacy for without necessarily like a long term privacy budget or whether it be K anonymity or whether it be pseudonymization, I can never say that either and so forth. So it's a lot of these like, you know, kind of extra add ons. And other than that, I think most of the privacy conversations has been really led by academia and Cynthia dwork's research on differential privacy and its implications also within machine learning, as well as some of the great research that Nicholas Papernot and some of the Google brain security researchers have been working on. These have been, I think, amazing contributions, but research. Right. Perhaps implemented at Google or at least with Cynthia dwork's work with Microsoft and so forth. But as far as available to, you know, data scientists at my own startup or something, this has really not been available in a, in a real way.

Right. And you've mentioned or we've discussed a variety of techniques such as anonymization, pseudonymization. Pseudonymization. There we go. Differential privacy. K anonymity. And we'll link to a bunch of references in the show, notes that people can check out with respect to the nuts and bolts of these. But my real question is, can we really anonymize data?

Yeah. This is of course like of much debate. Right. The gold standard is of course differential privacy. And the idea of differential privacy is that it's a fairly simple equation when you actually read it. And it's the idea that I would not know that you yourself as an individual were a part of any data set based on the queries or the data that I see from that data set, that there would be within a very small epsilon the ability to determine the probability that you are a part of that data set or not. And this is of course really elegant theory and I highly recommend reading dwork's work on this. But in terms of actually implementing it in the way that we use data science today, most of the ways that we guarantee differential privacy is using what is often referred to as a privacy budget. And this budget essentially tracks how much information. We can think of it almost as like information game theory, right? How much information about any individual was essentially gained by the other person via the query or via the data that they accessed. And once the privacy budget reaches a certain level, then we say, okay, then there can be no more queries that might reveal more information about this individual. And so this is difficult because a, in practice we often have changing data sets. So the data set that I can guarantee privacy on today and the data set I can guarantee privacy on tomorrow. This is like an ever changing right. We're like gathering more data as time goes by, we might have more information that we garner and connect about a particular Individual. And the more that we do this, of course, the less that we can guarantee privacy. And the second thing is that to keep the privacy budget, let's say indeterminately, this would mean that eventually our data would not be able to be utilized. Right. Because we would eventually hit the limits of our privacy budget. And unless that privacy budget is reset for some reason, then that person or that analyst or that data scientist cannot query any information that might be related to that individual. Right. And so what we see in differential privacy that's been implemented, for example, by the Apple Differential Privacy team or with some of the work that Google has been doing, this is normally a privacy budget within a limited time period. So resetting every day or resetting every few days or something like this.

When we think about anonymized data within any organization, and in particular from, you know, civilians who are, who are users of these products, I think one really important question is how do we know about how our data is being used as users? My question for you is how technical and how educated do civilians and users on the ground need to be to understand what's happening with their data? Data?

Yeah, I mean, this is, this is interesting and something that Andreas and I have been thinking about, kind of like doing a series of articles and so forth that kind of explain how privacy works and how de anonymization really works at a large scale. Because I think the average data scientist, you know, they've heard about the Netflix Prize, they know about the New York City taxi data in the sense that with an informed adversary or an adversary with access to potentially some open data, this is quite easy to de anonymize when we're dealing with large scale data sets. But I mean, if I were to ask my mom, let's say, or my sister, hey, do you know, like, if you upload that extra thing to Facebook and then any of your Facebook data leaks, do you know that the ability for somebody to de anonymize you is, you know, essentially like guaranteed? I think, you know, maybe not. I don't think we're there in terms of the public conversation, but I do think that breaches like, I forget the name, but the running application that recently had a data. They released an open data set, Strava, I think it was called. And their open data set essentially leaked information about so called Private U.S. military Bases or Secret U.S. military Bases across the world.

The fitness tracker which most.

Yeah, yeah.

People using it for American citizens. And then you could see in key locations in the Middle east and in certain African Nations.

Yeah, yeah, yeah. It's like you could see the military.

Compounds on the map.

Yeah. Because it would be, you know. Yeah. And this is what happens when we aggregate data and this is especially a danger for people that are like releasing public data. Right. Or you can even think of it if you're selling data or sending it to a partner or something that when we aggregate this data and even if we say we so called anonymized it, then data in aggregate can also release secrets. So this may not be secrets about an individual anymore, but this may be some sort of secret about the group that uses your application.

So it saddens me to say this, but we're coming to, to the end of this conversation and something I mentioned at the start is that gdpr, I think as you stated as well, is very necessary and timely. My question for you is, is it enough? And, or what do we need to be doing? Or what would you like to see in the future to make further steps in this direction?

Yeah. So GDPR by no means guarantees anonymization. And, and this I think might be something that we should really push for within the data science and machine learning community is how can we solve this very difficult problem or how can we at least make some inroads to this problem so that when there's a security breach or when there's some issue, or when somebody gets their laptop stolen and oops, they had a bunch of customer data or other sensitive data on it, when these things happen, maybe we can stop them at the source. Right. Maybe we don't necessarily need to always use complete personal data to build a model. Maybe we can start thinking about how to privatize our data in a way before we start the data science process. And again, this is definitely something we're thinking about and working on at KI protect. But this is something that I really hope overall as a field we can push forward and it has some interesting implications as well with ethics. So there's a great paper again, primary author was Cynthia Dwork comparing this idea of differential privacy to also the same basis of ethics. In a sense that if you do not know my race or if you do not know my gender or my age or something like this, you have the potential to build a fairer model. Right. And so I think that these have interesting overlaps and implications for our industry and I really hope that we start to think about them as a wholesale solution, not just as a. Oh, compliance only means I have to do this much. So I, this is what I'm hopeful for. And something that I look forward to seeing more in research and also chatting more with my peers and so forth.

Yeah, I like that because it sounds like, like in a certain way, mindful data science in the sense that you just don't take all the data you have and throw a model at it and see what comes out. Right. Yeah.

You think about the implications of any other data that you share that you expose both to your team internally and to anyone externally that you think about. Essentially, would I want somebody to do this with my data?

Data.

So the golden rule of data science.

Yeah.

Great. So do you have a final call to action for all our listeners out there?

Yeah, sure. A, you can check out all of the work that we're working on and we're looking for feedback still with KI Protect. So if you want to reach out, we're@kiprotect.com and then also just if you're working within the space, if you're thinking about these problems, keep, keep at it. You're not alone. And also feel free to reach out that I think that we need to kind of create a really vocal community within data science that these are important, these are essential and that this is not only for researchers. Although I'm really a big fan of what the research community has been doing, but this is also something that practitioners care about and that we want to be able to implement what we're seeing in research and the advances that we're seeing in terms of potentially guaranteeing privacy preserving machine learning. We want to see this like within the greater community and within the tools and open source projects that we love and use.

Katherine, it's been such a pleasure having you on the show.

Thanks so much, Hugo. I really appreciate it.

Thanks for joining our conversation with Katherine Catherine about data security, data privacy and the gdpr. We saw that the GDPR gives consumers more rights and knowledge as to how their data is used, such as rights to delete your data, to know how your data is used and processed, and to port your data to other services. We also saw that for organizations and working data scientists, although there is significant work to be done, it will result in paying off a whole lot of already incurred technical debt in terms of developing better documentation and building better systems for tracking and managing data. With a focus on user centered design, we also saw once again the growing importance of data scientists thinking about the implications of their work and building ethical models and the importance of both privacy and consent by design. Do we really need to incorporate a user's zip code into a model? Do we really need their last thousand purchases? And why do I need access to their personally identifiable information? Does anyone really, outside of perhaps customer service us make sure to check out our next episode, a conversation with Jonathan Nolis about organizing data science teams and the do's and don'ts of managing them with a dive into best practices for hiring data scientists. Jonathan is a data science leader in the Seattle area with over a decade of experience. He is currently running a consulting Firm helping Fortune 500 companies with data science, machine learning and artificial intelligence. We also tackle questions such as what is a more important skill for a data scientist? The ability to use the most sophisticated deep learning models or being able to make good PowerPoint slides. The answer may surprise you, but then again, it may not. There's only one way to find out. I'm your host Hugo Bown Anderson. You can follow me on Twitter Hugo Bown and datacamp at datacamp. You can find all our episodes and show notes@datacamp.com community podcast Sam.