Data management, regulation, the future of AI

Bandwidth for Changelog is provided by Fastly. Learn more@fastly.com we move fast and fix things here at Changelog because of Rollbar. Check them out@rollbar.com and we're hosted on Linode servers. Head to linode.com changelog this episode of Practical AI is brought to you by Hired. One thing people hate doing is searching for a new job. It's so painful to search through open positions on every job board under the sun. The process to find a new job is such a mess. If only there was an eas. Well, I'm here to tell you there is. Our friends at Hired have made it so the companies send you offers with salary, benefits and even equity up front. All you have to do is answer a few questions to showcase who you are and what type of job you're looking for. They work with more than 6,000 companies, from startups to large publicly traded companies in 14 major tech hubs in North America and Europe. You get to see all of your interview requests. You can accept, reject, or make changes to their offer even before you talk with anyone. And it's totally free. This isn't going to cost you anything. It's not like you have to go there and spend money. Get this opportunity. And if you get a job through hire, they're even going to give you a bonus. Normally it's 300, but because you're a listener of practically I, it's $600 instead. Even if you're not looking for a job, you can refer a friend and Hire will send you a check for $1,337 when they accept the job. As you can see, Hire makes it too easy. Get started@hire.com PracticalAI.

Foreign.

Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show@changelog.com community. Follow us on Twitter. We're @practical aifm. And now onto the show.

So Chris, are you terrified of the gdpr?

I am loving the GDPR from my standpoint, trying to learn more about it. But I think though it might be imperfect, it's about time we finally start addressing this in a public manner.

Sounds great. Well, I brought Matt and Andrew from Immuta onto our show today. Welcome Matt and Andrew.

Thanks so much.

Thanks for having us.

Welcome. Thanks for joining.

Yeah, so I met Matt and Andrew back, I think in the spring of of 2017. We ran into each other at a bunch of different conferences. And I realized that these guys have pretty much all knowledge around, you know, AI and regulation and data and privacy. And I was just learning a ton from them. So I think it's great to have them here to discuss some things around AI and how it should be regulated, how it is being regulated, what are, what the trends are there.

So I have so many questions for them.

I know, I know. This is, this is going to be. This is going to be great. So just to start out, Matt, why don't you give us a little bit of a personal intro?

Yeah, sure. So, you know, by trade chemist, went into the US Government, deployed to Iraq, Afghanistan, fell in love with technology, enabling that mission. And so leaving the government, went and started a services company, sold the services company, and then eventually got drawn right back into the government around the problem of how do we make the law and data science work together so where we can solve problems at the speed of the business but still maintain ethical and legal controls around our data. And so that kind of led in 2015 to the creation of Bemuda. And so to date, I'm the CEO of the company and we're tiny, going on 37 people, people, but are growing really fast. And it's a great market and just excited to be part of it.

Yeah, I imagine that all the hype around GDPR and other things is definitely not hurting your business.

No, certainly not quite the opposite. But I think the question really, and it would be great to talk about today, what does it actually mean? I think that's really, from a practical perspective, I think sometimes we get a little ahead of ourselves.

Yep. Awesome. And Andrew, I believe, is a lawyer, is that correct?

Yes, guilty as charged.

Well, tell us a little bit about how you, how you fit into this story.

Yeah, so my title at Immuta is Chief Privacy Officer and Legal Engineer. And the legal engineer part, I think, is particularly relevant for today's discussion. And basically my charge at Immuta is to think about how law and data science overlap and to think about what types of requirements law is placing on data science activities that we can embed within the platform. And so my background is I've been involved at Yale Information Society Project for a while. I'm a lawyer. I spent some time working for the FBI Cyber Division, really kind of understanding this conflict between compliance demands and, and the legal burden associated with data and all of the really new, really important projects that are going on in the data science community. And so the goal is to figure out how can we think about legal requirements and risk management in a new way.

Awesome. Well, it sounds perfect for this discussion. Are you the only law engineer in the world or there are more of those people?

Legal engineer, is it?

Law engineer.

Legal engineer.

Legal engineer.

So that's a hard question to answer. I don't know any others, but that doesn't mean they don't exist. And there's been this kind of like, funky history around even the term legal engineering. It actually was coined, I think, in the late 1980s by some folks at Stanford. So people have been thinking about what it would mean to like, embed laws within technology for a really long time. So instead, you know, your lawyer, instead of writing a memo, you write code. And people have been thinking about that. But I think really in the last few years with the intersection between data science and regulation, I think that we've seen a real need for this field of legal engineering. But I can neither confirm nor deny that I'm the only legal engineer.

All right, well, thanks for those intros. One of you guys, maybe Matt, tell us a little bit about. You mentioned Immuta. What is Immuta? What are you guys trying to accomplish? What does Immuta do?

Yeah. So long story short is Immuta is a data management platform for data science. And the creation of Immuta was really at the intersection of we have these three users that make up data science operations. We have data owners and they control the data. You get data scientists who want to use the data to do something with it, to provide insight. And then we've got these governance and legal teams that need to oversee that process. And the problem is that what we found, all of these three parties are kind of acting as antagonists to one another. It's a very human process. Lots of meetings, very slow. And so there wasn't a way to really provide each one of them with a single digital platform where they can all work symbiotically even if they don't know it. And so we had to create a muta because what we felt was data management for application development is very different than data management for data science operations. And there was a massive gap. So other than the few companies in the world like Google or maybe even from the government side nsa, where you have thousands and thousands of people that can write code, that can request ad hoc access to all these distributed databases, right? There's all these silos of data and they can just write ad hoc code to that to get access to it. Very few organizations in the world can do that. And the most successful can, but then there's all these others and they don't have people that can write code. So they built applications and based on predicting the types of questions their analysts would want to ask of the data, they'd throw BI tools up, they would throw their own custom apps on top. And so their version of data science was much more complicated as they brought data scientists in who need to ask ad hoc questions across all these silos. They're managing their keys separately, so they've got hundreds of keys to get access to these databases. They're making lots of copies of the data and bringing it local. So they're potentially breaking rules by doing that. And so it was very cumbersome and slow. So Amita, what we did was we built a think of us as a data control plane where we can connect to any store data in any storage system. We can then virtually expose that data in a read only fashion to BI tools. Data science platforms, ides through generic access patterns versus custom APIs so like File System, SQL, Spark, Hadoop, and connect to any of this disparate data through a single connection. And then finally what we did was, and I think the most valuable thing that Immuta provides right now is we then built a interface for lawyers to be able to, to implement rules on this data and dynamically enforce those rules as people ask the questions. And so what that allows us to do is data owners can virtually expose their data in a catalog. Data scientists can bring any tool to bear, connect it to that data, and governance teams and general counsel can actually implement the law as it changes on that data without impacting any of the other parties. And so the idea is that we can actually streamline the process by which people can get access to data so they can connect to it, they can control the data depending on the current state of law or where they are, what region they're in. But then we can also prove compliance. And some of that isn't just your user rights, but it's also an end. I'm sure Andrew will talk about this later, but why, for instance, with the Facebook data leaks or scandal, if you will, it's not just that people use the data. They legally could use the data, but it was around. But why are you using it and is it for nefarious purposes? So this concept of compliance and understanding, who's using the data, what data is it and why? And that's Immuta in a nutshell.

That's pretty fascinating.

Yeah.

So you guys are having thought about this for the last few years, are probably way ahead of us in terms of thinking about kind of where regulation around AI and data is at this point and kind of what the lay of the landscape is. Could you talk a little bit in the context, especially with GDPR finally being a reality for everyone and where in general regulation is going, and maybe even a bit about what GDPR is kind of getting right or wrong or could fix, give us a lay of the landscape and maybe what we should see in the next months and years.

Yeah, and we should probably say that the GDPR is. These guys are going to correct me if I'm wrong. The General Data Protection Regulation. Is that the right thing of the acronym?

Bingo.

Yes.

Awesome. Yeah. So it's an EU thing, right?

Yes, it applies too. So I'm happy to take this one. The GDPR basically applies to any EU data that could be considered personally identifying. And so the standard in practices was the data generated in the eu. And could you use it to figure out anything about a human being? You know, could I identify your name, could I identify where you live? IP addresses are widely considered to be identifying under the law. So if you're a data scientist and you're working with data that comes from the Euro, the answer is that in practice this applies to you. And let me get to. So I'm happy to talk about the AI issue. Those are really big questions, so jump in or follow up. But I think in a nutshell, every government in the world is realizing that the power of AI is new, it's a big deal, and they're talking about what to do about it. And so, on the one hand, governments like France recently and the UK are saying we need to have an explicit strategy to promote this new technology. And then on the other hand, regulators like the ones that enforce the GDPR are saying we need to control this kind of like a not so fast. And so the GDPR is really the first, I think, major regulation that's been implemented that is explicitly, or has parts of it that are explicitly focused on AI. And so in general, the way it's going to impact, really like machine learning models is that there are different types of requirements for explainability. You might have heard to it as a right to explainability, which in my own opinion I think is a little bit too much. But the basic idea is that when you're using models that are deployed autonomously or that might be inherently opaque, or as some call them, you know, black box models, the people who are subject to these decisions, whose data is being processed by these models, have basic rights and so those rights are, they should be able to understand how and why their data is being used by the model. They should be able to opt out or to say, I don't want a model to be making a decision, let's say, for a credit score. Instead I want human review. So those are the types of requirements that the GDPR puts into place.

So you alluded a moment ago about, you know, you thought that in that one instance they had gone a little bit too far or too much. And I'd really love your personal opinion on, you know, what do you think GDPR has gotten right where it could stand a little improvement and maybe even speculate a little bit about where you think regulation outside the EU or future versions of the EU might go. How will the US respond? And just, you know, what are some of your thoughts there?

Yeah, so the way, so the Europeans and folks like us in the US we really approach regulating technology very differently. In the US we tend to want specific regulations focused on specific problems. That's why we don't have one national regulation that just covers all data. And in the eu, they very much like the opposite approach, which is they'll call it a principles based approach to regulation. And so they want overarching rules. And the downside is that those rules come at the cost of very, very steep ambiguity and vagueness. And so I think there's a lot of good stuff in the gdpr and I think the intent is wonderful. I think the intent of trying to mandate certain levels of fairness and automated decision making, that's wonderful. But when the rubber meets the road, I think it's going to be very hard for a lot of data science and programs that are heavily investing in machine learning. It's going to be very hard. There's going to be a lot of fine tuning as to what some of these specific provisions actually mean. How specific does the explanation of the model actually have to be? And in particular, when is it not specific enough, when is it vague enough, when is an explanation not detailed enough that someone should actually be penalized? So those are really hard questions. Legal departments everywhere are focusing on them. And we're going to kind of, we're going to start to see how that regulation is enforced and I think we'll start to learn from it. I would say that I think one of the strongest points, and this is something we've been really focused on at Immuta, one of the strongest points of the GDPR in some senses. I think the genius of it is that it bets really heavily on Purpose based restrictions on data. Most regulations around data are focused on data at collection time. And a lot of that means that the emphasis is on consent. So I consent at collection time to give you, you know, X, Y and Z of my data. And that's the traditional approach to data. What the GDPR recognizes is that in addition to that, you need to be able to understand and restrict how that data is going to be used as it's being generated. And so that's a really new type of concept. And frankly, I think we generate so much data it is frankly impossible for us as consumers and individuals to understand what types of insights we're giving up as we generate this data, which is, you know, the beauty of data science. And so placing restrictions on that data's use as it's being generated I think is really like, I think that is going to be the future of regulation for data.

Yeah, that's super interesting. And I guess it kind of leads me into my next question. I was really trying, like while you were talking, I was thinking in my own mind, like, holy crap, how do I make my models explainable? And I don't know if you've had the same thought. It's a realization I have every once in a while. Chris. I don't know if it's come to you yet as well, but I kind of freak out a little bit when I think about that because I remember in my first position I wrote all of these PowerPoint, well, it was Google Slides, I guess, presentations on how my models were working and trying to explain it to just my own team. And it was incredibly difficult. But it sounds like what you're, at least partially what you're saying. In the near term, a lot of the focus is going to be on how your data was kind of processed through the pipeline. To what end? Not necessarily explaining a deep neural network to some random person. Am I getting the right sense there or is that wrong?

Well, sadly, the answer to both of those questions is is yes. And it's yes in the sense that the GDPR is.

You're not making me sleep any better.

Yeah, no, I'm sorry. Not to be, not to bring easy news. The GDPR has a huge compliance burden. There's no kind of sidestepping that data being used for any purpose like that needs to be documented. You're not going to be able to use EU data at scale in a data science shop without a plan for how you got the data, the legal basis for that data, what you're going to do for it at the Same time, there are also requirements on the types of models you use. I should say there are explainability requirements surrounding those models. So it's not that you're going to have to be able to explain the weighting on every single neuron in a neural net, but you are going to have to be able to say here is in general how the model is working, here's where it's getting data from, here are the reasons why it's being used. So it's not the type of explainability I think that might have you waking up in the middle of the night in a cold sweat. But still you're going to have to be able to provide very basic information about the models that you're using.

Yeah, I think the reason that you are concerned is that there's really no frameworks to automate this. So there's this now massive legal burden on the data scientist, who is typically not a lawyer, to expound on why they made certain decisions, what data was used, who potentially was in that data, for what purpose. And so it's not just about data provenance any longer, but it's also what types of activities were taken to ensure that there was proper ethical curation of the data itself before the model is being trained. And then it's what are the guardrails put in place to ensure that you're controlling the model as it's put into production. And these are things that historically we've had guardrails in place through software, through sas, they've always had these guardrails in place and other types of software kind of automated a lot of these controls. Now the problem is that the ability for anyone to be a data scientist and use the data and use open source tools, they just don't carry the rigor that's required. And I think that it's not just the GDPR, but we're seeing now even in the U.S. california is looking at enacting legislation around data privacy. This is something people care about because what they're afraid of is the open source community. And large organizations like Google and Microsoft and Apple are making it easier and easier for anyone to be able to design these models because they're incentivized to buy up small little AI companies. We need to put rigor into these, into guardrails around these pieces of software. And so it's tough and it's hard and people aren't educated around the law. So that's what makes it even scarier.

Yeah, and I think that like one of the things I'm Hearing from you, Matt, and also in your kind of, when you were talking about what you're trying to accomplish with Immuta, I mean this, this spans a lot of different areas, right? All the way from data curation to training of models, to deploy deploying of models, to building APIs in which models are deployed and with which they interact. There's a lot of different kind of teams involved in this whole process, right?

Yeah. And the challenge is that they're traditionally, if you look at most of these organizations, because mostly Global 2000 organizations that are really trying to accomplish this at scale, it's these teams aren't sitting next to each other. They don't have the benefit of through osmosis to coordinate effects. Right. A lot of them are operating on five to 10 projects because there's not enough data scientists, there's definitely not enough council that's caught up in the space. So they're trying to get smart on how does each project function, what are they trying to build and why. So it's all very slow and cumbersome. The worst part about it all is I think a worst case scenario is you build the most fantastic algorithm that can really change the way a legacy business is operating. That's the whole point of this. We can do things better and faster and then the whole thing has to be brought down because they have no insight into how it's working. The whole business is then crippled by this so that they don't want to make the investment in these advanced technologies technologies because they don't have the legal understanding or the internal engineering to be able to do this in an ethical manner. And that's the fear right now. And that's what the GDPR is for, I think, is I don't think they want to come around and, you know, destroy business. But I do think what they want to say is, listen, there's an ethical way and there's just like any engineering process, there's got to be a governance end of it as well. So, you know, hey, Global 2000, if you're going to use EU data, this is how you're going to do it.

Well, I'll tell you what, you guys have really had an impact on me and this podcast already. I started off the show saying, hey, I'm excited about gdpr and I think part of that is as just a consumer out there and using many services, I am conscious of privacy. But when you, as you've educated us on just how, what the burden really is that we're going to all be thinking about and the lack of tools to do that. A little bit worried. I was wondering, would you mind delving into some of the maybe specific industries where you see impact, maybe healthcare or transportation or others where you think that they're going to have very specific problems to contend with.

Yeah. So happily. Matt, do you want to. Is there any part of that question you want to take?

Go ahead and go ahead.

Okay. So I think, I mean, so to start with, I think one of the most fascinating things about being a data scientist is that the work that you do can apply to so many different industries and the same techniques can be incredibly powerful across healthcare or finance. So I think GDPR in general, I think is meant to be as broad and as far reaching as possible. So if you are an organization that really cares about data and is using data, the GDPR applies directly to you. I think what we are seeing from the global 2000 perspective is I think healthcare and life sciences, finance, transportation in particular, I think all of those groups and then obviously advertising and marketing organizations, I think those groups are the first to really get kind of get hit by the GDPR or to take it seriously. But I mean, I think the bottom line with the gdpr, GDPR is unlike in the US where we try to do sector specific regulation, where we have an FDA and the FDA only regulates specific drugs and products, this is really meant to touch everyone.

Yeah, to expound on that. I think the point of the gdpr, again, it's not designed to be punitive to business generally. Right. I mean we want the global economy to scale and it's all in one and it's in our best interest for that to occur. I think that the focus though is they wanted to put teeth around the regulation, hence the 4% of your annual revenue or 20 million euros, whichever is larger. They want to show people. Listen, there is a massive slippery slope when we start to use people's data at scale. I think that there's a couple core pieces to this. One is we've never had so many people in the world world that are producing data than ever before. It's only going to increase as we as Internet proliferates and 5G proliferates throughout the world. And so I think the goal here is this is a time in humanity where we can say, listen, we've got to put controls around this because every industry is going to be impacted. Everyone wants to build algorithms, everyone wants to operate faster. And humans yearn for instantaneous gratification of all their apps. I mean voice is the new cool thing. Right. Like very few people are now wanting to type in their search. They just use their assistant on their phone. Right. And so there's all these personal items that are being introduced to physical devices and there's algorithms behind them and no one really knows how or why it's being used. And I think that's like, generally the concern is that's every industry, it's every business in the world, small mid to large businesses. But I think the Global 2000 are going to be impacted the most because, well, let's just be candid, they have the most to lose and the data scientists in those organizations are now carrying the largest amount of risk.

Yeah, no, that makes total sense. I think I definitely, as Chris has mentioned, I've appreciated the candor and the insight that, that we're getting kind of in, in the trends that you guys have been, been following and examining. I think as a data scientist or AI ML engineer or whatever I, whatever I am, what I'm thinking is, you know, as, as me or as Chris who's, who's a chief scientist or as a software developer of, of AI apps, what are kind of some practicalities as far as, okay, we get that this is a big deal, give us some good news, what can we do? What are some kind of like initial practical steps that would help us kind of move in the right direction of being responsible in how we deal with people's data even if we're not in the eu. What are kind of some first steps that we can take?

Well, Andrew, go ahead. I think, go ahead and I think run through the big steps here is probably best.

Yeah, so happily so I think one of the biggest takeaways is that what makes for good governance actually makes for good data science. So basically good governance I think translates into good data science. And the reason is.

That's great to hear.

Yeah, exactly. Yeah. So it should be, I think, reassuring in the long run. I think a good way to think about the GDPR is a little bit of a paradigm shift, which is right now a lot of data scientists end up operating in a little bit of a vacuum where they say kind of here's a project, give me the data and then I'll get back to you and I'll tell you know, I'll play around with it and I'll tell you what we can do and that is good and kind of one off interactions, it just does not scale. And so the ways to, you've, you've.

Summarized well so many years of my life in which me too, I used To. I used to frequently refer to myself as the data monkey rather than a data scientist because I felt like that was more appropriate because I. Because of the way I operated.

Yeah, because you're just kind of bouncing around and climbing things and, you know, the view looks good from here.

Yeah.

Climb a different.

Yeah, no, no flinging of poo. So that was good.

But.

But in other ways.

Yes, I was going to. I was going to ask, but thank you for clarifying. So I guess the really big takeaway is that if you're going to do data science at scale and you're going to run a program, you need to really have a sense of what data is available. How do I get access to that data? As a data scientist, you need to have some way of documenting what it is you want to do with it and what you've done with it. If someone else gets added to the project or if you leave. These are just like kind of basic organizational measures to make sure that you can collaborate and that the documentation is there. And so if you split that up, what that really looks like is as soon as a project begins understanding kind of key objectives that you want to achieve, key objectives that you want to avoid. Another way of stating that is legal liability. And so that's where lawyers can come in and say, make sure you don't do this, or if you do this, make sure you're going to mitigate it in some way. So that ends up being really important. And then documenting it so people can be added and subtracted from that project. And then there's very specific ways that you can examine the data you're going to use and control. And Matt used the word guardrails and so kind of setting up some guardrails to think about potential biases embedded in the data, things you can do to try to compensate for it. And as we all know, you know, all data sets are biased. So the question is really just trying to prioritize what, what it is that you're trying to avoid. So I could go on here for a while. I think I mentioned to both of you before we started recording that Immuta is going to be releasing a white paper with an organization called the Future of Privacy Forum. And it's literally going to be a white paper designed to be a practical guide for managing risk in deploying machine learning models. And so it's meant to speak to both data scientists.

That's awesome.

And lawyers, and to give some real kind of depth to some of these suggestions.

And we'll definitely include that in the show notes, so that as it becomes available, listeners are able to find it.

Easily to deep dive into there. I think, as Andrew said, good governance, you know, leads to good data science and they kind of go hand in hand. I think, I think the first thing is this is one of the things I learned in the government is lawyers are actually there to help you, they're not there to slow you down. And I think in a lot of these, especially if you're a data scientist that's working in a big organization, starting to align yourself into the governance organization and asking for support early and often is really key. Understanding that most large organizations have gone through some sort of semantic context around their data. Right. They know what metadata they have and generally where it is and what are the rules around that and starting to understand risk levels. Right. And so bring those people in and working in your programs, it seems so simple, but yet when we talk to clients, this is not occurring. Right. It's on a one off kind of relationship where every so often they ask.

Yeah, I've never seen that.

Yeah. And it's unfortunate because, you know, the thing is that I learned in the government is they will very, very easily relieve a lot of risk off of you. Right. And then you can start looking at it differently. Right. It's the risk isn't on you. You've now brought in others to where you've gone to counsel, you've gotten review and you can start looking at that. It doesn't need to be technical. Right. A lot of times it's like, hey, I'm using this data, I'm using this data, I'm using this data and I'm bringing it together. You know, where do you think my potential risk is here? Are there any regulations where, you know, maybe it's not PI, but this is considered sensitive data or not, should we be doing masking on this data or not? You know, and sometimes just the general rules change when you start bringing data together. Right. And mashing it up. And so I think early and often bringing governance in and having a good working relationship, potentially even creating a working group to where you review different types of projects and potential risks as part of your development cycle.

That's a fantastic idea.

Yeah. Then I think once you get to that point, then the second thing is it really comes down to what kind of data are you using. I think a lot of times we just tend to copy all data and then figure out what data we want to use. We figure out how we want to analyze a problem and then look at a bunch of data sets and bring it in. And I think at times we could do a much better job data engineering in the sense of planning it out and looking at it in a generalistic pattern. Meaning if I'm going to build an algorithm to look at X rays, do I really need all images with all the organs in there if I'm just looking at bones? Is there any value of trying to automate a boxer fracture for any of you that had teenager angst? A boxer fracture is when you punch the wall. It's almost. Or you punch someone's face. Hopefully you just punch the wall because you're angry, right? At your parents or something. And it's a very easy break, you know, and I've been on the wrong end of the wall a couple times. And so, you know, we won't go.

Into that story that's right here.

But the point is incredibly easy for an algorithm to look at your hand and look at the X ray and see, yep, that bone is fractured in such an angle. That is a boxer fracture. And here's the prognosis, and here's what we can do for you. Right? Very easy. But you probably don't need to see all the tissue. You don't need to see all these other things in there where potentially there's like, oh, well, there's calcification, there's this, there's that. Well, why do you need that information? Does my insurance company get that information or not? So these are the leading issues that we see where there's derived information out of the model itself that could potentially be misused against you. And so only provide the data that's really necessary. Like, really plan this out and think it out. Use kind of mind mapping, right? Like, I only need bone to this. Why do I have any other data in there? Why don't I remove out tissue out of the image itself? How do I isolate the bone? So, like, those are the types of things that I don't think people really think through when they're going to their models, because I think they have a. Which is great. They're super positive about why they're building something. And it's just like they're looking at the upside, right? Which is, hey, I'm going to be able to solve this problem for radiologists, and they can focus on really complex issues, not realizing that there's a potential nefarious use of the derived results of that model itself. So how do you keep that out? And then building that exercise into data engineering is just as critical as the data Cleansing and the data preparation that goes into the model and feature building itself.

Yeah, that's super interesting. I don't know, maybe you guys have seen this trend too, but I think we've kind of gotten into this trend of kind of pawning off a lot of the intuition around the features in our models into kind of deeper, more complex models that kind of figure it out on their own. Right. And we never really go back and say, well, these features or this data that we put in isn't really necessary. Why are we using it? It's both causing us potential compliance issues, but also it's just making things harder because it's more data and all of that. So yeah, I don't know. That's definitely one trend trend I've seen.

Yeah, more data isn't better. That's why I've always kind of despised about big data is just because you have a lot of data doesn't mean you should use it. Right. The point is, and I think this is just generally good data science, actually just generally good science, right. Is you're usually on a mission to solve or answer a question or solve a problem. Right. And then work backwards. What do you need for that? But I think just throwing more and more data at a model for it to figure out intra model potential patterns and useful features out of the data itself isn't necessarily useful holistically to the consumer. It's useful to you, not necessarily the consumer. And I think that's the challenge is that we have to take into account who's in the data just as much as the problem we're trying to solve.

So it almost sounds like that maybe in if you look at the, the AI space versus more traditional data science like you know, just analytics or ETL or visualization, as you talk about feature engineering and the fact that, you know, more data isn't always the right way to go, is that a particular concern that I guess we're going to see in the AI space going forward is given the fact that we're used to throwing so much data at our models and letting, letting the neural network figure out which features matter. Is that something that AI practitioners need to be particularly concerned about, do you think? Or is it really just the same level as the others?

No, I'll let Andrew chime in on this. But we have to stop being lazy. Lazy is bad. We're engineers for a reason. There's a process and laziness leads to bad action and there are bad actors out there. And as we move to a world which I think is fantastic where machines can provide the necessary intelligence to augment human decision making. It's on us collectively to ensure that we hold ourselves to the highest standards. And just randomly throwing a bunch of data in there. Just because we are able to collect it, process it and make decisions on it doesn't mean it's the ethically right decision. And I think what I'm concerned about is, and maybe this is just a broader theme is people are really afraid of the Amazons of the world, right? I mean the inertia behind an organization like that. And I think people are willing to take shortcuts to try to catch up through augmenting business operations through machine intelligence. So whether that's AI or it's just like a simple linear regression, right. They're trying to automate as much as they can because they feel like that's the only way they can compete against an Amazon. And so I think that's going to lower the barrier of entry to deploy into production. And that's my biggest fear. Honestly. I'm not worried about Skynet. I'm really not. I'm more worried about we're going to start making bad decisions without understanding the potential repercussions to the direct consumer. Not the air is going to take over, but rather our AI is incorrect and all of a sudden we're not providing mortgages to a subset of Detroit or insurance premiums are going up to for the 30 to 60,000 per year socioeconomic sector is that we tend to based on the data we have, isolate and polarize. And that's based on bad data governance in my opinion.

So you might summarize it as bias versus Skynet.

Yeah, I mean I think inadvertent bias. Right. There's always bias in our data. I don't think that's ever going to. I mean otherwise there's no statistical relevance. Right. But I think bias for the wrong reasons, without us knowing it is potentially increased based on the more data we throw at because it's not possible for humans to run through all of the data. And I just don't think that algorithms aren't good at looking at risk where humans are. And so we need a good way to quantify the risk based on the type of model we're using to the type of policies and the existence of regulations and law on that data and the potential negative outcome of the algorithm itself. And we need to merge that together to really think about quantifying risk in a different way. The days of parameterization around data to quantify risk are over. The data is too big and vast and complicated. It's really outcome based. Decision making is really our future in the AI space. It all starts with good governance and understanding what data is going in and why we're using it. And based on that, choosing the right models to attack the problem.

Yeah, and let me just add in, I think what Matt said is 100% correct. And one of the reasons why I think more data is not just kind of blanket, always better, is I am worried about both bias and any potential failures. And I think what we're looking at is a world in, let's say two to four, five years, something like that midterm future where no one fully understands where all the models that are deployed have gotten their data from. And so if there's a problem, it's going to be really hard to identify exactly why that problem occurred. And so I think the type of like tech debt that we're looking at when we're in a world where machine learning is something we're really relying on, it's going to change the paradigm. It's going to mean that we need to do a lot of this governance and, and risk management upfront. Otherwise we simply won't be able to fully understand failures when they start to emerge.

Yeah, this is great. I don't know, I've thought for a long time, I mean, software engineering and software engineers have had a long time to kind of come up with their standards and process around responsible software engineering and we haven't really done that with AI and data science. And I think a lot of what you guys are saying is, is super relevant to that discussion and are great takeaways. I mean, be ethical, don't be lazy with your models and your data, talk to your lawyers early, write docs, which everyone should be doing anyway, but docs with regards to explainability and think about and quantify fairness on the outcomes and policies that you're trying to enforce in your outcomes. I think all of these are just super, super helpful. I know we only have a little bit of time here to discuss all these things. I think you guys have done a great job at giving us kind of a crash course into regulation around AI and practical steps we can take. Where can listeners follow up? You mentioned that there's this white paper that you guys are coming out with. I'm super excited to read that. You mentioned some institute that was associated with the, with that as well. What are some other places if our listeners who are out there in the trenches developing models and developing software where can they start out there to try to get some more practical info about this that they'll be able to consume and hopefully bring back to their teams and their discussions?

Yeah, so there's not, sadly, there's not a lot out there. And that's one of the reasons why we wrote this white paper and why, frankly, we're so enthusiastic about it, because I think there's this real need for guidance and practical guidance, and it's really hard to find. So that's going to be released. Plan right now is mid June. The organization we're co releasing it with is the Future of Privacy forum, and that'll be on the Immuta website. We're giving a talk at Strata New York on basically exactly this, on practical ways to govern machine learning. And that'll be, I believe that's in early September. Matt, are there other places you'd recommend that listeners go to get more info?

No. I mean, I'll kind of second that. There's not a lot on the open web. There's not a lot of formal reading out there. What I would say is this, though. In each domain there is an abundance of proper governance frameworks, specifically verticals, different regulations around management, PI. And I would say look internally for those who aren't individuals or small companies, but reside in large companies, I think the place to start is meeting with your internal governance team. Ask what type of documentation exists out there, and then also talk to application teams as to what has their documentation process been, what is their standards, as to how they work with governance teams and with internal data stewards and potentially even internal data protection authorities. I think the Data Protection Officer is becoming a real title now where people actually, that is their job. So I would say look internal first. Externally, of course, there's this thing called Google and you can search it. But yeah, Andrew's right. It's a shame, but there isn't a lot out there to really get started.

All right. Yeah, well, appreciate that. Definitely come see these guys at Strata, check out their white paper. I know that I've learned a ton from them already and hope to hope to see you again at a conference soon and maybe go out afterwards and avoid those boxer fractures for sure. But appreciate your guys with us wisdom here. It's been. It's been super helpful for me.

Thank you so much.

Thanks, gentlemen. I appreciate it.

All right, thank you for tuning into this episode of Practically AI. If you enjoyed this show, do us a favor, go on itunes, give us a rating, go in your podcast app and favorite it. If you are on Twitter or a social network. Share a link with a friend. Whatever you gotta do. Share the show with a friend if you enjoyed it. And bandwidth for Changelog is provided by Fastly. Learn more@fastly.com and we catch our airs before our users do here at Change Law because of Rollbar. Check them out@roar.com changelog and we're hosted on Linode Cloud Servers. Head to lino.com changelog check them out. Support this show this episode is hosted by Daniel White Knack and Chris Benson. Editing is done by Tim Smith, the music is by Break Master Cylinder and you can find more shows just like this@change law.com when you go there. Pop in your email address. Get our weekly email keeping you up to date with the news and podcasts for developing developers in your inbox every single week. Thanks for tuning in. We'll see you next week.