When data leakage turns into a flood of trouble

Cool things on Twitter. Like cool, like little visualization blog posts to kind of play around with. I think there's a ton of good stuff out there that I kind of like playing around with. Transformers and NLP is an easy one. In visualizations you see things like Deck GL that you can just make the coolest kind of visualization pieces around. There's a lot of cool stuff there. But I will tell you, part of it is as I always tell people, they're getting into data science. Don't get suckered in by that. Most of the data science that you could use that's useful in an enterprise hasn't come out in the last three years. It's been around for much longer.

That's a really good point.

Don't get into reading the latest archive posts and trying to take those account because going back to those classic problems, those classic either Kaggle competitions or other projects, and learning those techniques is going to get you much further along than following the latest pieces.

Don't start with GPT3. It's your very first thing.

Bandwidth for Changelog is provided by Fastly. Learn more@fastly.com we move fast and fix things here at Changelog because of Rollbar. Check them out at rollbar do and.

Were hosted on Linode Cloud Servers.

Head to linode.com changelog.

Whether you're working on a personal project or managing enterprise infrastructure, you deserve simple, affordable and accessible cloud computing solutions so you can take your project to the next level. Simplify your life with Linode's Linux VMs to develop, deploy and scale your applications faster and easier. Get started on linode today with $100 in free credit for our listeners. You can find all the details at Linoden or if you're not at your desk, just text changelog to 474-747 and get instant access to that 100 bucks. Linode has 11 global data centers and provides 247365 human support with no tiers or handoffs regardless of your plan size. In addition to shared and dedicated compute instances, you can use that $100 credit on S3 compatible object storage, manage Kubernetes and more. Visit linode.com changelog and click on the Create Free Account button to get started. Or just text changelog to 474-747. Get started today on Linode. Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show@changeaw.com community and follow us on Twitter. We're Racticalai FM.

Welcome to another episode of the Practical AI Podcast. I'm Chris Benson, a principal emerging technology strategist at Lockheed Martin, and with me, as always, is Daniel Whitenack, who's a data scientist with SIL International. Hey, how's it going today, Daniel?

It's going really good. It's a beautiful day here in Indiana and this weekend. I know in the last couple times we've started out the podcast, I've had a few projects, like tech projects, like the network at my wife's business and those things.

You always got something going on.

I always got some sort of tech. But this weekend was like. I was outside most of the weekend, so took a hike and then had a bonfire at my wife's family's place and birthday party. And so we were outside most of. The. Most of the weekend, which was kind of nice. Nice break from being in front of a screen.

I'm so jealous because as we record this, we just came out of the weekend where Hurricane Delta came in off the Gulf, and so we have had torrential rains all weekend inside. Not fun, that kind of stuff. Yeah, not. Not good. So I'm looking forward to sunny weather coming up.

Yeah, I also tried out this. There's an app called Inaturalist and you, like, take pictures of, like, birds or plants or other things, and there's like a whole, like, crowdsource community on there that will help you, like, identify what you're looking at and that sort of thing. My wife and I, both in the spring and through the summer and fall, we like to go mushroom foraging. And I've tried, like, all sorts of different apps that try to identify. It seems like something that should be, like, totally doable, but there's definitely a user risk in that as well. So talking about, like, a classification model and the risk associated with it, that's a pretty high one.

Yeah, I was. I was thinking that as you were saying that. I was thinking if I was doing that, I would either end up dead or I would end up with an inappropriate level of something I shouldn't have in my body, definitely.

So, yeah, so it's cool to find this inaturals thing. It's like a citizen science sort of deal. And, yeah, if you're interested in, like, you know, plants and animals and, you know, fungi and that sort of thing, it was pretty cool to find that.

So, joking aside, I actually do have that app and it's wonderful. I've been using it for a while, so I like it a whole lot. Yeah, for me, I'm just like, I'm hoping for good weather here. I'm excited. I got to tell you something. Tomorrow, by the time this comes out, it'll have just been in the past, but I believe a recording will be available. I'm giving the keynote at an IEEE conference tomorrow.

Oh, cool. Congrats.

Yeah, it's the Digital Avionics Systems Conference, and I will be doing artificial intelligence and autonomy, State of the Union, which is kind of ties where the entire science of autonomy and the sciences of artificial intelligence come together and where they intersect and where they don't.

Cool. Do you know if that will be posted online?

I think it will. I'm going to check, and if so, I will certainly include it in the show notes. Cool.

Yeah. Excited to listen.

So we have a very exciting guest today. Based on reading some of the stuff online on Twitter that we've looked at today, we have Rajeev Shah, who is a Data Scientist at DataRobot. He's an AI researcher and he's also a professor at the University of Illinois at Chicago. Welcome to the show, Raj.

Thank you very much. Thank you for having me.

So wondering if before we dive in to our topic today, which is going to be all about leaking information from your training and data sets, if you could tell us a bit about your background and kind of how you got to this point.

Like a lot of early folks kind of into data science, I probably have a long, kind of meandering journey, but partly I started off and as an engineering undergraduate, didn't want to do engineering, went off, studied law, ended up with a PhD in communications, where I ended up kind of studying just that intersection between technology and people was just really interesting and solving a lot of interesting research questions. And I used kind of various methods, both qualitative and quantitative methods, to kind of solve these problems. And, you know, this was at the time the Internet was booming, so there was a lot of interesting questions around that. I did that for a while, kind of moved out of academia. I was a sysadmin for a number of years, so worked in it. And then this data science thing came around and I was like, hey, wait a minute. I have a little bit of a background in math, and my social science background had taught me how to ask interesting research questions and kind of probe them. And then the nice thing is Andrew Ng made these wonderful courses, and I was able to kind of skill myself up on some of the newer algorithms and approaches that I hadn't learned. And so that was kind of my introduction into data science and landed my first data science job at State Farm and worked there for a couple of years, went over to Caterpillar, worked there as a data scientist, and now I'm over at Datarobot.

So just out of curiosity, was that. Was that his original machine learning course by any chance?

Well, yes, it was his original machine learning course. Yes.

I did the same one. It was kind of the first big, major deep learning course that he put out there. And I think it started a lot of us down this path in a professional context.

Yeah, I can't say I finished the.

Course, but, yeah, understood. It was a hard course. I remember it kicked my rear.

Yeah, no, he has a great knack for explaining things. Right.

He does.

His side gig was starting Coursera, so you got to like that.

Do you find that you're sort of like, with that background, thinking about people and technology, which is. It seems super relevant to AI, specifically in terms of, like, you know, I'm thinking there's a lot of interesting thought around, you know, augmenting humans with AI, but also systems. Like, you know, if we're talking about, like, active learning or something, which I think is extremely powerful, where, you know, you are sort of having a human in the loop with these systems. Has that made you think maybe more about those or brought a fresh perspective on those as opposed to just, you know, kind of thinking about the tech side of it?

Yeah. So it really helps me, like, remember how kind of socially grounded everything is. I think one of the valuable things of where you do a PhD and you collect your own data, you end up with kind of learning some skepticism towards kind of quantitative data because you realize, like, how it's collected. Like, what was the questions asked? Like, how does that formed on the survey? Right. You know, how do they handle the missings, all those different types of things that we often just accept when we're running algorithms. And so kind of having that understanding is really strong. The other pieces that's really kind of come around now in data science is, I think there's a movement in data science away from just focusing on algorithms, but thinking about the entire value chain, going from your raw data to how you finally use that model in a production setting, as well as the larger questions like, how does that interact with the rest of the organization? How does that interact with the rest of society? Where now. Right. I mean, we see now at conferences where they're being asked to say, you know, yes, you can tell us about how you solve the problem. But we also want to know what are the larger social, political, economic impacts of your research and wanting researchers to be cognizant of that.

Yeah, and of course that dips definitely into, you know, you mentioned a little bit of a background in laws as well. And of course that dips into like, you know, ethics and of course governance and all of those things which we're seeing. Seeing a definite surge in. And I'm guessing that you have thoughts there with that background. But it's really interesting to hear your perspective coming from that background. And have you found very many other people in data science coming from communications or from law? I hear a lot coming from maybe engineering or science or, you know, math or physics or whatever it is. Are people surprised when they hear about your background?

A little bit, but I think there's a few people now kind of coming in this way. Right. I mean, you had Patrick hall on kind of a couple of weeks ago. Right. Talking about, you know, what are the implications that data scientists or their people that are owning the data science products need to think about within liabilities, within an organization. So for me, it's kind of natural as data science expands where now the decisions that we're making are really impacting organizations and society, that, hey, there's a lot more people that are going to be involved and kind of giving scrutiny to what's going on.

Yeah. Well, I'm curious, maybe kind of from that perspective, how did this topic, you know, Chris mentioned that we're going to be talking about this topic a little bit today about data leakage. How did this topic come to the forefront of your mind? How did it become something that you started thinking about?

Yeah, I mean, so target leakage is a very common problem in data science. I think most data scientists who've been out there practicing for years, they know it, they're aware of it, they know that. And just to kind of level set for everybody, what, what target leakage is is when you use information from the future, when you're making a prediction, it's a very common problem that can occur. And so a simple way that it might occur is you're building an HR model. You want to predict what the salary is going to be for kind of your new computer, your CS hires, and you build a model, you take on all their background and it spits out a number and says, this is what my, the annual salary should be for this particular person. Now, what happens if, when you're predicting annual salary, one of the variables you used was the monthly salary. Because, remember, you start off with historical data from the past, and especially in an enterprise setting, it's easy. You have lots of features and variables in the data sets. You grab them and you might have something that's entirely related to your target when you're building the model. So it's kind of leaking that future information into your model. The great thing is when you're building this, the model looks flawless. Often when you have target leakage, the performance of your model looks really good. And it's only when you actually put it into production and you realize, wait a minute, I don't have that monthly number. Right? Like, it's always missing now. And your model all of a sudden doesn't look nearly as good as when you were training it. And so this is kind of the fundamental problem, and it happens in many particular ways where essentially kind of your model cheats a little bit and kind of gives you performance. That's really misleading.

Is this something that happened to you personally as you were starting to get into data science, where you had some of these experiences, maybe where you had to answer for a production model that was having issues?

So it didn't happen to me all the way at the level of a production, but I see it all the time. I mean, my kind of intuitive guess is like 75% of all data science models at one point have some target leakage. Now, typically, right, as you're kind of building the models, hopefully you're debugging it and trying to understand how well the model's working. Right. Using your explainability and interpretability tools, and hopefully with that and some domain expertise. This is where, as a data scientist, you really have to understand what the data elements are in your model. If you don't, it's very easy to fall into target leakage. But with those usually most of the time, 99 out of 100 times those errors are caught, hopefully before you finally get to production.

So what was it? Because as we're all working in this space and stuff, you've really put some thought into this and you've really noticed it. You've probably more than most have. What really oriented you on this particular issue? Because it's fascinating, but I'm not sure it's something many of us pay attention to very well upfront. And so what kind of got you focused on this in particular along this line of thought?

So I really like data science, and one of the things I like to do is often is when I see other interesting projects that are out there, I like to Rerun them and just try to understand, like, how did they solve this problem? What did they use? For me, I have a hands on learning style and it's just much better to kind of be in the data and with the code rather than just reading a page of formulas. And I remember years ago, Chicago was one of the leaders of open data and they put forth a model for predicting kind of restaurant inspections. The idea was, we want to figure out and sign our inspectors to go to restaurants that likely we're going to fail inspections versus ones that aren't. Right. Inspectors are rare resources. We want to just allocate them efficiently. And I remember when I went through that model, they had a couple pieces of target leakage. One is they used weather. Is if you're building a model that you're predicting way out in the future, often you don't actually have the actual weather. What's going to happen a month from now, for example. Right. So that's a form of kind of target leakage that can happen. And they also had a more subtle thing where they were using the inspector ID information in their model. And the reason this was target leakage is because in the past, some inspectors often kind of were very easy inspectors. Some were really harsh. I'll find everybody. But when they actually were going to deploy this model, they wouldn't know what inspector was likely to go to the restaurant. And so going through this process, I realized, like, wait a minute, they made a mistake. And I talked to the folks in Chicago and the actual model they implemented was a bit different than what they had shared publicly. But it kind of got me attuned to, you know, these types of problems happen if we kind of don't carefully scrutinize the data. And then just along the way, I see it routinely when I work with other customers where they bring in their models. We walk through them, I'm like, wait a minute, like this variable is really good. Should this be in the model? And then somebody thinks about it and they're like, oh, no, you know, you're right. Like that shouldn't be in the model. That's leakage. You see it in kaggle competitions. If you look at the history, history of kaggle competitions, there's many kaggle competitions where along the way they discovered target leakage. And they're like, oh, we have to redo the competition. Or the competitors all use the target information and make these unrealistically good models as part of the competition. And then the one that really highlighted it to me was there was an Article in Nature studying aftershocks. So kind of earthquakes and you have aftershocks. And the great thing was the researchers put the data out there, they put their code out there. So I was able to kind of play around with it. And once I started looking at it, there's kind of some funny things that kind of made me a little concerned. One thing is whenever I see a paper and I don't see a baseline model, that always concerns me.

Right.

This is kind of for me, a data science best practice is before we kind of jump into using fancy deep learning methods, like what does a simple model do? Right. We do this if we're working, whether we're doing a time series problem or kind of if an enterprise has a rule based system. Right. Let's find out like if we just do the bare minimum, like what the starting point is and then we can see, hey, now I'm going to apply this fancy data science methods and see how much of an increase I can get. Does that sound fair?

It does.

And so that was one issue that I kind of had with the paper. The other issue when I started looking at it was how they had organized and partitioned their data. So within data science, typically there's an assumption often that the rows of our data are not related to each other. And so what happened in this case was some of those aftershock earthquakes actually happened near the same time, near the same place. And so I made the assumption, well, if you treat them like they are related to each other, we can still solve the same problem. We can still use data science, but what we use is a different partitioning method. We organize the data differently when we're training the models, typically kind of the default. You'll see pretty much every blog post out there goes with random partitioning. All your rows of your data are the same. It doesn't matter where we put them. But the reality is often in a lot of problems, there's some relationships in your data. Maybe for example, you have data and it's broken up by state. So you have a bunch of observations for Texas and you have a bunch of observations for Florida. And what happens is there's some regional variations that if you know something about a little bit about Texas, you can probably make a pretty good assumption what the next person from Texas is going to do. And so what happens then is when we randomly distribute this data and we put some of Texas data in our test, kind of where we want to see if our data will generalize to the model can very easily cheat it can Say, hey, this is what I know about Texas versus all the other characteristics that we really want the model to learn from. And there's an easy remedy. It's something called group partitioning, where we just make sure all the Texas information stays together in one partition. When we train, all the Florida information stays together in one. It's a very simple thing to do. It takes a little bit of extra data science work to think about it and to kind of code it up. But then you can run it and see, hey, is there a big effect? Is there an effect of this state, you know, that is actually kind of leaking information as I'm solving it. Okay. I said a lot. I kind of want to make sure you guys are with me on this so far.

Definitely.

That was good. Thank you.

And that's what brought me back in this aftershocks. One is that's what I noticed is that there was some type of kind of leakage going on. And for me, the big issue isn't that there's leakage. I mean, like I said, it happens all the time. Most data science teams have it and I hopefully. I mean, most data science teams have confronted leakage. I think most of us who have experienced data science, it's not a big deal. It happens all the time.

Right.

It's nothing to be embarrassed about. You fix it, you move on and you've learned just another kind of data science battle scar that you have and you've learned from that. But my concern in this one case was this got to an article that was published. When I brought up the issue, it wasn't like we were on the same data science team and we're like, oh, shoot, let me fix that. Let me rerun the results and see is it going to make a difference or not. But there was quite a bit of pushback. And so for me, this brings up kind of the larger concern about as we welcome more people into using our data science tools and techniques, which I'm all for. Right. I work for an AutoML company, right. The goal is to kind of make data science available to a much larger set of people. But how at the same time can we balance and make sure that folks kind of learn some of these fundamental concepts and best practices for how to do data science and solve these problems.

So I'm curious, with this particular problem, it seems, at least in some cases, like one of those things, like you may not be able to like, deduce or it may not be obvious to you up front when you're, when you're first like creating a model and trying to solve a problem where you might be having data leakage. It's sort of like you see some weird behavior afterwards, or you see like degradations in your production model, or you see some maybe suspicious evaluation results and that might trigger you to kind of dig in deeper. And we can dig into those here in a second, like kind of detecting where you're leaking. But are there ways up front as you're getting into a problem that you could kind of set yourself up for success, you know, before you have to kind of retrospectively debug where, where you have leakage?

No, absolutely. And I think part of it is just that good problem framing up front. Data scientists I think are getting much better about this. But thinking about what is the problem, what are you trying to solve, how are you going to implement this at the end of the day? Right. What is the data looking like? What are the production systems that are coming in? A common problem that often happens is data gets updated in databases all the time. And if you're not using kind of snapshots, your model could fail to that, it could fail because of that. Let me kind of explain that. Imagine you're doing a model on claims and you're using the text of the claims. Well, if the text of the claims is constantly being updated during the life cycle of the claims, from when it was taken to when the final process was, well, that information is changing over time. So hopefully you have that conversation when you're setting up the model and the project and being like, hey, is this data that I'm using a training time? Are the fields going to be the same as when I actually go and set up my production pipelines at the time of prediction? So I think a lot of this can be addressed by just thinking through and having the domain experts and knowing what the data looks like at the time of prediction. Does that help?

Yeah, it does. I'm wondering, like, how do you connect this? Also like, is this connected to like you were talking about partitioning your data and you know, there's a lot of talk when you first learn data science about training and test and hold out sets and like cross validation and you talked about this sort of group partitioning, is there one or is it sort of a case by case basis in terms of like every project you do, you should consider what unique partitioning is. A lot of data scientists sort of just use one size fits all, basically, like cross validation is easy because I can just write this like one liner in my code and then like I'M set for not overfitting. Right. So do you have any thoughts on that?

And you might want to define a couple of those things along the way, by the way, for listeners who may not be familiar with everything.

Yeah, no. So that's great. And it kind of gets to. Right. Are there some good best practices I could use to. For avoiding target leakage? Right. And one of them is what is my default partitioning scheme? Right. When I get the data right. We talked about the issues with the random. How can I do it? Well, I think kind of the general best practice for data science is to use a technique called nested cross validation, where what you do is when you partition your data, you're making sure that you kind of break this up into. I don't know if I can sufficiently define nested crop into kind of different folds. But what we want to do is make sure there's different data that we're using for both the validation as well as the hyperparameter tuning of the models. That's another subtle thing that can come into play. So this is where kind of I. For folks that aren't new, I just have them set right. Nested cross validation is the default. It's a pain in the butt to code up, but it's kind of gets you in the best place. But kind of like we said with the aftershocks, that's not a hundred percent bulletproof. I mean, this is where you still want to have folks that understand a little bit about data science and have been burned and are just very skeptical. I think one of the takeaways is always to be skeptical of your model and its performance and think about what else could it be that's going wrong? Because otherwise it's going to happen when you go to move its production.

Absolutely. And as you were talking about that, I was also thinking like on the aside, you know, at least you have a lot of the control as you're busy doing the, you know, planning out the model and figuring out your training data and you can find those things and correct them. When you're dealing with a production environment or maybe plan planning at this stage for the production environment, how can you apply this line of thinking to that effectively? Especially if maybe you don't have control of that, you know, exclusively. You may be working with, you know, infrastructure people, database people, all sorts of different ties. Even if you've recognized it for training, how do you get it out into the real world and accommodate these same issues?

I've seen far too many models that data scientists have made that haven't accounted for all the production issues that end up just sitting on laptops that never actually get put into production. So even if it's a pain to deal with those IT folks on that and figuring out those pipelines, what data they have, getting snapshots of real production data, you got to do it. It's the only way to actually get your model in there. I realize lots of data scientists would rather just sit on their laptop, be handed some data and just kind of plug away at it. But if your goal is to actually bring value to the organization, to get models implemented into production, like, you got to have those conversations and figure that stuff out. And it's much better to do it earlier before you have, you know, you or your team spending time kind of looking at these models than doing it. So.

So let's say that I've tried at least to, you know, use some reasonable partitioning scheme. I tried to do the best that I could, but I put my model into production and I detect that there's something suspicious going on. How would you go about sort of retroactively looking into this issue of leakage? It could be other things, right? There could be like model drift or something where the state of your data was one way and then something fundamentally changed about the data or the problem or the time period or whatever, and you've got some sort of drift of the performance. But it could also be some like this leakage issue or something. So how do you kind of go about that when you have to do this more on the debugging side?

So one thing is, I really like that you're talking about monitoring your models and thinking about the data drift and concept drift, because I think that's an important element, right? Those things could happen and it could be entirely separate from target leakage that could be affecting your model. So, yes, who knows for kind of thinking and doing that. And then, yes, then we have to start kind of taking this apart and deciphering and seeing has the data changed from the time I train the model to when it's in production? Is that's what's causing kind of this lack of performance or going back and looking at those features? So I have a longer talk that I have on target leakage that I did with a colleague of mine, Yuri, who's, who's spoken at Open Data Science Conference, and he has it kind of different categories of target leakage. So like, we've talked about data partitioning as one of them. We've talked a little bit about kind of that initial set of data is another source of leakage. Thinking of that example of using that correlated feature or overwriting information is often another piece of there. Another example of when that can happen is if you have, for example, image data and it has some type of labels inside it, like using the explainability tools, interpretability tools, hopefully, then you can figure that out and go back and then fix that data for that. But two other places that it often happens is feature engineering, that this is one, it's a very subtle way, but when we kind of engineer and build those features that we're talking about decoding and deciphering, often people will make the mistake of doing it and creating this feature engineering on their entire data set. Right. People will do EDA on the entire data set. And that allows you to learn the ins and outs of your entire training data sets. But that also subtly leaks information. Now, because you don't have that holdout data set that's really kind of partitioned off from you where you haven't seen it from. And so that's a real subtle one that you won't be able to catch until you really think about how they actually did the feature engineering. And they're like, oh, shoot.

Yeah. So, yeah, I think that's a really important point that you bring up and something that really is not stressed at all. So, like, I think if I'm getting what you're saying, like, let's say that you have a data set, you have a feature, and you want to know, like, oh, is this normally distributed? Do I have to apply some sort of power rule to this to make, you know, fit the assumptions of my model? Or like, what? So you could do those things and look at nice histograms or plots on your entire data set. Right? So like, you get your whole, like, customer transaction data, you do that, but then you kind of partition off and you train your model on only part of that data. But the way in which you've created those models or the transformations you've done on those features is. Is informed by data that you're not using in training.

Exactly. Yeah.

So I guess, I mean, one thing you can do, of course, is when you're preparing, you can adjust your exploratory data analysis methodologies. But yeah, I could see how that one would be a really hard one to catch because it also would require, I guess, like really good documentation around how you did that feature engineering initially, which I know maybe were at least myself, not always the best at documenting things the way I should. How does that fit into this and the teams you've worked on in terms of maybe not so much the data side of things, but the process side of things. What are some of your best practices around how you document your process on projects?

Yeah, no, I mean, this is a very tough one to get because. So I think one of the initial Twitter thread that we were kind of talking about a month or two ago that led to this podcast was a thread by kind of Jacob Schreiber that looked at a machine learning package in genomics that it was doing this exact same thing that was using all the training data to get the insights. And then later, yes, they used cross validation, but they'd already kind of been kind of corrupted because they'd already learned everything from the training data. So solving this problem is very tricky here because it relies on your data scientists being very aware of these issues and not doing it because I think. Right. Decoding it from somebody's code is a bit tricky to kind of go back and see how they engineered their code, did they do the proper splits. And if we start thinking about things like time series, where you start having lagged features, it gets really tricky to be able to diagnose and look at somebody.

It's a hard one.

Yeah. I don't have a very good answer for you on the best process for it because.

Well, I have another way of going about it which kind of put you on the spot for a second and you can make it up, because I literally was wondering if I'm trying to tie everything that we've covered together because there's a bit of complexity to it and I'm trying to make it easy to understand for my own selfish purposes, so that as we get past the episode, I can start applying. Can you kind of think through a use case and it can be fictional, or it can draw from something that you've done, but kind of take us from the beginning to the end a little bit in your thinking and say, I'm doing this now and then I'm doing that can be anything. There's no wrong answer, or, you know, it's whatever you want, but. But kind of sequentially take us through so that as we come out of that, myself and maybe other listeners that are interested can kind of go, okay, I get that. I can. I can literally turn to my work after this podcast and go apply that.

So I'm not sure I could do it quickly. In 30 seconds, I will say we have built out a webinar that goes into detail on target leakage as well as there's an online course available that. So for me, this target leakage is really a fundamental concept that anybody new to machine learning really needs to kind of go through and learn all these different types of areas. So I've spent some time kind of actually taking all these different pieces together and building a little course with example problems for people to kind of go through. And that way it kind of triggers. So that way, hopefully they can learn from seeing some other examples of where target leakage might occur and they don't actually have to go through the pain themselves of experiencing it.

Awesome. And we'll make sure and link that in our show notes as well. I think that's great. And something I want to go through as well.

Yeah, I mean, there's a couple of rules of thumb, right, about just partitioning that data early, using all the interpretability and explainability tools to understand what are those factors in your model, there's having the domain knowledge, so making sure you're understanding what you're trying to predict. What are all the predictors, you know, is there anything that's kind of leaking in understanding your IT system? Making sure that, for example, records aren't being updated using good practices for, you know, creating features for training your models. We talked a little bit about nested cross validation. So another common mistake data scientists will do is on their hyper parameter tuning. So algorithms, for example, have. Some of them literally have tens, maybe hundreds of different knobs and levers, hyperparameters that we can kind of turn and modify when we're building out our models. A lot of data scientists, not all, a lot of them like to spend a ton of time, wait, if you ask me, way too much time on hyperparameter tuning. But a common thing that can happen is what you're doing is you're testing, you're moving the knob, you're moving the switch one position, you test the model, you move it another position, you test it, and you literally do this hundreds, thousands of times. And if you're doing all of this and using the exact same data set to test it on, what can happen is you can actually do what's called overfitting to your model. You can essentially kind of learn that validation data set that you're using to test it on by testing it a thousand times. And that is another form of kind of leakage that can happen where then you're. You've kind of built a model that's much more optimistic that you have much that has memorized its testing data. And so you Think it's working quite well, but actually it isn't. You just kind of got the figured out. The one lucky one that understands this testing data.

It's memorized rather than generalized.

Exactly. Exactly.

Changelog is the best way for you to directly support practical AI. Join today and unlock access to a private feed that makes the ads disappear, gets you closer to the metal, and helps sustain our production of practical AI into the future. Simply follow the Changelog link in your show notes or point your favorite Web browser to changelog.com Once again, that's changelog.com.

Changelog it's better.

So, Raj, it's been great to talk a lot about the sort of practicalities of this issue. I know there's things even, you know, we've talked about that are for sure, you know, things that I want to help better integrate into, into our team structure as well. I was curious. Now you know that you've, you've been with Datarobot for a while. You've been in, in academia. You also have a sort of unique perspective, I think, on machine learning and AI and are concerned at the same time, very concerned with these sort of practical side of things and productizing things. What are some of the things that you're excited about exploring over the next year or so in terms of maybe it's new verticals that you think that AI or machine learning is really going to make an impact in, or maybe it's methodologies or even just practical things that you're implementing that you think are going to make a big difference for you. What are some of those things? What are you thinking about as you fall asleep in the evening if you're thinking about machine learning and AI type things?

There's two pieces for me for this question. One is just for the techno side of my data science of just feeding me in terms of cool things on Twitter, cool little visualization blog posts to play around with. I think there's a ton of good stuff out there that I like playing around with. Transformers and NLP is an easy one. In visualizations you see things like Deck GL that you can just make the coolest kind of visualization pieces around. So I think there's a lot of cool stuff there as part of it. But I will tell you, part of it is as I always tell people that are kind of getting into data science, don't get suckered in by that. Right. Most of the data science that you could use that's useful in an enterprise hasn't come out in the last three years, right? It's been around for much longer.

That's a really good point.

Don't get into reading, you know, the latest archive posts and trying to take those account, right? Because going back to those kind of classic problems, those classic either Kaggle competitions or other projects and learning those techniques is going to get you much further along than kind of following the latest pieces.

Don't start with GPT3.

Very first thing, I kind of have a, maybe a follow up to that which is like, you know, you're talking about like those Kaggle data sets or other things and you've emphasized previously in the conversation like one of the like really important pieces of this whole puzzle in terms of creating value is like understanding infrastructure, understanding like production systems. What do you recommend to like, because sometimes I talk to new people getting into data science about this very thing, right? Like is it more useful for me to learn this sort of state of the art models or what should I learn? What do you tell people that are sort of just getting into the field in terms of how heavy they need to jump into things like databases like DevOps and CICD, like those things that are kind of common software engineering world things. How do you kind of prep people that you talk to?

It's tough, right? Because the expectation now for data scientists is that they have to know it all, right? That you, especially if you're going out on the job market, if you don't know all the pieces, you really feel like I'm not measuring up. But then when you go actually study and work with data scientists in an enterprise, most of them are very, kind of in a very narrow piece there and they're complaining that they'd like more challenges and like to do different things. When people are first getting into the field, I encourage them to know a little bit about everything. So you know, spend a weekend just playing around with Spark so you kind of have some idea of what Spark is like. Don't, don't spend two months on Spark, but just a weekend so you have some kind of ability to kind of know what the tool is and where it fits in to all of that and kind of go down the road like that. For a lot of these kind of technologies, databases is just have a little bit of understanding. So if you're at a conference you could have a conversation and know what are the things you'd want to go to versus not. But I think you can't go deep in all of them, right? There's just not enough time to go deep into all of them. And what I always encourage people to do is a much more project focused way of learning data science, where you solve some type of project, hopefully it's a passion project, something you care about, and you solve that end to end. And along the way you're going to learn a lot of those subtle skills of.

Right.

How to, you know, set up a webpage or how to kind of fire up a web server in Python, whatever. But you'll pick up those skills along the way to kind of solve the problem and then you'll really own those skills. Because if you solve a problem with it, you really kind of free that domain. You totally understand how to solve the problem. And you can go deep with somebody on that.

Yeah. Because if it's like, even if it's a side project or a passion project, what you're saying, which I definitely agree, I think, like, if you're going to get into this, like, choose something that you're going to want to work on because it makes it just so much easier to put in that. That time. But yeah, it's not going to be like, you know, let's say that I'm creating like a special webcam or camera, you know, detect my pets versus, like, you know, raccoons or, you know, something that, you know, is like, bothering my.

Trash or like, you have a problem with raccoons, Daniel.

Yeah, well, actually, right now we have a mouse hanging around, but I don't know if I could catch that on the camera.

Sorry, didn't mean to mess you up.

Yeah, if you do that, like, the, you know, the solution to that isn't going to be a Jupyter notebook. Right. You may want to, like, play around with your video and whatever in a notebook. I'm not saying anything actually bad about Jupyter notebooks because they use them every day. But at the end of the day, how you're going to solve that, it may be specific. Maybe you want something that you can look at on your phone, like a phone app, or maybe it's a webpage or a dashboard or something that you want to look at with that. And so it forces you to think about some of these intricacies of integrating a model into other things, which is something. I don't know if you agree, Raj, but it's something I see missing in a lot of. In a lot of training is like a lot of buildup around, even sometimes around model management, which some people are getting a little bit more into that, but not as much into sort of integration and that sort of world.

Yeah. And just to Insert before you answer there. I mean, I think that's a surprise for a lot of data scientists. Is that to your point, Daniel, is that at the end of the day, a model that's going into production is a software component that has to be deployed?

So, no, it is a good point because. And it's been often missed in data science, right, where the conversation has been around algorithms and not necessarily what's going to take to kind of work with it to get your model being used. And you're right. I mean, it often has specific requirements that you want. And you know, going back to your earlier thing about the Jupyter notebook, I'd say go ahead and try running your Jupyter notebook and you know, using your production model off that and that way you'll learn what can go wrong. Why sometimes the best way to learn is to kind of fall down a little bit. True, but no, absolutely. Often I think about data science as kind of the webmasters of 2020, right? When I remember there was, when the Internet was coming along, there was kind of the webmasters that they did everything right. They designed the web page, they run the networking switch, they kind of came to your house, solved your modem issues. And data scientists often kind of are put into that same category of having to fix every problem. And like the piece there where we're talking about productionizing, this is where, right. Many places now have said, hey, you know, right. If you're a data scientist, you're going to explore data. You'll be on this part of the data science team. If you're the software engineer type that is good at coding, that knows how to take that and put it into production, you'll be, let's say an ML engineer or data engineer and do that. So, you know, as kind of data science matures and we get a little bit more of these specializations and hopefully boundaries around this to help kind of do that. Because you're right, we need those skills at the end of the day to kind of get your model working for sure.

I think it's a great message to end out with here on because, you know, as the name suggests, Chris and I are both concerned with sort of the practicalities of this and bringing a practical side to AI. So I really appreciate your perspective there and perspective from really spending time building data products and productionizing models. It's really useful. We're going to link to those things that you mentioned during the show, the webinar and all of that. And so I encourage our listeners to definitely check that out. Also check out like we mentioned, Raj and I have talked a bit on Twitter so you can connect to him and I and Chris and the show there. We've got our community on Slack. If you go to changelog.com community you can talk about, you know, join our community and talk more about the topics there. And LinkedIn. We just had someone from LinkedIn on our show last week and that's another place where you can find us. So make sure and connect with us. And I really appreciate you coming on. Raj is a great discussion and appreciate the very practical ideas that you've given me as well.

If you enjoy practical AI, we would enjoy a 5 star review on Apple Podcasts, a blog post in response to something said on the show and or a recommendation to a friend or colleague. Those word of mouth recommendations really do make a difference. Practical AI is hosted by Chris Benson and Daniel Whitenack. It is produced by Jared Santo with music by the mysterious Breakmaster Cylinder. Thanks again to our partners who support this show's existence. Shout out to Fastly Linode and Robar. That's all we have for you today. We'll talk to you again next week. Sa.