#71 Scaling Machine Learning Adoption: A Pragmatic Approach

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI Transformation Journey on datacamp.com.

Hello, this is Adele Neme from datacamp and welcome to Data Framed, a podcast covering all things data and its impact on organizations across the world. Over the past year, the AI space has been animated with the transition from experimentation to operationalization. More and more so new schools of thought around MLOps and AI ops Data ops are emerging and data scientists are increasingly leveraging new technologies to ship models faster. One can say that this is a pragmatic philosophy towards AI development and this is why I'm so excited to have Noah Gift on today's podcast. Noah Gift is the founder of Pragmatic AI Labs and lectures on cloud computing at top universities globally, including the Duke and Northwestern Graduate Data Science programs. He designs graduate machine learning, MLOps, AI and data science courses, consults on machine learning and cloud architecture for aws, and is a massive advocate for AWS machine learning and putting machine learning models into production. Noah has authored several books including Practical MLOps, Pragmatic AI, Python for DevOps and Cloud Computing for Data Analysis. He has created content around AWS for top course providers including Udacity, O'Reilly, Pearson and Datacamp. You can find many AWS examples from Noah by following him on LinkedIn. Throughout the episode we discuss his background, his philosophy around pragmatic AI, the differences between data science and academia in the real world, how data scientists can become more action oriented by creating solutions that solve real world problems, the Importance of DevOps, his most recent book on the practical guide to MLOps how data science can be compared to Brazilian Jiu Jitsu what data scientists should learn to scale the amount of value they deliver, his thoughts on automl and automation, and more. Also, we'd absolutely love your feedback on how we can make Data Framed a better show for you and which guests you think we should bring on the show. I left a survey link in the episode description. Please make sure to fill it out as I greatly appreciate it. Noah, it's great to have you on the show you're someone who has a very prolific career in the data space and is a fountain of knowledge when it comes to creating and operationalizing machine learning models. I'm excited to discuss with you your thoughts on Machine Learning, MLOps, and your most recent book on it with O'Reilly. But before we get started, can you give us a brief background about how you got into the data space?

Yeah, my background in getting into data was a pretty long winded way. And part of it is that I studied nutritional science in college because of my interest in sports back in college. I did football, basketball in high school. So I've always been interested in sports. And nutritional science was to me like a fun way to learn more about myself. And at the time, we didn't know what the word data science was. But in a way, actually, my opinion is nutritional science is one of the easiest ways to get started with data science because it's your own body, right? And we actually did data science on our body in school. And I had a really fun teacher who let us do many different experiments in class. There's a class I took called Experimental Nutritional. I'm sorry, Experimental Nutrition. And what we did was actually centrifuge our own blood. So, you know, we would pull our own blood out, you know, centrifuge it, figure out what the LDL HDL level was. And that alone was actually a very cool data science moment because the intuition you would have was that, you know, the very healthy people in class, I mean, most people in nutritional science are extremely healthy, but like the most healthy of the healthy who eat carrots and, and, you know, celery all the time and exercise two hours a day are going to have, you know, awesome HDL, ldl, you know, ratios. And it turned out that there were some outliers. And one girl in class was, was, was actually one of the ones that I was very shocked with. I was friends with her and she was just, I mean, just exceptionally nutritious. You know, like, she was always, you know, like, doing the best things. But genetically there was a history of heart disease in our family and so we were able to diagnose from those experiments. And then there's other experiments we did with megadosing vitamin C. And then we measured the vitamin C output. And it turns out that, you know, basically vitamins are bs, you know, so, so I learned some really cool stuff that was, was very data science oriented in nutritional science. And I think it's actually one of the more interesting, maybe bachelor's degrees that someone could get if you're interested in, you know, you know, some of the things that you can later use for, let's say, a master's in data science. So then that was kind of what got me started. And then I worked at Caltech for a while. That also really helped me in thinking more scientifically. And I met a lot of people that were into Python and I just kind of accidentally picked it up. And then later, in fact, actually one of the people I worked with, which was, who's now in the news, is Dr. Koonin, who has written a book that's actually very polarizing called Unsettled. And it's. I don't know if anyone's ever heard of it, but basically what's so shocking about the book to people is that he's saying that climate science is unsettled. And that's probably one of the most taboo things you could possibly say. And the, the gut reaction probably when you hear something like this is you say, oh, of course, you know, you're a bad person, and how dare you, you know, even question science and everything. But it turns out that I actually worked for him for three years and I got to know him very well and I would say the least likely person I've ever met in my entire life to be a troll. Not interested in science, not very thoughtful, like, incredibly intelligent person. He was friends with Richard Feynman, like all this stuff. So I think that's kind of another interesting, you know, data point for data science was getting around these kind of people who even have the guts to say something that maybe is very, very unpopular. And I think that's another aspect of data science that, you know, I guess maybe stuck with me a little bit. And then later when I worked in the film industry for a bit, and then when I was in the Bay, finally moved to the Bay Area, I got more and more serious about probability and statistics and, and so one of the only ways I could actually get more serious about that was to get an MBA while I was time. And I took every probability and statistics class that I could, and I, I did a lot of actually Python programming in my MBA program, linear optimization and things like that. And it turns out that a lot of the stuff that I did is very similar to what I teach today, which is basically Python and analytics and probability, and it just, they didn't offer it in 2010. And then since then, I've gotten kind of more and more interested in machine learning and data science. And when I was a CTO of a startup, we had multiple Probability models that we would use to detect, who to, you know, do business with, and so on and so on. And so currently I teach at the Duke Data Science program, the MIDSIS program. Also I teach in the engineering program at Duke. I teach a course called Operationalizing Artificial Intelligence and Machine Learning. And I also teach data science at Northwestern. So I've moved more into teaching and doing consulting in the last several years.

That's great. And I'm very excited to discuss all of this with you. I think one thing you're very known for, and this is something that you can see in a lot of your courses and your books, is a pragmatic and practical approach to data science and machine learning that creates impact. This is something that stands in the title of one of your books, Pragmatic AI. Do you mind walking us through why you think a practical and pragmatic approach is so important for data scientists to adopt? And how can data scientists be led down a non pragmatic road when it comes to data work?

Yeah, I think there's a. Let's, let's riff a little bit on that book by Dr. Koonin, because I think it's a great example. So his point in the book is that there's several different points that he makes. One of them is that it's difficult to make predictions, which I would hope any data scientist would say that's true. And one of the things that he brings up is that the damage that's already been done to the environment is baked in for the next, let's say, 50 years. And I think that forget about whether you believe him or not. And I think I really don't know enough about it to, to say how, how severe climate change is. I really just am ignorant, so don't listen to me on that. But the part that I think is interesting is if it's true that the damage is baked in and we really, for the next 50 years have to kind of live with what happened, then I think this is where the pragmatic approach comes in, is like, we can't argue about, well, we should do this or we should do that about something that in the next 50 years we have to address with actual adaptation. And I think that's where things could get interesting, where, you know, what could we do to operationalize climate science? Right. It doesn't mean that, let's say if you're the most liberal, most progressive, you know, climate person, and I would probably put myself into that category a little bit. I mean, I have electrical, electric vehicles, solar battery, Smart home, blah, blah, blah. But the part that I think is interesting beyond all that stuff is if we can't change for the next 50 years, what's happening? How do you operationalize the effects of climate change? You can't just pretend that like it won't happen. And so this is where I think the pragmatic approach comes in, is what could we do? And then, and then it really gives you constraints when you know that there's no way to, to really get around something that's happening. Like if sea levels are going to rise, well, what do you do? You can't just say, well, it's somebody else's fault and, and I'm mad at them, right? Like you what do we do? Do we, you know, apply technologies that can, you know, you know, maybe address coastal areas with, I don't know, houses that go up and down or something like a, you know, a desk or, you know, can we figure out ways where, you know, we can live better with higher, you know, climates? Maybe we have more, you know, solar powered homes with default material. Right. So you could basically, all homes are basically solar, you know, material. And then because the temperature is high, it doesn't really matter because you can automatically it based on the new energy you have. So, so that concept of there's a constraint that is immovable and then you must do something with that constraint, I think is really the pragmatic approach. So then now let's get into data science, machine learning. If you, if you say I must get something into production that helps my company or improves the, the experience of a cancer, you know, a person that's been diagnosed with breast cancer, for example, if, if you, if you move the constraint to whatever I do must improve their health outcome. That's pragmatic. I think the academic approach is, well, is like kind of thinking more about the why. Like, well, why do people get cancer or what can we do to get this tool to be better or whatever. It's not that those things don't need to occur. In fact, I would say we're way over optimized for the why and the research and everything. But what we have very little of is, is people with a sense of urgency that look, there's people dying, the world is in trouble, we have misinformation problems. Those are like urgent, urgent problems. And they don't necessarily require like, you know, hand waving. What they do require is actually specific actions that improve an outcome. And that's what I would call pragmatic AI.

I couldn't agree more. And it's super important to adopt a pragmatic AI approach to solve these real world problems as an industry and as a species. You hinted at this throughout your discussion, but I think oftentimes there is a disconnect between what data scientists learn in academia or do on Kaggle versus what is expected of them in the wild once they join an organization. Where do you think this disconnect shines the most? And why do you think it's a struggle for academic institutions to bridge the gap between academia and industry?

Yeah, I think that's a great question and I think you could. It's not one thing, but you could definitely make a checklist. And I think let's just start, you know, anywhere, let's. One of the things that I first comes to mind is that most people that teach at the university level have never had a real job, right. By a real job, I mean an industry experience. So that, that is a problem, right? I think then that's probably one of the factors of why things have not been implemented into production. Now, I don't necessarily want to, you know, throw stones at people who are researchers because I think it's a very important job. I think that maybe those people shouldn't be the only people teaching data science machine learning. So I think that's one thing is there's a mismatch, right? If, if you've never had a job, you know, imagine someone you go to like your local hardware store and the person that owns the hardware store, that is really a business person and, and they know about buying lumber and like selling lumber and. And it literally have never built a bookshelf. And you say, hey, how do I build a bookshelf? They could theoretically probably tell you how to build a bookshelf. But the person that works there, that builds bookshelves all day long, that's a carpenter. That's who you want the carpenter to help you. Not even though the person who owns the hardware store is very prestigious and wealthy and has high status, that doesn't necessarily mean that you want that person in your house, you know, installing the bookshelf. And I think that's one of the issues. Another issue is that the tools that are necessary to teach data science are very different than the tools that are industry tools. So a good example is I'm working with Duke right now on analyzing whale sound data. And the data set, I believe is around like 30 terabytes. And, and so, you know, you can't just be like, oh yeah, let's just, come on, let's Put it on my laptop. Let's just play around with that a little bit. It's like that just doesn't fit. And then also you can't be like, hey, import pandas as PD and let's make a data frame with this. 30 terabytes doesn't work right. So it's easy to show trivial tools when you're teaching data science, but then the real world is way more messy. And then it turns out that, oh, now I got to use a different tool. Like maybe I have to use data bricks or I have to use SageMaker and I have to do all these pre processing and spin up clusters and, and that's not necessarily something you just, in the first day of class you're like, okay, come on everybody, let's spin up 20 machines and then we're going to set up data bricks and then we're going to analyze 30 terabytes of data. That's, that's not like a kind of a, you know, you just kind of like flip that out of your notebook and start teaching it. So I think that's, that's probably one of the other aspects of it.

Given that a lot of data scientists are exiting academia and entering the industry, what do you think are considerations data teams should make when designing and scop machine learning or data science projects that are often not taught or discussed in academia but are learned on the job?

Yeah, I think the realism of building a software system is really not taught. And I think in particular DevOps. And what DevOps is, is the ability to automatically test your code and improve the quality of your code and deploy your code. And so I think that's something that a company would be wise to do with anybody that's hired is make sure that they understand the fundamentals of DevOps. So can they test their code? Can they lint their code? Do they know what a build system is? Can they do automatic continuous integration and then can they deploy, let's say a flask app into production somewhere? And I think that alone is really important and that might address 80% of the issues that people have.

That's great. And you mentioned here continuous integration and DevOps, what are some of the tools you tell students to learn in order to bridge that gap between traditional data science education and DevOps.

Yeah, I think with Python in particular, I think it's the tools that I would recommend would be GitHub actions, I think is probably the easiest way to get started with continuous integration. And then another thing that people could use is pytest for testing and then some linting tool, either py, lint or flake. And just those would be, you know, I think, go a long way to, to making someone successful in doing continuous integration and continuous deployment.

Another angle to this when working in industry is about bridging the gap of value. And ultimately the data team is producing a model that is solving business problems. And these solutions will be owned by a functional leader in the business units. What do you think are some of the pitfalls data scientists fall into when working with business units that they're aware of before joining an organization?

Yeah, I think one of the things that I see a lot is, and I learned this initially, I learned the solution to this initially when I was in graduate school was I had a really wise statistics professor that told me that when you're presenting something to an executive, you don't want to necessarily show them a bunch of code. You want to have a summary. That's a paragraph that says, here's the problem I'm solving and here's my recommendation. And I think that's probably one of the big things that data scientists don't do is very quickly state what the problem is and how they're solving it. Because then the business person who may not know what code is or may not have, you know, knowledge of symbolic notation or whatever, they can just go, oh, you're trying to solve this, and then this is what you recommend? Well, we don't want that. Give me another problem statement and a solution. I think the quicker you can get to the problem statement and how you solve it, that will at least, you know, let you work on the right thing. Because the, the, the, the tragedy would be to work nine months on something that's really complex. And then it turns out that they didn't want that to begin with. And I think that initial disconnect is the most important feedback loop. So that's one. And then the second is that I think because the feedback loops have been so long that by the time you come up with a solution, it may not even matter anymore. And so another approach that I think many data scientists would be wise to do would be to embrace tools that do a lot of the automation for you. And, And I think AutoML in particular is one of those ones where I just really just how, how I'm confused about people who don't want to get vaccinated. It's like what, you know, like, it's like, you know, make polio great again. It's like what, you know, like it's vaccinated. It's the same thing when people don't want to use AutoML, it's like, what? Why would you not want to use a tool that initially helps you solve the problem? It doesn't mean that that's going to be the ultimate solution, but initially, why wouldn't you use some automation? And I think that's one of the places that data scientists could, could really get much more productive use of their time with, is that they embrace as much automation as possible and again get to the end of the feedback loop which is presenting the results to the business leaders. Because it could be that you're working on the wrong thing. So who cares if you hand coded all the hyper parameters yourself if they don't even want what you built?

You know, having a tool agnostic approach in data science is so important. And I'm excited to table our discussion around AutoML to the end of the show. Continuing on your book Pragmatic AI, a big focus is leveraging cloud computing technology to deliver high impact data science. There's a variety of cloud platforms data scientists can learn, like aws, Azure, Google Cloud. But you're a big proponent of starting off with aws. Can you walk us through why you view AWS as the better cloud platform and how does it help data teams?

Yeah, I mean, I think part of it's just from a data science perspective, they have the largest market share and that that's probably the number one reason I think AWS is probably the safest choice for most people to go with. And then likewise like, well, why would you use Python? Well, one of the reasons to use Python is that most people know it. And same with aws. If most people know aws, it's easy to hire people. So I think in general, and this is me as an engineering manager CTO for many, many years, is that one thing that people don't really take seriously enough is that the best tool isn't necessarily the one you should use. And so for example, I used the Lang, the language Erlang for a lot of my career because in theory it does actually have some very interesting characteristics about high concurrency. In practice, nobody knows it and so you're dead in the water. So. So the same thing with, you know, a language like Julia or an exotic cloud platform like let's say Google Cloud or AliCloud. It's not that they're bad. In fact, in some cases they may have better technology for certain aspects, like BigQuery for example, I think is an incredible tool. But now we get to the implementation details and does the majority of people that you hire do they know it. And I think that's probably the reason why AWS is the best choice initially. Now, I think a close second, though, is the Microsoft cloud. And I think there's a lot to like about Microsoft, and they're really challenging. So I would say, you know, again, my recommendation would be from the data perspective, just literally looking at the numbers of who you could hire AWS first. But Microsoft is probably not a bad choice either.

Definitely Microsoft is making a lot of gains in the cloud space, especially given the recent work with OpenAI and GPT3. I think this marks a great segue to discuss your most recent book, Practical Mlops with O'Reilly. Before diving into the details, do you mind highlighting why you wrote the book and what are the main pitfalls you find organizations face when operationalizing machine learning models?

Running data science teams, multiple data science. I've hired all the data scientists from scratch and then, you know, run that. I've also, as a consultant, hired and built data science teams. And one thing that I saw was that, that really we could. We hired some incredibly talented people, but the constraints were set up in the wrong way. Where and I would blame myself partially for this, is that if you don't tell someone what they're building or what they're working on, then how could you even measure whether it's successful? Right. Again, going back to, you know, like Covid or something, it's like, if we don't have data that says who's vaccinated, who's not, how many people have it? Like, you need to have a dashboard that tells you what's good. And I think that's one of the biggest problems with data science is that nobody even knows what's good. I think we're getting closer now, but if you just say, hey, we have all these PhDs in our company and we're doing data science, it's like, that means nothing. And I think that's one of the reasons I wrote the book is, you know, kind of getting into the details about, like, what is good, what should you do? And how could you measure whether someone is successful or not doing data science? That's one. The second is that I was at something called O'Reilly's Food Camp, which is this thing that you get invited to sometimes if you wrote a book for O'Reilly, it's kind of cool, you know, geek out session. And I was talking to Tim O'Reilly, who owns O'Reilly, and, and Mike Lucatis, who's Another person who's been with or a long time, and we had a discussion about, you know, why is it machine learning isn't getting results? Why can't we be 10 times faster? And, and really one of the things that we came up with was looking at things like when there's a pandemic like COVID 19 or there's things like breast cancer or climate change or whatever, where you know that there's some real big problems that could be addressed by machine learning. Why don't we just have a sense of urgency and solve those problems? Why can't we be 10 times faster? And I think that's really something that was the, really a driver for the book was let me see if I can write a book that makes the case for machine learning being 10 times faster, data science being 10 times faster, where instead of focusing only on the technique, we're focused on the result and the result is more important than the technique.

And why do you think machine Learning is not 10 times faster today? What are some of the reasons behind it?

Yeah, I think it's, you know, I think people are too precious with the technique and not focus enough on the, on the actual outcome. And I, and I, and I share the story in, in the book, in the final chapter where, you know, kind of out of accidentally I got into Brazilian jiu jitsu. I wasn't like looking to get into martial arts. I just happened to work somewhere where there's all these pro, you know, pro UFC fighters training and I became friends with them and then I started, started training with them and, and, and I spent years and years doing grappling, but more of like a, a real world kind of grappling where, where you're, you're training towards, you know, pro fighters. And one of the things I discovered was if you went to somewhere that was more academic and that more academic grappling is when you're in a, in a GI and so you're in a uniform and there's certain techniques that only work when you're in the uniform. And that's very similar to, to, to university. Right. There are certain things that only work in the university because it's constrained. You have a very constrained model. And then when I would train with people who were experts who are better than me, and, and then they would try a technique that always works with the GI uniform. And then when they're out of it, it didn't work anymore. And, and you know, someone would try to get my arm in a lock, which, which if you're Again, if you're in the uniform, you're in the academic setting, it would always work. But, you know, I've even been able to pull out and free my arm from Olympic judo. People who have meddled in judo because they know this a certain constraint. So I would say the same thing with machine learning. One of the reasons why people, I think, have had so many problems is they're looking at things only from an academic perspective and a technique perspective. And really the technique is interesting and it's important in some, some sense, but ultimately it's like, does it work? And I think that's the part of machine learning that we need to be more focused on is, is forget that you had a better technique or didn't have a better technique or whatever. Is, did your company use your machine learning model? Did it improve things and how quickly did you implement that solution? And I think that's really the sense of urgency on how to work on the right problem and at the right time.

That's awesome. And I really like the comparison between Brazilian Jiu jitsu and data science and machine learning. So the book does a great job of introducing a lot of modern tools machine learning practitioners need to work with to start operationalizing machine learning models. Do you mind walking us through the different subdomains of MLOPs and what are the most important tools practitioners should learn today?

Yeah, so I think one of the first places to start, which is probably the weakest place for most data scientists, and I know this because I teach a lot of data scientists, is DevOps. I think you must have a mastery of DevOps, so you should know how to do continuous integration, continuous delivery. Fortunately, it's not that hard. You can probably learn the majority of DevOps in a day or two. And so I think that's a fundamental. Now another step as well is having some kind of a mechanism to manage your data. And data governance is a very large topic, but in general, in the cloud, it means that your data is most likely in some form of a data lake. And so you have it in Object storage. With S3, for example, once you have your data in the cloud and it's in something like object storage, you have the ability to have infinite capacity of disk IO or, you know, CPU. And then you can use tools like SageMaker or Spark or Athena or some other big data technology. So I think that's probably the second big one, is you must have it in an environment where you can use modern tools. And then in terms of more of the machine learning implementation, I think a big Question that people need to have is again initially when you're building a prototype, how much of the work do you need to do yourself with tuning hyper parameters? Should you really be training the first version of your model 100% yourself? Like for example, if you're just, you have X and you have Y, you have two columns in a data set, do you really need to hand pick each one of these algorithms? It's, it's kind of a reductive problem. Why not just click and have the machine do it for you later? As you get more and more sophisticated, maybe you don't want to do have the machine do everything for you, but I think again embracing the high level tools from the beginning with machine learning and then the final part of machine learning would be, you know, where are you deploying this model? Are you bespoke, building some, you know, complicated system or are you using a tool like let's say SageMaker that has automatic ability to scale the model up and down version the models, do a B testing. I think reinventing the wheel is a very bad idea for people. You should just use the best tool that's available. And, and again a good example of one potentially would be something like Edibu or SageMaker.

In the final chapters of the book, you highlight some of the key critical challenges data teams face in mlops. Key amongst them are ethical and unintended consequences, lack of operational excellence, a hyper focus on small details as opposed to big picture thinking. Can you share your thoughts here? How do you view the future of MLOps? Maybe alleviating some of these challenges?

Yeah, I mean just to kind of address them one by one. So I think the ethics part, you know, is, is easy to get hand wavy and self righteous about and I think most people are offended when you're self righteous. It's just not a good technique to, to get someone to adopt your point of view. But that doesn't mean we can't talk about ethics and misinformation and, and some of the impacts. I, I think one way to address it would be, you know, if you're a talented person that be careful about who you work for. And I think that's one of the ways that you address the problem is if you know that you have the skills to do incredible things. I personally would recommend you work for something that's helping the world make it a better place. And I think one, one of the ways you could figure this out is with information. You know, I could categorize things in three levels. You could have harmful information so again, telling people that the world is flat or that they shouldn't get vaccinated because it's got a microchip that's harmful disinformation or misinformation, you're, you're really hurting the world. And then there's B, which is neutral, right? Like let's say you're making Disney movies or something like, okay, you know, it's entertainment, but that's neutral. And then there's, there's the category C, which is you're actively working on things that work, make the better the world a better place, like helping cure cancer, you know, or you know, providing housing for low income people or educating people or whatever. And I think that's really one of the ways to address this is just don't work for companies that do misinformation and disinformation. Eventually they'll crumble because no, no talented person will work for them. So I think that's one of the ways you can address it without being self righteous and hopefully people will, will do that. Now the other aspect of it is, you know, smoking. Focusing on the small details versus the big T details. Yeah, I think this is again goes to the technique versus the implementation. Is that it? It doesn't mean that small details are not helpful. But when you're first getting started, if you realize that urgency is very important and that, you know, a good example would be if you look at World War II and Britain in particular, there was a period of time when they were under siege by Nazi Germany. And, and they really didn't know if they were going to be, you know, completely destroyed or not. And so they have a window of, let's say, you know, 30 days where they have to figure out how to shore up their defenses, turn off the lights so they don't get bombed. Should you be working on things that don't solve that problem? I mean, that's a really good constraint, right? It's like we will literally be eliminated by a terrible force unless we do xyz, which is turn off the lights, you know, you know, you know, whatever, whatever the things are that can prevent that invasion. So I think the same thing. Not that you need to be worried about, you know, Nazis invading you, but that if you focus on, okay, what is the only things that I can do that will get this model into production. It doesn't mean those other things are not important, but how do I focus just on the things that immediately help me? That's a good heuristic that you can use to be more Productive.

And what do you think our tactics, data teams can do here to adopt this heuristic and operationalize it and formalize it?

I think one of the things would be to look at maybe a KPI and one KPI could be what is the frequency of models into production?

Right.

And if I think that's a, that's probably I, I know this metric for software engineering in particular is a reason, number one, which is how often do you deploy your software? Right. I've worked for companies where they deployed once a week and I've worked for companies where we deployed 20, 30 times a day. The ones where we deployed 20, 30 times a Day, we actually had a very, very healthy culture where things are constantly being improved. The ones where we only deployed software once a week, that's typically a sign there's something wrong. And I would say the same thing is if you can't deploy tons of different models or experiments every day, doesn't necessarily mean 100% production. But let's say quasi production where there's people in your company that are using those models. If you can't do that on a daily basis, I think that could be one of the things that you could measure and fix. Right. So you know, let's measure that as a KPI. Why aren't we producing multiple models per day into production? And, and let's fix it. Let's figure out the root cause and use the five whys technique. You know, why can't we deploy models into production? Well, every time we deploy a model to production, it blows up. Okay, well why does it blow up? Well, we, we don't have continuous delivery. Why don't we have continuous delivery? You know, you kind of break, break it down until you get to the bottom. It's like, oh, actually it turns out that there's really no reason why we don't deploy models into production. That's rational and we can easily fix this. Let's go ahead and do it.

And to play devil's advocate, do you think over optimizing for a KPI like amount of models deployed can lead to a data team to models without necessarily having ethical considerations before they deploy them?

I think any KPI needs to be taken with a grain of salt. So just like, I mean especially data scientists should be aware of this. Like you can't just over optimize and overfit right on one feature because then you're, you're, you're basically making a bad prediction that doesn't, doesn't conform to the real world. So I Think this is just one, one of many data points initially though I think it's probably a good one to start with because it will flesh out some of the excuses of like, wait a second, we only deploy a model once a year. It's like, let's, let's solve the ethical problems after we can deploy a model at least once a day. Once we've deployed a model once a day, and believe me, I'm a huge fan of ethical concerns of this. But, but let's first make sure we can do anything. Then once we can do something, then let's dig into maybe then we wait, wait things out and maybe there's another KPI that's equally important that says, you know, what's the impact? You know, let's say you're a credit card company all of a sudden, you know, minority groups don't get credit card offers anymore for the next 60 days. Yeah, that's really bad. You better fix that. You know, and so you could look at that as a KPI.

That's spot on. I think there's a great opportunity to integrate risk management best practices when deploying machine learning models at scale. Pivoting away into the future. Given the evolving nature of the data science and machine learning space, what do you think is advice you give up and coming practitioners to keep their skills competitive and remain hyper focused on creating impact?

Yeah, I think one thing is that I wrote an article a couple years ago that said I don't think there'll be data science job titles by 2029. And it was actually a fairly polarizing article and I think I've actually been proven true. But, but my, my point here is that, not that I think data science is bad and you shouldn't do data science, in fact, I think it's actually incredibly helpful. But you can't just be a data science. My opinion, just data science is not enough. I think, think you're, you're using that as part of what you do. And let's go again back to mixed martial arts, so to say, like I'm a grappler. That, that's, that's true and that's helpful. But if you're fighting, you're using striking wrestling, jiu jitsu, muay Thai, you have all these things that you're doing. So I think a data scientist, if you only focus on the fact that you do data science, it's, it's a little too narrow. But if you can say I'm a biologist who does data science or you know, I'm, you Know, a machine learning engineer who does data science, or I'm a journalist who does data science. I think that's a better direction for your career because you can't be pinned down and kind of like, oh, well, you just do data science. Like. No, data science is one of the things I'm an expert in and that's why I'm so effective at journalism, for example.

I think this touches upon a career philosophy you have as well. What do you think about the trade off between being a specialist and a generalist and the importance of being multidisciplinary?

Yeah, I don't know if there's that many examples of specialists that the risk reward ratio pays off. Maybe if we look at the Olympics, that's a great example of a specialist. But does actually that pay off for most people? Probably does not pay off to be a specialist at that level. But I think if you look at most of the things that happen in the world, having a multidisciplinary approach is really the best way to go and it allows you to be flexible about what happens in the future versus only being focused on one particular thing. So I, I think in particular with data science that it's happening, it's changing so quickly that, that it's better to have more of like a, an opportunistic or greedy approach to it, which is, you know, for example, the next five years I could see now clean energy becoming really a big deal. Like it used to be more of like, you know, know, a super hardcore progressive agenda and people were turned off by it because of that. But it actually is not fake anymore, it's a real thing. And in fact I, my, my hunch is that it's going to be pervasive in, let's say, five years. And so if you're only focused on data science, maybe you're going to lose out on that because you're, you're kind of leaving that up to grabs or another one. Is healthcare, right? Like healthcare is potentially going to go undergo rapid changes. So if you're more interested in kind of having the flexibility to jump into anything, I think you're more suited to being on the cutting edge.

I wholeheartedly agree and I think there's an aspect of personal reward that only being a multidisciplinary person provides. Now we've mentioned AutoML quite a bit throughout our discussion today. Given the evolving tooling stack and data science, auto machine learning tools and NLP innovations like GPT3 and GitHub Copilot are definitely shaping up to be disruptors of a lot of the data science workflow. What do you think are the major trends in machine learning and data science that will disrupt how we think about data work and that can act maybe as a backing up of data science roles, not necessarily being the same in 2029?

Yeah, I would say just to be clear. So I think data science, the job title I think may not and we'll see if I'm right or I'll even clarify say 80% of data science job titles. Let's do the 80. 20. Probably my opinion won't exist by 2029. But what I, what I do see with all the automated tools here is that either you embrace the automation or you fight it. But the people that embrace it are going to be very wildly successful because you're able to get results more quickly and you're, and you're focusing on things that matter versus you know, re implementing things poorly that a machine can do better. And I think that's really what we're going to see with a lot of domains is that the people that can embrace whatever happens and use the best tool for the job are going to be wildly successful. And so I think that's one of the big takeaways is that, is that humans when they use advanced technology are massively productive. A really good, I mean one of the simplest examples would be, you know, a drill, right. I mean if, if you've ever taken a screw and screwed it into a wood yourself, if you're extremely strong and you have a very strong grip. Yeah you might be able to just screw it into like a board or something. But why would you do that? You could just take a drill. It takes literally under a second to drill, drill, you know, a two inch screw into, into a board. It's the same thing. It doesn't, I mean how does it even related that a carpenter is a worse carpenter or a better carpenter because they used a drill, it's non sequitur. So I think the same thing applies to data sciences. The best data scientists are going to use the best tools because they're going to get more output, they're going to get better things that they build and that's really the future.

How do you see the data science workflow looking like given rising automation, do you think that even data preparation and cleaning will be automated?

I think some of the cleaning will be automated. I mean I think it may be more like the self driving car thing where and again we don't know what's going to happen. I'm A little skeptical that in the next five years we'll get self driving cars. I think we'll get better and better assistive technology. I think the same thing with something like data cleaning. That's a pretty complex topic. I think we could have some really good you know, assistive technologies that again why would you not use these assistive technologies? I'm not 100 sure you can automate manically do all of it but you can do a lot of it. And I would say one way to think about AutoML is that I think in a way it's, it's kind of like a false flag. You know people like look at, they look at Automail they're like, they're like automail. It's like for a second that's just one of the things it would be like saying a drill is, is the only thing to think about when you're building a bookshelf. You could have a drill, you can have a saw, you can have a level. The machine learning, hyperparameter tuning and the automatic training is one aspect. But I would call the whole thing Kaizen ML. Right. Which is, which is basically like every single aspect of what you're improving is touched. The business problem is touched. The software engineering problem is touched with DevOps. The data engineering aspect is touched with these automatic cleaning tools. And, and I think all of that together or Kaizen ML or ML Ops, those things are the things I think that we're going to see a lot of changes on. So I think getting too caught up in just the word automl again is really a distraction because it's one of many things that's automated. And if you get hung up just on the fact that it's automl you're losing focus of the fact that the whole thing should be automated. Every single thing. And if it's not automated it's broken.

So what do you think then will be the differentiator of high impact data scientists? Once a lot of data work is automated what are the marks of a great data science portfolio?

That's a great question. I would say re implementing things that can be done done by automated tools is the worst portfolio. So let's start there right? So like if you show how you did a bunch of hyper parameter tuning I wouldn't work on that. Now I think a more interesting portfolio would be that for very exotic topics. Here, here's, here's one that's a personal topic that I think is interesting is homelessness. And I think think we still don't have comprehensive data on this. But imagine, you know, you know, building like, like really comprehensive analysis with specific recommendations and conclusions around homelessness. Like, for example, you know, X percentage came from here. X percentage have this. Or like, like that would be a great example of, you know, thought leadership around data. So any kind of a thought leadership that's bespoke and creative, I think is a tremendous data science project. So that's one. Because you can't really. A human versus a machine doesn't even make sense in that, in that context. So don't compete with the machines at things they're better at than you compete with the machines at things that they can't do, which is add context to a problem. The other one, I think would be showing that you have very strong software engineering skills, which in a way is a great way to compete against other data scientists. Because if almost all, let's say 80% of current data scientists in 2021 don't do software engineering and you do, you're going to stick out. And so I think that would be a great way is show, you know, that you can do continuous integration, continuous delivery, that you have maybe a website that you built that automatically does deployment. I mean, that's something I cover in the Coursera course that I built with Duke is doing a Hugo website that automatically deploys. I think anything you can do that shows you have strong software engineering fundamentals would also be a way to stick out.

I think what's exciting here as well is that AutoML can act as a tool to lower the barrier, to create this thought leadership. Finally, Noah, before we wrap up, do you have any final call to action?

Yeah, I would say just in terms of, of data science, there's so many intelligent people that I've come across, thousands of people I probably interacted with in teaching or books or seminars that my advice to you would be that, you know, get as much skills as you can, but then apply those skills to things that are unambiguously good for the world and you can get paid very well for those. I think that's easy to have, you know, workforce, maybe a company that is unambiguously bad, but it's just as easy to work for a company that's unambiguously good. And if you're talented, you owe it to the world to really focus on things that help humanity, not destroy it.

That's very inspiring. Thank you so much, Noah, for coming on the podcast.

Happy to be here.

That's it for today's episode of Data Framed. Thanks for being with us.

Us.

I really enjoyed Noah's insights on how to scale value with AI by using a pragmatic philosophy towards AI development. If you enjoyed this podcast, make sure to please leave a review on itunes. Our next episode will be with Shafri Bahar, VP of Data Science at Gojek, on the Data Science Powering gojek. I hope it'll be useful for you and we hope to catch you next time on Data Framed.