#1 Machine Learning Engineering for Production (MLOps) Specialization [Course 1, Week 1, Lesson 1]
Andrew Ng introduces MLOps and why getting ML models into production matters. He walks through a factory defect inspection system to show how deployment actually works, then explains the gap between proof-of-concept and production: data drift, concept drift, and the fact that most production work happens outside the model itself, in data collection, verification, and monitoring.
Chapters
Hi and welcome to Machine learning Engineering for Production. A lot of learners have asked me, hey Andrew, I've learned to train a machine learning model. Now what do I do? Machine learning models are great, but unless you know how to put them into production, it's hard to get them to create the maximum amount of possible value. Or for those of you that may be looking for a position in machine learning, many interviewers will ask, have you ever deployed a machine learning algorithm in production? In this four course specialization, the first course taught by me, the second, third and fourth courses taught by Robert Crow, who's an expert at this from Google, we hope to share with you the practical hands on skills and techniques you need to not just build a machine learning model, but also to put them into production. And so by the end of this first course and by the end of this specialization, I hope you have a good sense of the entire lifecycle of machine learning project for from training a model to put into production and really how to manage the entire machine learning project. Let's jump in. Let's start with an example. Let's say you're using computer vision to inspect phones coming off a manufacturing line to see if there are defects on them. So this phone shown on the left doesn't have any scratches on it, but if there was a scratch or crack or something, a computer vision algorithm would hopefully be able to find this type of scratch or defect and maybe put a bounding box around it as part of quality control. If you get a data set of scratch phones, you can train a computer vision algorithm, maybe a neural network to detect these types of defects. But what do you now need to do in order to put this into production deployment? This would be an example of how you could deploy a system like this. You might have an edge device. By edge device, I mean a device that is living inside the factory that is manufacturing these smartphones. And that edge device would have a piece of inspection software whose job it is to take a picture of the phone, see if there's a scratch, and then make a decision on whether this phone is acceptable or not. This is actually commonly done in factories. This is called automated visual defect inspection. What the inspection software does is it will control a camera that will take a picture of the smartphone as it rolls off the manufacturing line. And it then has to make an API call to pass this picture to a prediction server. And the job of the prediction server is to accept these API calls, you know, receive an image, make a decision as to whether or not this phone is defective, and return its Prediction and then the inspection software can make the appropriate control decision whether to let this phone move on in the manufacturing line or whether to shove it to a side because you know was defective and not acceptable. So after you have trained a learning algorithm, maybe trained a neural network to take as input X pictures of phones and map them to Y predictions about whether the phone is defective or not, you still have to take this machine learning model, put it in a prediction server, set up API interfaces, and really write all of the rest of the software in order to deploy this learning algorithm into production. This prediction server is sometimes in the cloud, and sometimes the prediction server is actually at the edge as well. In fact, in manufacturing we use edge deployments a lot because you can't have your factory go down every time your Internet access goes down. But cloud deployments, where the prediction server is a server in the cloud, is also used for many applications. Let's say you write all this software. What could possibly go wrong? It turns out that just because you've trained a learning algorithm that does well on your test set, which is to be celebrated, it's great when you do well when you hold a test set. Unfortunately, reaching that milestone doesn't mean you're done. There can still be quite a lot of work and challenges ahead to get a valuable production deployment running. For example, let's say your training set has images that look like this. There's a good phone on the left, the one in the middle has a big scratch across it. And you've trained your learning algorithm to recognize that phones like this on the left are okay, meaning there are no defects and maybe draw bounding boxes around scratches or other defects it finds in films. When you deploy it in the factory, you may find that the real life production deployment gives you back images like this, much darker ones because the lighting factory, because the lighting conditions in the factory have changed for some reason compared to the time when the training set was collected. This problem is sometimes called concept drift or data drift. You learn more about these terms later in this week. But this is one example of the many practical problems that we as machine learning engineers should step up to solve if we want to make sure that we don't just do well on the holdout test set, but that our systems actually create value in a practical production deployment environment. I've worked on quite a few projects where my machine learning team and I would successfully nail a proof of concept. And by that I mean we train a model in jupyter notebook and it will work great and we will celebrate that you know, you should celebrate it when you have a learning algorithm work well in a jupyter notebook or in a development environment. But it turns out that sometimes I've seen many projects where that success, which is a great success to the practical deployment, is still maybe another six months of work. And this is just one of many of the practical things that a machine learning team has to watch out for and handle in order to actually deploy these systems. Some machine learning engineers will say it's not a machine learning problem to address these problems. You know, if the data set changes, some machine learning engineers think, well, is that a machine learning problem? My point of view is that our job is to make these things work. Um, and so if the data set has changed is I think of it as my responsibility when I work on a project to step in and do what I can to, to adjust the data distribution as it is rather than as I wish it is. So this specialization will teach you about a lot of these important practical things for building machine learning systems that work not just in the lab, not just in the jupyter notebook, but in a production deployment environment. A second challenge of deploying machine learning models in production is that it takes a lot more than machine learning code. Over the last decade, there's been a lot of attention on machine learning models. So your neural network or other algorithm that learns a function mapping from some input to some output. And there's been amazing progress in machine learning models. But it turns out that if you look at a machine learning system in production, if this little orange rectangle represents the machine learning code, the machine learning model code, then this is all the code you need for the entire machine learning project. I feel like for many machine learning projects, maybe only 5 to 10%, maybe even less of the code, it's machine learning code. And I think this is one of the reasons why when you have a proof of concept model working, maybe in jupyter notebook, it can still be a lot of work to go from that initial proof of concept to, to the production deployment. So sometimes people refer to the POC or the proof of concept to production gap. And a lot of that gap is sometimes just the sheer amount of work it is to also write all of this code out here beyond the initial machine learning model code. So what is all this other stuff? This is a diagram that I've adapted from a paper by D. Scully and others. Beyond the machine learning code, there are also many components, special components for managing the data, such as data collection, data verification, feature extraction, and after you are serving it how to monitor the system, monitor the data comes back, help you analyze it. But there are often many other components that need to be built to enable a working production deployment. So in this course you learn what are all of these other pieces of software needed for a valuable production deployment. But rather than looking at all of these complex pieces, one of the most useful frameworks I found for organizing the workflow of a machine learning project is to systematically plan out the lifecycle of a machine learning project. So let's go to the next video to dive in to what is the full life cycle of a machine learning project. And I hope this framework will be very useful for all of your machine learning projects that you plan to deploy in the future. Let's go to the next video.
