Beyond Jupyter Notebooks: MLOps Environment Setup & First Deployment

Hi everyone and welcome. This is beyond Jupyter notebooks, MLOps environment setup and first deployment. My name is Ryan Keenan and I'm leading the product team at DeepLearning AI. We appreciate you taking the time to join us for this event. I can see that we have people joining us from all over the world, so I'll say good morning if you're in a time zone near me, I'm in California right now. Or good afternoon, good evening, maybe even good middle of the night to you. If you're joining us from the other side of the world, wherever you are, we're glad to have you with us. This event is a workshop that focuses on setting up your computer to build and deploy industry standard machine learning applications so that you're ready to maximize your impact during your first week of the job. If you have questions during the workshop today, we'd like to ask that you follow the slido link in the description box below on YouTube, and even if you don't have a question, you can click that link to see the questions that others are asking and upvote your favorites. Our speaker will do his best to answer all your questions during the Q and A portion of the event. And so, without further ado, I'd like to present Dr. Greg Lochnane, head of product and curriculum at Fourth Brain, where he leads their portfolio of machine learning, engineering and MLOps courses. Over the past decade, Greg has worked as a product manager, a data science consultant, and machine learning researcher, but his real passion has always been in teaching. He's fascinated by where the workforce of the future will go and what it'll look like, and enjoys coaching young professionals on how to future proof their careers against 21st technologies like AI and 3D printing. And so with that, I'll hand it over to you, Greg.

Hey, thanks so much, Ryan, for the great introduction. I'm really excited to be here and to talk today about how and why it's so important for you to get your MLOps development environment set up so that you can accomplish your first deployment. All right, what are we talking about today? Well, we're going to put MLOps in context first, and then we're going to dive into a few of the best practice MLE development tools out there on the market today that you need to know about. I'll talk a little bit in that section about the typical data science interactive development environment learning journey that may be similar to what you've gone through, and we'll talk a few preliminaries before we get into the Live demo. We'll come back for conclusions at the very end. All right, so first let's talk context and put MLOps into some have you seen this meme? Hey, Technical debt. Why do you always wear that mask? Well, if we zoom in on this and we get to know MLOps a little bit better, this is actually pretty funny. And this is a very, very famous picture here behind the mask of mlops. In fact, this is from a very famous paper called Hidden Technical Debt in Machine Learning Systems written by a famous guy, David Scully and many others at Google, who is well known in the data centric AI and MLOps space. The purpose of this figure is to show that ML code is really a small component of, of an ML system overall. And this is something that more and more people are starting to really open their eyes to as businesses try to create value with AI systems across the world. Now, interestingly, Jupyter notebooks have been a really important tool for data science. And Jupyter notebooks have also been sort of growing in scope and scale and their ability to include some of these other aspects of beyond data and model within the Jupyter notebook. So, you know, we're getting questions coming into this event. You know, how can we keep up with new tools in the market? What about these low code, no code tools? Yeah, you really do need to watch out for those. One click deployments from Jupyter notebooks, you know, one line deployments from Jupyter notebooks. These things are real, but they're not industry standards yet. And so to get out of the notebook in 2022, to get that model deployed, to get your serving infrastructure set up, you still need to get out of the notebook and today we're going to talk about how to do that. Now historically, non AI digital product systems are, have primarily been built by software engineers and then maintained by DevOps engineers. You know, it's important today as an MLE to really develop a skill set that exists somewhere between creation and infrastructure. And that is actually exactly why the MLOps specialization from deep Learning was subtitled Machine Learning Engineering for Production. Julian Simon, the chief evangelist of Hugging Face, told me and our audience at 4th Brain in a recent event that you really want to be your own DevOps engineer in today's world. So concretely, in order to get started with this event, we, we really want to understand, yes, what's inside of that ML code and what's going on with the data. But it's infinitely, potentially more important today for you to be the one on your team that Helps data science get out of the notebook. All right, so that's about it for MLOps context. Here we're asked, you know, what are the recommendations for moving from notebook based delivery to production? And that's where best practice MLE development tools come in. Now, in short, what is the best practice MLE development tool? What we're seeing in the market is very short, simple answer. It's VS code. Is this part of the MLOps course at 4th Brain? Yes, it's part of all of our courses and it's part of all Silicon Valley startups that I've seen. But let's back up, let's talk a little bit about the data science learning, learning journey that you may have gone through. And that is pretty typical of many learners out there. So consider Sarah. She's a data scientist and former researcher. She graduated a few years ago and she got a job as a data scientist. She made that transition that so many of us tried to make a few years back. Now, despite trying out a few different ides along the way, like Spider and pycharm, she really kind of settled in to just mostly leveraging Jupyter notebooks. And in fact, when she goes to work now, she logs into aws, but she logs into AWS only to open up a Jupyter notebook instance and they get right back to work exploring data and developing insights. And she's not really seeing the fruits of her labor over the past few years translate to real business value or translate to real impact. So she started to feel like she wasn't really growing enough. She went out, she looked for a new job for the past few months and she finally landed one. So she's going to move from a bigger company to a smaller Silicon Valley startup as a machine learning engineer. She just got her new Apple MacBook in the mail and it's pretty cool. She's excited about it. She, she was an Apple user before, but this is obviously the new one and she wants to set up herself for success. She knows she's got a lot to learn and she wants to try to make the most out of this machine learning engineer job. So she's done some research, she's not really sure which IDE she should get set up with. And this is where her story begins. What tools does she need to succeed? What are some of the best practices when setting up an ML environment at work? Well, first let's talk a little bit about what needs to go into an ide. Let's break it down into its constituent parts. First and foremost, what is the hardware and software that you're using. A lot of us out there are using Mac, certainly in the fourth brain audience. But you know, many of us are using Windows, many of us are using Linux. It's very, very, very important to understand this first and foremost because this is going to sort of drive many other things down the line today. And Sarah and are both going to be on, you know, I'm going to be on a Mac M1 chip. You may have heard of these fresh out of the oven. Not so fresh. There's M2 now already. But these things are new and they're pretty cool. And one of the really cool things about them is that they are actually GPUs and CPUs in one. Now this is cool for some and in other circles, not so cool because it kind of creates a lot of problems using Apple Silicon if you've been a developer on intel and more classic GPU setups for quite some time already. So once you get it figured out what your hardware and software is, you really need to dig into which tools are you going to pick up and then it's the implementation of these tools that is going to be affected by that hardware and software baseline. So the tools that we're going to go through today, number one is version control. So number two, number two is your command line interface package and environment management is number three. And then we're going to look at web app frameworks and containerization. So first version control. Now it's important to try to understand, you know, what the core concept of each of these constituent elements of your development environment are. And the core concept of version control is git. And you know, GitLab, GitHub, BitBucket, AWS, CodeEdit. What is the core aspect of git? And the core aspect is that everything, excuse me, everything is local. So what you want to understand about excuse me, what you want to understand about Git is that you actually pull everything down from a remote repository so that you're developing locally. Git ops is a new big thing. And you know, we recently did an event with Iterative AI. They're a leader in GitOps for MLOps today. Really encourage you to check that out if you're interested in learning more about GitOps. So we will look at Git today. And the command line interface is the second tool that comes up. And the command line interface is essentially your computer's text interface. It's not a gui. And have you ever seen people, command line ninjas out there running a bunch of command Line codes one after another. I'm going to try not to do that today and I'm going to try to make sure that we keep it pretty tame. However, we are going to move a little bit quickly in the demo, so I'm going to do my best. So we use the UNIX command line at 4th Brain, and it's an industry standard. The UNIX command line is important because, number one, it gives us a shell. And the shell is simply a computer program that exposes the operating system to a human like you or I, or to another program. It can be a command line or it can be a gui, but the shell is the important concept here. And Unix, like operating systems like Mac and Linux exist, but then we also have Windows and you may have heard of Windows subsystem for Linux. It's a way to sort of emulate Unix like operating systems and to be able to use the same syntax. And depending on your view of Windows and wsl, you may view it as a bit of a superpower in some cases and you may view it as a bit of a. Well, less than a superpower in others. So it's, it's a very powerful tool and it's a way to align your syntax and your development with all the other developers on the planet. So it's very, it's up and coming and it's growing in usage. All right, so package and environment management is next. And what. The tool that we pick up in data science generally and that we recommend at 4th brain is conda. So conda is a package management and environment management system. And what that means is it means it helps you manage your libraries, pandas, numpy, etc. It also helps you manage your environments. That means where you're actually developing your Python code. For today's demo, I'm going to pick up miniconda because as it turns out, miniconda is the one that we really do need to use for the Apple M1 new chip. And finally we're going to look at a web application framework again. We're going to look at the most important one, most used for data science. And last year it was kind of interesting when we did an event with Deep Learning. We did this question and Flask was in second to, I have no idea which web framework to pick up. When we did a poll this time around, we said, hey, which application framework would you choose? And Fastapi won with 4th Brain's audience, Fastapi One with Deep Learning's audience. And we even got Sebastian Montano to comment, which Was pretty cool. So shout out to Sebastian, creator of Fast API. So Fast API is fast and it is up and coming and it's one that you should know about and that we'll be diving into just a little bit, scratching the surface of today. What about containers? Well, the answer for containers today is Docker Enough said about that. So putting things together, VS code is the chosen one because you can put everything in it all at once. And this is what we'll see in our demo. All right, so let's get to it. Let's go ahead and get started by we're going to share a link in the chat, which is for 4th brains software development for MLOps 101. And what this is is it's a tutorial to get you set up no matter what operating system and hardware that you use. This is currently supports Windows. I think this is written for Windows 10, Linux, Debian and Ubuntu. And this Mac OS setup is actually for Intel. But what I'm going to walk you through today, I'm going to give you some hints on how to get the M1 set up. And we do have some plans to release some longer form videos that sort of start from scratch and in the coming weeks for all of you that have registered for the event. So we're excited to send those out to you very soon. All right, so we're going to get started with Mac os. And the first thing that we see is we want to open the terminal on Mac. So on Mac we can, you know, we can search for the terminal or we can simply go find it somewhere. And the terminal is a place going to blow up the text on the terminal. Terminal is the place where we can interface with the operating system. First thing is we want to make sure Homebrew is installed. Homebrew is essentially PIP for Mac. It's a package manager. And we've already obviously installed Homebrew to prepare for this event. So we just want to make sure that everything's up to date. So we're going to go ahead and do brew update and we are already up to date. We've installed wget. So if we needed to take the and go get the Anaconda package, we could do that for the Mac M1 chip. What we actually need to do is we need to download the Miniconda package for the new Mac OS. Apple M1 64 bit. That's what you go ahead and you just download. If you click it and you open it up, you run through the installation, you're good to Go Git is already ready to rock and roll for us. And for VS code you do a similar thing that I just did with Miniconda, which is you go to VS code, you find your appropriate setup. In this case it's Apple Silicon setup. Okay, so that kind of gets us started setting up our environment. Now the next thing that we want to do before we jump into VS code is we want to make sure that conda and that our environment that we're going to use for development is set up. If you're familiar with Jupyter notebooks, you probably already have your environment set up and you're probably comfortable with commands like CONDA create. But what I'm going to do now, I'm not going to go ahead and create an environment because we really don't have time to wait. In today's short session I'm going to just activate the environment that I've already set up for this event which is called Beyond Jupiter. And as you can see, the base environment is now showing as beyond Jupyter. So that means we are locked and loaded with our libraries and packages. Okay, so now onto VS code. Once you get it installed, then what's nice is the very first thing that, that we would recommend that you go ahead and do is you just start working in VS code. So forget the terminal because you can actually use the terminal in VS code. So we're going to open up VS code dark mode obviously and we're going to go ahead and not get started, but rather. We're going to open up the terminal. So. Let's see, Vs code. Is. Let me go ahead and. Not seeing the terminal here. So let's just make sure.

That.

We have the right VS code set up. We have Apple Silicon versus code. Everything's good to go here. Not seeing the property, not seeing the. There we go. That was weird. I'm not sure what was going on there. But when you open up VS code you should see things beyond file edit and selection. So you go over to terminal and you open new terminal and there's the terminal. So perhaps that was an M1 buggy issue. I'm not exactly sure what that was. But if we go ahead and do the same exact thing that we did in the terminal that was not in VS code, we can just do conda activate beyond Jupyter and you can see now we're in the Beyond Jupyter environment. And so straight away we are just now in VS code. So great. We are going to go ahead and take this next step which is to install the proper extensions to get us ready to rock and roll. Intellicode is the famous AI powered coding assistant. So make sure that's installed Jupyter obviously make sure that's installed and then the last piece, rather than another extension you actually want to set set the Python interpreter. And the interpreter in Python is simply a way to interact with the Python programming language. So you know, you can open a file or you can simply do shift command p on Mac and you can start typing, select interpreter. And what you're going to select is you're going to select exactly the conda environment that we installed earlier. So for me that's beyond Jupiter and I'm going to go ahead and select it here and I don't really need to worry about this issue down here. So now I'm ready to go. Now one of the things I I wanted to show is just to give you a feel for how this works in Jupyter Notebook. So first of all, let's go ahead and you know, save it as not Untitled 1, since let's say at least we're doing MLOps here. Let's do hello MLOps and let's go ahead and just see, make sure that we can rock this out, hello MLOps software dev. And you can see it's starting to do intelligent things. We just run this and there we are. So we can also sort of set this up and check it and make sure that it has numpy.