A Kubernetes Primer For Data Scientists, Part 1
Hamel Husain deploys JupyterHub on Google Kubernetes Engine to teach data scientists core Kubernetes concepts. He walks through creating a cluster, using kubectl to manage pods and deployments, installing JupyterHub with Helm, and debugging by viewing logs and accessing containers directly.
Chapters
Today I'm going to show you how to install JupyterHub on Kubernetes, but with a twist. I'm going to— my goal is to show you a little bit about Kubernetes and to learn Kubernetes along the way. And so no Kubernetes experience is required or assumed. And the reason for doing this is because I think Kubernetes can be a really dry topic, and just reading the docs reads kind of like a dictionary. So you need to do things, like real practical things that you might care about, to actually learn it or to motivate something. And one tool that is really popular amongst data scientists is Jupyter Notebooks. And so, um, by showing you how to deploy JupyterHub on Kubernetes, hopefully you'll learn a little bit of Kubernetes along the way. And I'm going to gear this towards data scientists. So I'm not going to be talking about a lot of Kubernetes administration stuff. I'm going to be talking about the stuff that might be interesting to you as a data scientist in terms of debugging, troubleshooting stuff, just understanding what things are. So this is not a foreign language. And I don't want to— you might be wondering, well, why should I even bother learning Kubernetes? Are, what's the point? I'm a data scientist. I talk about that in another video. I'll link to that video or blog post, and I may not have a video for that, but there'll at least be a blog post in the description about why you should learn Kubernetes. And the way you should watch this video is like, you don't have to follow along at all in terms of writing code. I would suggest you just watch me on the screen and sort of listen to what I'm saying and look at what I'm doing and do it by yourself after I'm done. And the reason for that is the documentation is really good. If you follow this documentation step by step, you'll be able to do the exact same thing I'm doing. What I'm adding to it is a lot of commentary and and a lot of explanations about what things are. And so JupyterHub is a shared Jupyter server that you can use to collaborate with your coworkers, that you can use as like a hosted Jupyter Notebook service. And that's pretty useful. You know, let's say you work at a company and you want some common infrastructure for everybody to use. You know, you may want a VM with the right resources, GPUs. You want something that's connected to all your data infrastructure, databases, and other things. And having a hosted notebook that is— that can access everything in your corporate environment that you can hack on is— it can be very useful. And so how do you install this? So just like a lot of data science tools, you, it, you need to deploy it somehow, usually on the cloud. And a lot of times nowadays, the paved path for deploying these tools are to deploy them on Kubernetes. So let's go ahead and click on that. Okay, so this is the documentation. Okay, so first things first, let's set up Kubernetes on the cloud. And I recommend using GKE. The reason is because GKE is frankly the easiest thing to use. You can— it has this really cool thing about having a terminal in your browser window, which I think is super handy. Go to— go and enable the Kubernetes Engine API. I've already done that. Choose a terminal. I can open one of these things, activate Cloud Shell, which is really cool. Great, now I'm in my Cloud Shell. It's gonna tell me I need to install kubectl in the Cloud Shell. Okay, it's telling me that I need to install kubectl. kubectl is sort of the way you interact with Kubernetes. That's the client, that's the tool that allows you to do things in Kubernetes. Let me just get rid of this stuff. and then it says create a managed Kubernetes cluster, and I'm— it's going to create a, a small Kubernetes cluster with 2 nodes, and it's telling me to use this command. So let me just copy and paste this here. Let me make this bigger. There we go, that looks better. Okay, I'm going to get the zone. I'll try this, West A, since that has Oregon. And I'll call the cluster name k8s-data-science. And looks like we filled everything out. So I'm gonna go ahead and hit enter. Wants me to authorize the shell, and it says it's creating the cluster. Okay, so it takes a while for it to get started. It took a while for me, and what you want to do is just take a look to make sure it makes sense. Here, you know, it'll give you the IP, it'll give you the machine type, the number of nodes. Remember, we started a 2-node cluster. But all of that is not terribly important. What's interesting, it starts to get interesting now. So it's telling you to run a command kubectl get node. And kubectl allows you to see things, different entities in your cluster. So we're going to say get node. kubectl get node will show you different nodes in your cluster. You can ignore all of this stuff up here, that doesn't matter. You just pay attention to this stuff that I'm highlighting on the screen. And this shows you basically two nodes. And there's nodes, there's also something called pods. So pods, pods are kind of things that run containers. So pods can have more than one container, usually a Docker container. So just think of Docker containers. And pods are things that run containers. It's, it's kind of the, you know, you package up your application, you put it in a pod. And so if we do kubectl get pod, um, it'll show no resources found. That's because we haven't launched anything yet. This is a kind of a, a blank Kubernetes cluster where there's no pods running, at least in this namespace. Now Kubernetes uses pods to run itself, And, um, you know, has— and you can see that if you do all namespaces, you can sort of see here. Let me just zoom out a little bit. You can see here that these are all the different pods that are running various applications. And so you can see things like there's— these are all internal Kubernetes things. These are all running in the kube-system namespace. And namespace is just kind of a logical separation between concerns in Kubernetes where you can have different applications for different things, different sets of services, different applications, you know, that can be segmented from each other for both security and both for organization. But, you know, you have things like logging. This Fluent Bit is logging. Pod that helps, you know, this is very Google Cloud specific. So if you did this on your computer, if you did, if you use something called Minikube and you did it locally, you wouldn't find as many, you wouldn't find as many of these pods because Google Cloud gives you a bunch of extra stuff. The next thing it wants you to do is create something called a role binding, cluster role binding. It says give your accounts permissions to perform all administrative actions needed. So you might be wondering what the hell is the cluster role binding? If you didn't know any better, you would just copy and you'd be like, whatever, I'm just gonna copy and paste this, like who cares? But the point of this video is to like explain like all the things to you. And so cluster role binding, so there's things called cluster roles, okay? Let me show you about cluster roles. Cluster roles. So if I go here to kubectl get cluster role, you will see all of these different cluster roles. And these are basically permissions. These are default, like, permission types. And there, there should be one called cluster admin. Yeah, so it says, see this --cluster-role cluster-admin? So it's saying it wants you to— so it wants you to create a cluster role binding. And what a cluster role binding is, is just saying, hey, I have this new user, I want you to grant it permissions of a certain cluster role. And in this case, we have a cluster role called cluster-admin. And cluster admin just basically allows you to do anything. And so we were saying, hey, when creating a user, this new user has this, you know, you just have your Google email account for the GKE, and then just bind it to this cluster admin thing, which is here. Now you can look into this. If you, you could say get cluster role, you can For anything, any kind of entity with kubectl, you can put a forward slash and you can kind of get that thing. So I can, you know, see, see just that if I want to. You can, you can also say you want to get that thing and do -o yaml. And that will describe sort of a little bit more about it. But this is this star, this is saying give me access to everything. So first of all, don't get intimidated by YAML. If you don't understand the YAML, that's totally fine. You don't have to understand the YAML at all. A lot of this stuff is boilerplate. You don't have to understand everything. The point is just to kind of get the general gist. Because a lot of this YAML is copy and paste. If you're going to, if you do create any of this, you copy and paste it from somewhere and I'll show you where to copy and paste it or how to go about it. Anyways, you— so that's what this does. So let's do it. Okay, let's bind this new user that we're creating. In this case, it's me. So I'm gonna say, that's my email address. And I'm going to create a role binding for myself. So great, it got created. And if you want to check it, you can check it. You can say kubectl get cluster role binding. It's not case sensitive. And you can see all the cluster role bindings in here. And at some point, you know, you would see, you would likely see one for me. It's definitely this one. So it's this cluster admin binding. I didn't really give it a name and just let it go with a generic name here, which is fine. But you can tell this one because it's only been alive for— it's only 15 seconds old. And we can go and check this a little bit more. So you could say, okay, And, you know, we can— I can do the --show-yaml. You can see my email address. And another— sometimes you can also do, instead of doing --show-yaml, you can also use describe. And that will show you some other things. They'll show you kind of like a different view of things. And that can also be helpful. Okay, so node pools. This is an optional step. So what is a node pool? So sometimes it's useful to actually look at the documentation. I've opened it here. This is one, um, such documentation about node pools. But basically it's a group of nodes that have the same configuration. What does that mean for you as a data scientist? Just think about different nodes with different hardware, like maybe you have a group of nodes with GPUs, or group of nodes that are high memory, high CPU, and you might want to run different workflows on different nodes. So node pools is basically a way of kind of like tagging those nodes. So you can do, you can like perform operations on those nodes as a group. So that's node pool. We're not going to do that. It's optional. And now we can do this other thing is set up Helm. So what is Helm? Helm is basically a package manager for Kubernetes. Like, if you— it's basically like pip for Kubernetes. So some— sometimes you have really complicated applications or things that necessarily are not just one container. There are many different services with networking, um, maybe different kinds of things that talk to each other. Like JupyterHub will have some kind of master, like node or web service, and it'll have these Jupyter kernels. And anytime a user logs in, it might spin up another session and another container just for that user. And so you don't want to have to configure all of those things. And, you know, it can be tedious. So Helm is like a really nice way to allow you to install these complicated apps in one command. With, you know, and it gives you parameters. So it gives you little knobs that you can kind of tune to like customize the deployment. And Helm is really cool. It's like one of the coolest things about Kubernetes that make it really approachable, honestly. It's saying to first install Helm. So let's do that. Okay, installing Helm. And let's set up JupyterHub. To get— so let's look at what, what is Helm? Like it says, like initialize a Helm chart. Now, what is a Helm chart? Like, where did that come from? This is the GitHub repo where you can find the Helm chart for this project. And we don't have to get really deep into it. So I just want to give you a rough idea of what Helm chart is. Don't worry about schema.yaml. That's, that's a, like an optional thing for Helm. And we don't— just so don't worry about it. The, the most important file in this whole thing is values.yaml. The values.yaml are basically the parameters or the options that you can set, and they all have defaults, and you can override them either here in the YAML file or on the command line. When you actually go to install JupyterHub, you can pass some one of these arguments on the command line with a key-value pair, and it'll override whatever's in the value in this YAML file, or you can just edit the YAML file itself, whatever you want to do. So there's this values.yaml, and then also you have chart.yaml. Chart.yaml just has like metadata in it, tells you the version, what's the name of this Helm chart, so on and so forth. Every Helm chart will have this. Usually you don't really end up messing with this, but this is just something that's in every Helm chart. And so the way you go about having Helm charts or dealing with them is you first have to add the Helm chart. You have a, you know, you have a Helm repo or a set of Helm charts. And so you have to add that. So You add the repo and you update it. This is kind of like Linux. It's very reminiscent of Linux package management is that you add the repo and then you update all of the, you know, all of your Helm repos to pull all of the latest charts down so that you can then decide to install whatever one you want. Okay, so So we added the Jupyter Helm chart here. I'm going to clear the screen to give us more space. And we can see what Helm charts we have actually by doing some— we can do helm available. So a good thing to do is you can always use your— use the help. And you can see that, okay, you can do search. There's a way to search. So we can do search. So helm search. And helm search allows you to say you want to search for a particular repo. So we do helm search repo jupyterhub, for example. If you can see, these are all the value— these are all the different versions of JupyterHub that are available to install, or versions of the Helm chart. So what we can do is now we can install the chart. Now this says, now install the chart configured by your config.yaml. So now you know what that means, because you know how, what config.yaml does, and that it is the parameters for the templating. Okay, so we want to install this Helm chart. There's a Helm install command, but these folks are using the Helm upgrade command, which also works. That's also fine. You just have to, like, understand what it's doing. So I'm just— first, actually, I'm going to say Helm search repo JupyterHub. Just to see what we're dealing with here. So we have this, we have this one, and I'm going to copy and paste this command. I don't want this values, uh, I don't need this values flag with the config.yaml because I don't want to change anything. I'll just keep the config.yaml how it is. And the chart version, I'm going to do 2.0, the one that's here. It's the only one I have really. And I do— okay, it's gonna ask me the namespace. So I'm gonna name the namespace jupyterhub. Remember I showed you namespaces before? There's a namespace with everything and that has Kubernetes, like the kube-system namespace. But I'm creating— I'm gonna create a new one just for this application. So the application gets put in its own namespace called jupyterhub. And And since it doesn't exist, you have to do this create namespace flag. And then saying install, and it wants me to put a release name. This is something I can name it. So I'm going to also be— I'm just going to call it JupyterHub here. And it's going to install this application. Now it's saying JupyterHub doesn't exist. That's because I— this is arbitrary. I named it. You know, it's— you can see all of the, the Helm charts that you have installed by using helm list. But we haven't installed anything. So right now it's going to deploy JupyterHub. Okay, so this is great. So it's deploying JupyterHub. Now it's telling you to do this post-installation checklist, and that's kind of useful because you can learn a bit about Kubernetes if you do this. So kubectl namespace jupyterhub get pod. Okay, so another way to do that is you can say kubectl get pod. You can also do get po. There's, there's abbreviation for everything in Kubernetes, even this silly one that is only abbreviated by one letter. That's fine. And -n is the same thing as --namespace. And we can say jupyterhub. You can see, okay, these have just been created. These are different sort of pods running different containers here. And so that's interesting. There's also something called deployments. And deployments are something that run pods. Now you might be confused by that. You might have said like, But you just told me pods run containers. What the hell are deployments doing? So deployments are something called a controller. And it's something that watches pods. And if a pod fails, for any reason, it crashes, it'll restart the pod, and it'll bring it— it'll try to bring it back. And so kind of deployments are the thing that tries, you know, tries to make pods resilient. By watching them and restarting them or rebooting them if they, if it needs to, or replacing them. So that's enough about pods. Now, now these things are running here. And it's saying, so you can look at the namespace. You can, you can look at a particular pod, we can do a describe on a pod, but we don't really need to do that. I don't think that'll be terribly interesting. You can look at the logs on a pod if you want to. So you can even— yeah, you can go and say kubectl logs, and you can get logs for a pod. Let's say this one. This one might be interesting, has the word hub on it. Um, let's see. And I have to do namespace. So you don't always have to do namespace. Um, you can set JupyterHub as your default namespace if you want to, but you don't have to. I'm not going to do that because I don't want to bore you with those details. But this— I'm getting logs from that particular pod, and po-name of the pod is the way you do that. There's one way to do that. Okay, that's interesting. Now let's, let's see what else they told us to do here. You can verify an external IP is provided by getting the service. Okay, so we covered pods, we covered namespaces, I mentioned deployments. I'll show you what deployment is again if you would like. So let's kubectl get deploy. These are the deployments. Deployments is like probably the best way to understand the different quote applications running in your cluster because you usually won't have just a pod running without a deployment because remember deployments are managing pods. So um, you know, the deployment level is a kind of a good one to look at. So that's, uh, that's deployments. So we covered pods, deployments, namespaces. You already learned 3 things in Kubernetes. That's really good. So now, now it's saying service. Like, if memory here says verify an external IP is provided for the K8s service. What is service? Service is networking. So in Kubernetes, services provides networking, networking internally inside the cluster, externally to the outside world. With services, you can do things like load balancers and all kinds of stuff. So services are kind of, are pretty important in Kubernetes, and that's the way you can look up like how to reach in application. So it's saying to do kubectl get service. Okay, let's actually see. So kubectl get service, and I don't have to do service, that's the abbreviation for service, get svc. And I can see all the services in JupyterHub. And it'll tell me I have 3 services. Now say there's 3 types. One is are— there's two types. One is ClusterIP. That, that just means that's a service that's internal to the cluster. Like, you can't, you can't really get to it from the outside. It's just internal networking. It's like an internal DNS, part of the internal DNS system. But then there's a load balancer, and the load balancer allows you to get to this from the outside. You notice like these cluster IPs don't have external IPs. They have a cluster IP, allow things in the cluster to talk to them, uh, but no external IP. This load balancer has an external IP and it's an external IP serving on port 80. That's interesting. So if we come here and let's try selecting this, And let's put it in here just for fun in the browser. And this is not, uh, this is not HTTPS. This is not HTTPS. Um, you know, we're not, uh, we don't have that set up. So we can just do continue to site. And you'll see that we, we are now, this is, uh, the JupyterHub. Okay, and in their, in their documentation, so we did the install, you can read this on your own, I want to read it to you. It's not nothing too important, we already did some of this, this, they tell you, you can set your default namespace to JupyterHub so that you don't have to constantly type in -n jupyterhub. We can do that if we want to. Why not? Let's do it. Because even I'm getting tired of, of typing that in constantly. Okay. And then so now, if I do get pod, you'll see stuff for Jupyter in it, you can do -o wide. It'll give you a little bit more information. For example. So, so now JupyterHub is the default namespace, and you'll see all the pods in that. And it says entering any username and password will let you into the hub. So if I just do foo and foo, it will work. And this is interesting, it says it's automatically spawning something. Let's see. If we can see anything that is spawning here. Look, 11 seconds, you have jupiter-foo, and that has spawned. This is a new pod that has started that has my Jupyter session running in it. That's really interesting. So this is really, that's really cool. Like you can kind of see like if, if you're a data scientist and you're using an application like Jupyter Hub or any other data science application, you can see the logs of what is going on. In fact, let's do that. So let's come here and say, uh, let's do kubectl logs and this jupyterfoo thing. And you can see the logs of JupyterLab. So something happens, something crashes, you can see that. You're not beholden to some infrastructure guy. You don't have to, you don't have to send like an annoying message to an infrastructure guy, say, hey, it's not working. What you can do now is you can kind of poke around, look at the logs, and sometimes you can figure out what happened. And you can have like a much better conversation with your infrastructure folks and say, Hey, like I see this such and such error is happening. Or sometimes you might be able to figure it out. Just like when you know how to see logs and things is super helpful. And beyond that, so let me actually clear the screen here. And okay, so another thing I want to show you is how to, how to actually get into one of these pods. Let's say you want to do some more debugging, right? Like this JupyterFu thing where your JupyterFu, where our JupyterFu thing is running right now. Let's say you wanted to do, you want to actually like get in there and run some commands into that container. So you could do kubectl -it exec. And you could say, bash. Okay, and now I'm in the Jupyter— I'm in that container. And now I can see what's in there. For example, if I can go in here, and if I, if I created a file, let's say hello.markdown. And you would see that these files are there. So this is very useful because now it empowers you. You can do interest, you know, you can— this is very helpful in my experience being a data scientist. If you kind of understand what's going on, you understand what a pod is, you kind of understand where to look. You might not know what namespace something is in. That's okay because you can do kubectl get namespace, and you can see a list of all the namespaces. You can kind of get an idea. Usually things— people name things in a sane way. You can kind of like figure out where the application is. You can figure out where the IP is, um, you know, you can figure out the logs, or you can get the logs. You can get into the container, you can run commands. All this stuff is super useful, allow you to be way more empowered to do things. And I think that's great. So you, so you have learned a lot about Kubernetes, like a lot of practical things about Kubernetes. So you have learned about namespaces, pods, deployments, services. You have also learned about Helm charts. And you have learned how to look at logs how to, you know, how you might SSH into a container or a pod, and a lot of things that I think are very practically useful. So by deploying JupyterHub, we've kind of run into a lot of stuff. Now, this is not everything Kubernetes, like we haven't gone over some important things like mounting storage or secrets, things like that, but that's okay. You can't learn necessarily everything at once. You— we— to do some of these more advanced configurations, you would have to get into those. But I just want to convince you that it is approachable to you. Like, you can, with some knowledge of Kubernetes, you can do things that are going to be really useful to you as a data scientist. I hope that helps. Thanks.
