Deploying Python on AWS

Sam.

Hello everyone and thank you for joining today's session. My name is Recent, I'll be your moderator for today. We're going to get started at the top of the hour. We're just waiting so everyone has a chance to join. If you haven't already, make sure that you register for this session. You can do so. There's a Link in the YouTube description. If you're watching on YouTube. I would recommend watching on YouTube rather than LinkedIn, but feel free to watch from wherever you like. Yeah, there is a link to register for today's session. If you can't find the link, you can head over to deadcamp.com forward/webinars. I'll also update that the QR code that we've got on screen as well. So yeah, you can register for today's session. Also, if you've already registered, you will have noticed that We've got a GitHub repo that is the link to today's session. So that's what we're going to be working from. If you do plan on coding along with us live, you are going to need to check out the readme in the GitHub repo. So please do check out that there's a link to to that in the comments. I will post it again very shortly. So yeah, if you want to go along with us live, you're going to need an AWS account that's active. And yeah, you're going to need to check the readme for the setup instructions as well. So yeah, if you don't have that set up, I would consider this more of a watch along. Obviously you'll get the recording in your inbox if you've registered for the session and that'll come through tomorrow. So yeah, please keep your eyes peeled for that and make sure that you register for the session. Aside from that, we are going to be taking your questions. So if you have any questions at any point throughout the session, let us know in the chat. We're going to be running through your questions for the last 10 minutes of the session, so make sure that you stick around for that as well. I'm just going to repeat these messages for anyone that's just joined. So hello, if you've just joined, welcome to the session. My name is Rhys and I'll be your backend moderator for today. Please do register for today's session. That means we can send you the recording and the resources for right to your email inbox tomorrow as soon as it is up in our resource center. And yeah, if you want to go along with us today, you are going to need an active AWS account. There is a GitHub repo that we are working from. You can check out the GitHub repo via my comment in the chat. I'll be posting it again very shortly. There is also a QR code on screen as well. So yeah, check out the readme in there if you'd like to code along with us live. But also, yeah, do check out the resources that we've put together for you and all the extra links as well. If you have any questions at any point throughout the session today, let us know in the chat. We're going to be running through your questions for the last 10 minutes of the session. Brilliant. I think that's it from me. So now I'll hand you over to your host for today's session. Richie. Richie, please take it away.

Hi there, Data Scamps and Data Champs, this is Richie. Good to be back for streaming session. It's been a while since we've done one of these. All right, so kicking things off. If you are a data scientist or a machine learning scientist, then once you've done your analysis or you've created a model, that's often not the end of your work. So these days your work often needs to be incorporated into other software or perhaps otherwise placed in a production environment in the cloud. So that's what we're going to learn about today. It's about how to deploy code into Amazon Web Services. And this is one of the most in demand skills for data professionals. So very exciting to learn the techniques. Now that's a whole career in itself. There's lots to learn about. So today we're going to focus on a sort of more narrow slice of that which is going to be around using AWS Lambda, which is the serverless computing service. Say that 10 times fast. Service computing is a bit of a misnomer. There are services involved, you don't need to care about them. I will let today's guest explain more about that. The other thing we're going to be looking at is giving G Radio, which is a streamlit fork. So this is basically a machine learning app development platform. So you get a user interface to your models. Our guest is Eric Riddock. He is the director of Machine Learning Platform at the E commerce platform Pattern. Hi Eric, great to have you here.

It's good to be here. Thanks for having me.

Wonderful. So Eric runs the machine learning engineering team and his focus is on building tools to allow data science teams to self deploy data and machine learning products. He's got decades of experience both as a data engineer and a machine learning ops engineer. And he also teaches MLOps at MLOps club. Wonderful. So with that, please take it away. Eric.

Yeah, thanks. I'm sharing my screen. I have a slide up I realized I should probably introduce. My plan is to motivate how I got started with aws because it'll. I'm hoping to get you excited about what it is we're building. And so this is why it would have been exciting to me early on. So anyway, so my name is Eric Reddock, like Richie said. And I went to school at Brigham Young University and I studied math and I really struggled to get a job doing anything after studying math. And so like, I think many of us have found ourselves in the position where you have to go out, go beyond what school is asking you to do in order to be enticing to someone who would hire you. And so I finally did get a job as an analyst at a company called Inside Sales. And I, coming from math, I thought that was so beneath me. But you know what, that, that's where I learned that the majority of value in data comes from basic counting. Like how many things did we sell or how long has it been since we reached out to customer whatever. So it was a healthy experience. And then I moved to working at MLOps at a company called Ben Labs. And if you've seen into the spider verse, there's all these different spider men. And one of them has this VR headset and, and you can see she's holding a bag of Fritos. And we actually are the ones who placed that product that we did that product placement into the spider verse. And we're also the reason that Bumblebee is the Transformer, is a Chevy Camaro. And we're the reason Tom Cruise wears Ray Ban sunglasses and maybe a whole bunch of other brands. And we also had a Chrome extension to help YouTubers succeed on YouTube. And one of the things, one of the features I worked on was like this regression model where we'd take a YouTube thumbnail and we'd guess what the click through rate would be. And so you could plug in all these variants of a YouTube thumbnail and we'd try to help you, like, rank them in order for like most clicked to least clicked, you know, guessing. So we created an ML platform and we went and presented at Netflix about how we built it. That was great that we did that because then I got laid off. And I think the prestige from Giving a talk at Netflix definitely was a key part of me getting my next job. But in between, I started MLOps club and made a course about AWS because I had noticed working with data scientists that a lot of data scientists don't understand what they're, let's see. Like, like I, I think they're comfortable in a notebook creating a POC or maybe, or you know, you could. You talk about AI engineering too. They're comfortable like creating maybe like a chain lit app, like little chat application locally, but they can't. But so many people get stuck on actually deploying something to the cloud and, and I was hoping that would be like the sugar, the spoonful of sugar. Hey, we're going to show you how to deploy your stuff that we would bundle with a bunch of medicine, which is like. And by the way, I also want to make sure, you know, good practices that make you a co worker that I don't hate, like testing and clean code and that sort of stuff. So, so yeah, then I joined Pattern. Pattern's the biggest seller on Amazon besides Amazon themselves. And we basically run people's E commerce for them and there's a whole bunch of data science use cases there. So the thread through all this was the thing that really helped me get started, was a personal project that I did that made Ben Labs want to hire me. And it was basically a giant AWS architecture and this was a language learning app. And the thing we're going to deploy today is a pretty decently sized component of this architecture, probably better than I did it at the time. Here's the plan, here's what we're going to code and deploy. If you've heard of Streamlit before, I think more people have heard of Streamlit than Radio. Streamlit and Gradio are both tools in Python. These libraries that help you create a UI really quickly and they're really popular for demos. And so Streamlit and Gradio have two use cases, I would say, like one is personal projects. If you just want to get a quick demo, like if you, yeah, if you want to like deploy something for yourself, you know, show an employer or just show your friends, like a chat app or something that you built. You can easily create a UI with Streamlit and Radio in just Python. But also a lot of companies use Streamlit and Radio as well for internal apps. So you can kind of be a hero. If you're an AI engineer or a data scientist and you create a Streamlit app that you send some, some business users in your company that Lets them like interact with your models. So, so this will be sort of like, I think this project is going to be a nice, going to be a useful thing for two groups of people, like people who are already working to deploy internal apps and people who would like to get a job and be able to deploy their own personal project. And full disclosure, I will say that I was hoping to impress you by making a chat application kind of like ChatGPT and it does work. Ish. It definitely works locally. I think this chat interface with Gradio does struggle on AWS Lambda where we're going to deploy this. So yeah, chat applications in particular might be something that Gradio struggles with, but simpler things like buttons and forms and fields probably have a better time with. So okay, so with that said, let's start from the architecture and then I'll start a fresh project and just kind of live code it with you and let's see. Richie, do we have the ability to have questions come in live, like livestream style?

Yeah. So the audience asks questions in the chat. You can either answer them as you go or we can wait till the end to answer them as you wish.

So yeah, maybe use your discretion. But like I would say I would feel less lonely if, if I saw stuff coming in like that I could react to real time.

So okay, note to the audience then ask as many questions as you like. Please stop. Eric. Feeling lonely.

Nice. And I can always just ignore you. So like, anyway, so okay, so here's. Here's the plan. Here's the plan. Here's my architecture diagram here. So, so AWS Lambda, if you haven't heard of it, it is a serverless function, a cloud based function service and it has a lot of competitors. Google Cloud has one, Azure has one, Kubernetes has one. There's a lot of equivalent tools out there. So if you're learning how Lambda works, you're learning how like a whole class of serverless tools work. So basically what's going to happen is we as a user, we'll go to our browser, we'll go to this URL where our app is going to be, then our browser is going to hit aws, it's going to hit this AWS Lambda service and it's going to say, hey Lambda in aws, give me whatever app I have registered under this URL. Then the AWS Lambda service is in a just in time fashion going to reach out real quick and download our code. It's going to download our code into this runtime and then run it once and then dispose it. So the thing about a lot of recipes out there for deploying like a, deploying an app involve paying for a long running service like you're paying for something to run around the clock. So Maybe it's like 4 cents an hour and you know, what is, what is, let's see, so $0.04 times 24 hours, times 31 days in a month. So it's $30 a month for something that costs $0.04 an hour. So you know, even for a personal project like when I was in college, $31 a month, that would have felt heavy to me. That still feels needlessly heavy to me. With Lambda, with this architecture, you only pay if people are actively using it. And let's be honest, most of our personal projects sit and don't get used 99.9% of their existence. And so this will be virtually free. I mean I have serverless apps that I deployed years ago that I may be getting charged like 2 cents a month for just to sit there and exist. So, so anyway, this is, this is very cheap. Oh, and by the way, the bugs that I talked about earlier, like if you want to do a chat interface with the Gradio, those seem to show up on Lambda. If you want to pay for a version of this that does run around the clock, those, all this, this becomes a non issue. So I think, I think something about the serverless nature of Lambda, where it like disposes your environment is what causes the bugs in the chat interface anyway. So yeah, so what our Lambda function will do though is it'll basically serve us this HTML like this, this ui, the thing that you see here. And then when we interact with the ui, like put messages in the search bar, sorry in this chat box, then the Lambda function will have some python code in it that reaches out to AWS bedrock and hits an LLM and then we're having a conversation. So that's the plan. That's the plan, that's the architecture. At this point I'm just going to start coding. I would venture to say that this will probably be fast paced enough that if you try to follow along, you might get lost, but it will be followable. So definitely, if you want to recreate this after the live stream, what I would recommend is just ask questions and pay attention and learn during the live stream. If you try to follow along live, you probably will get lost. Maybe recreate this after, when we publish the recording of this. Of course there are resources here and there's an answer key you can use to get this running as well. Let's get started. I have here a completely fresh blank project in Cursor, which is a terrifying place to start. The place I always like to start when I'm making a new project is with UV initializing a basic Python app. That's what we're going to do. We're going to make a UV app. I already have UV installed. If you don't know, I hope you know. But if you don't know, UV is like the greatest thing that ever happened to the Python ecosystem, and it's essentially replaced PIP as well as several other tools. And so one of the things UV can do for us is get us some boilerplate. So here is. So I'm going to run this UV init command, I want to make a library, and I want to use Python 3.12. And so now that I'm running this, it's going to generate a bunch of boilerplate that we can use. It gave me this src folder with an init PY file. Fresh Python application. One thing that I really like about using Lib is it makes it particularly easy to have imports work between files. I think we will. Well, actually, for this application, we're probably only going to have one file, but if you wanted to add more and make it more complex, this is the right. This is probably the best scaffold for that. Let's see here. I'll run UV sync. And what that did is what UV had created here was a package. UV just installed that package and created a virtual environment where it'll go. So, all right, that was step one. We got a boilerplate project. So now let's make a Gradio app. Actually, I'm so sorry. The first thing we should do before we do anything, there is later on in this, in this live stream, we're going to be hitting AWS bedrock to get. To get responses from its LLMs. There's actually. We actually have to ask for approval from AWS to have access to one of its LLMs before we can hit it. So I'm going to go submit my request for that right now so that in case it takes a few minutes for them to grant approval, that by the time we get to that point in the stream, it's there. So, sorry, quick break while we go do that. So before the stream, I created a fresh AWS account, so I will go to that account and let's see, my font's probably small, so if you go to the AWS console, go to the Amazon Bedrock service. And in Amazon Bedrock, go to Model Catalog. And in Model Catalog, there's a whole bunch, there's a list. There's basically a registry of foundation models that you can hit via the AWS Bedrock service. I want to hit Claude. Probably. Let's go. Cloudsonnet. So I've scrolled down to cloudsonic. I could search for it. I mean, you could really pick any of these. I'm going to do this one. So click the three dots. Modify access. Let's see. Oh, I did notice this. This yellow box has been here for a little while, but hopefully this whole step of requesting approval may not even be required after a while. So, okay, let's go. Claude, just need. Okay, available to request. We need to make that request. How do we do that? There's probably. Let's see. Enable specific models. Okay. Quad. Oh, here we go. Check the box. Scroll to the bottom. Nice. Enter company name MLOps. I hope they grant my request for this. Given that, you know, I'm presumably bringing them sales. I always worry that they're not going to let me use their LLMs. Okay. This is going to be for internal employees only. Okay, next submit. Okay, so my request here is in progress. So hopefully by the time we get to that spot in the talk will have access to this. So. Awesome, awesome, awesome, awesome. Okay, let's start working with radio. So I'm going to Google search, build a chat application with Gradio. And one of the first things that should come up, I think since I've seen it before, it's the first thing for me. But creating a chatbot fast, this is actually a really good guide. So Gradio, of course, with the hype of LLMs and chat interfaces, they have a pre made, highly opinionated component that helps you make a chat interface in like one line of code. So it's pretty fantastic. So we'll just give you a quick, quick rundown of what they're doing here. So let's see. So this, this right here is a completely valid radio application all by itself. So we're going to import the radio library. We'll instantiate this thing called a chat interface. And all we got to do is call launch on this and suddenly we'll find ourselves with a web server running on localhost that has a chat interface. That chat interface, whenever we chat with it and put things in the search bar, it's going to call this function, whatever function we pass here, and the function will receive our chat, it will receive all the previous chat messages and then it'll do Something what we're going to have it do is reach out to AWS Bedrock and send the full history plus our new chat message to Bedrock and get a response. Anyway, let's try. I want to grab a less trivial. Now that's actually fine. We'll just take their basic code example. We'll go in here and inside of our app. I'm going to create an app py and paste in the code. Now let's see if we can actually get this to work. That's not defined. I'm going to copy the little sample function here. Cool, nice. They have a function that's taking in a message, which I think is a string, might be a dictionary. And then a list of dictionaries is what this takes the string or dictionary and this list of dictionaries represents the message that we just put in. And then all the previous messages that were like prior in the chat as well. And we send all that stuff to the chat interface and potentially to the LLM every single. Every single time we chat. So cool. So I'm going to try. I need to install the radio library. So I'll do uv, add Gradio and that should add radio. Cool. It's installed. So now UV run. Let's see. SRC Gradio on Lambda Live uv. It struggles with its autocomplete unfortunately. I wish I could tab complete this. How are we doing? Do we have. Cool. So I've got a little URL and it looks like it opened my wrong. Yeah, here we go. Nice. So. Hello. Yes. That's what it says. What? Yes. No. Yes. No. I was hoping to get it to say no. Okay, nice. So this is our chat interface. So this is what we're going to be deploying to aws, hopefully with some smarts to actually hit an LLM as well to do that. So we need to hit an LLM and we need. We need to. Let's see, we need that. Hitting the LLM needs to happen inside of this function here, the one that the chat interface is using. And I just want to point out really quick that like this function currently is returning a value. We could yield a value. So this would be. Make it so we could yield one thing at a time. If you use yield instead of return, this will allow us to stream like get that effect where it looks like we're streaming in token by token a response into the ui. Anyway, my plan is in this function. We'll use some library to hit an LLM and then get its responses back token by token and then we'll yield each of those responses back to the user. That's the plan. I was thinking about how I wanted to do this, because we could use Boto3, because Boto3 is AWS, is SDK that you use to hit all the services in aws. But then I thought to myself, like, well, what if I want to leave Bedrock at some point? I would hate to have to make extensive code changes to my application every time I want to change LLM providers. And also, I don't trust AWS to design an SDK that's ergonomic. I mean, AWS is good at many things, but it's also bad at many things. And I would say one of those things is ergonomics. And so I wondered, is there some sort of SDK where I can write code once but point it to any LLM provider? There are multiple. The one that I found that I got working was Light LLM. Light LLM, LightLM, they have an enterprise service, but they also have a Python SDK that's well maintained. What I wanted to show you is. Look how many LLM providers they support and image gen and other gen AI providers as well. Let's see, I'm on their GitHub page. I just googled it. And look at this table. They support OpenAI, Metalama API, Azure, tons of different services because there's an enterprise offering behind LightLM. I trust them to keep this working. I don't. It does. I don't think this is a SDK that is at risk of one day having a significant breaking change because I think they do have customers that are relying on at least their enterprise service and presumably their SDK is part of that. So that was my reaction when I discovered them. So cool. I would really just like to vibe code this this bit. But I'll start at least by installing Light LLM UV add Light LLM. Awesome, awesome, awesome. And let's do some vibe coding. Please use Light LLM SDK to hit the Claude 4.5 AWS Bedrock LLM to make the chat interface dynamic and actually plugged into an om. Okay, while we're waiting for that to roll, let's go check our. Oh, good. Our request has been granted. That's good. I noticed yesterday when I ran through this that my request was pretty quickly granted, so hopefully that's true of you too. Um, and this next bit was confusing to me and so shout out to my friend Jeremy Mumford who explained this. But so how do we actually go about hitting these end points? So what I did is I went to model catalog and then I found Claude 4.5 sonnet and let's see, click on this. Okay, so here's, here's like this page in AWS for Claude Sonnet 4.5 and it has a model ID here. And so I was trying to plug in this model ID to light LM and have it hit this model. It wasn't working. We will need this, we will need this model id. But there was another piece of information that we also needed for this to work. And it's called an inference profile. And from what I understand, an inference profile is a separate concept in AWS Bedrock that allows you to track the. Basically track your calls to one of these foundation models. So you can create a bunch of inference profiles, maybe one per person or one per team or one for application. So you can at the granularity level of a particular profile, track the number of invocations and the cost and so forth that you're making to these models. Anyway, so you have to come down here to model access. So I'll open that up in a new tab. And here if we search for our model, it shows should. It should show us the inference profile. Let's see. Claude 4.5. Yeah, we're here. Let's see. Oh, I'm sorry, I think it's in infer. Actually here we go. Cross region inference. I'm actually, I'm clearly not a bedrock expert. This is more just flavor for the, for our deployment here. Okay, so here is the model ID for Cloudson at 4.5 and that's great. But also over here there's the inference profile arn. So that's how you find there's like a default inference profile, I believe for each model you can also create your. And so we can use the default one to track all of the calls we make to this LLM as sort of one unit. But if we had different teams or apps, we could make our own inference profiles. So that's my layman's understanding of how that, what, what function that's doing. Let's see. So we are at 9:25. Okay, I gotta go a little bit faster. So let's see, how are we doing here? We got, got some generated code, so we're importing LightLM Completion Chat with Claude and. Okay, great. Yeah, it's. We're calling this completion function and we're passing in this model id. I already think this is probably wrong and that's fine. So let's pass. Let's go grab this one. This model and pass this in here under the model function. And then there's actually another field we have to set called Model id. And this is where we pass the inference profile. Great. Okay, so. Excuse me. So for this to work, I will need, on my local machine, I will need access to an AWS profile. So I am going to, let's see, do AWS SSO login. So I'm gonna create a profile called MLOpsClub. I'm gonna be using that profile. Great. So you can't see this, but on a browser there was this whole handshake that just happened. And now, now I'm. I'm logged in via the AWS ui and those, those login credentials have been persisted into an AWS profile. You don't have to create a profile that way. If you know how to create a profile using an IAM role, that works too. So this profile has pretty sweeping access. I give it administrator access to aws, even though the only thing it really needs to do at this point is invoke this model. Later on, I'll be using this profile for, like, deploying the infrastructure. And that's why I've made the permission so broad for this stream. Okay, so UV run and let's see. Oh, I'm sorry, I'm sorry. I just ran UV run. To run the app, I need to set as an environment variable my profile name, MLMS Club. And hopefully this will make it so that this app is using these permissions. All right, we've got a little URL. Let's go look at this. Hello, error. What'd we get? A nice thing about the chat interface is you can, I believe, make it show errors. Let's see. Maybe that's actually in the launch command. Let's see. Show error. True. Let's try that. No. Module named Boto3. I just saw that. Just saw that error in the console. So let's do UV add Boto 3, which is used under the hood in light LLM to actually call bedrock. Okay, let's try this again. I just made two changes at once, so we'll see. But what Show Error does is if we get an error, it'll actually show us the error up here. Like a toast. But look at that. We have a chat application developed very rapidly. Cool. Nice. Anyway, it's not the world's fanciest chat interface, but hey, I mean, look how quickly we're able to get a chat interface going, so fantastic. Let us now deploy this thing to aws. The plan for that is we're going to use AWS cdk. So we're going to AWS CDK as an infrastructure as code tool. There's a couple different options for infrastructure as code. There's Pulumi, there's Terraform, there's AWS cdk, there's Cloud formation, there's AWS Sam. So there's a whole bunch of options. It's fairly overwhelming for anyone starting out and for. For teams of a small size who are all in on aws. I actually recommend AWS CDK every time because it's Python and it has some features built in that you would otherwise have to sort of like configure yourself with some expertise or buy from a vendor if you're using Pulumi or Terraform. So to get a great experience, enterprise features for free. As long as you're all in on aws. Because CDK is AWS specific, I would say CDK is the right choice. So just plug for cdk. That's what we're going to use. So let's keep all these and now would be a fantastic time actually to commit because we're in a working state. Let's initialize this as a Git repo and Git Add all and Git Commit local radio app. Working. Nice, nice, nice, nice. Okay, so, so with AWS cdk, you need to. You need to have Node JS installed on your machine and then you do something like AWS install Global CDK or AWS cdk and this will. This will install the AWS CLI for you. Oh, maybe it's just cdk. Sweet. Awesome, awesome. So basically it's CDK is, interestingly enough, a Node JS project, even though we're going to be interacting with it, with Python. And so you actually manage its installation using the Node ecosystem. I guess you could have used PNPM instead of npm, but I'm just a JavaScript peasant, so I'm not. Don't have that set up. Okay, cool. So I'm going to create a single file called Infrastructure PY infra py and I'm going to ask our Vibe coding app to please fill out this infra PY file with a CDK application and a single stack that deploys our Gradio app as a lambda function. Use the following Docker file. There's a couple blogs out there that show. Well, actually, I'm just. This is one spot where I'm going to cheat and grab stuff from the answer key. To speed us up, here's a Docker file that I wrote before if you've never used Docker, you might be like, whoa, this is moving fast. Because now we're just writing a Docker file, which is this Docker concept. We're about to build a Docker image, write a Docker file and deploy that Docker image. Obviously we don't have enough time to explain how Docker works, but fundamentally this is going to take all of our. It's going to take our radio app, it's going to take all of our libraries for our radio app, it's going to take going to package them into this zip file called a Docker image, and then that zip file is going to be what Lambda downloads and executes when it runs. That is the very short version of what's going on here. Let's make this Docker file. I need to change this a little bit. The main thing that's happening here is. Okay, so yeah, what's happening here? I'll explain this copy statement if I get time later. But we are going to first install all of our. Let's see, we're going to first install all of the dependencies like Boto 3 and radio and such for our application in this line and then we're going to copy all of the stuff in our source folder into this Docker image zip file thing. Once it's all there, when we want to start the application, what we'll do is we'll run Python 3 and then we'll directly execute our app file. This is going to actually in this Docker image, in this Docker container, it's going to start the Gradio server. What this line doing here is it's a separate process that's going to be running in the container. This is actually the entry point. It's going to be acting as this process proxy interface sitting in front of our radio application. So it will be able to understand basically Lambda events or it'll understand the payload that gets sent to AWS Lambda and translate that into a format that our Gradio server can understand. If that didn't make sense, we'll talk about some resources where you can go learn more. I think if we can get this thing deployed in the next seven minutes, then we are doing well and the live is done. Let's see, use a Docker image based function and use the Docker file in the project for that. Note that this. Well, we'll just start with M. Okay, so let's make sure it has access to this context. I don't even know. I'm not a cursor expert, but I assume this will help. It may already be smart enough to get that. Let's run this. While this is running, let's talk about cloudformation a little bit. So CloudFormation is AWS Infrastructure as code tool. It is a saner way to create large amounts of resources in aws, like buckets, roles, lambdas, containers, all that stuff. If you have a group of resources that belong to one app and are configured sort of as a unit, then infrastructure as code really is the way to go to define how those things are deployed. Great. So let's see here. This is funny. Quick anatomy of a CDK app. At the very bottom here, there's this concept called an app. We're making an app. An app in CDK is a set of stacks and a stack is a set of resources that get deployed. We have one app and on that app there is a stack and on the stack there are resources like the song. Well, the code to define the stack we'll see in a moment. But the stack is going to end up getting rendered as a JSON file. It's CloudFormation JSON that we didn't have to write ourselves for generating programmatically, that JSON file is going to be sent to AWS and it's going to represent our order of like, here's all the resources I want to create. Cool. So let's see what's in the stack here. The stack has a couple different resources. So we have a Docker image function. This is going to create a lambda function. And this is. I think this is absolutely awesome. So the code for the function is this Docker image asset and we're going to point to our Docker file in our directory. So as part of our deployment process, CDK is going to take care of actually building and pushing our Docker image out to like the image storage area in AWS where it'll be needed by. Where it'll be referenced by AWS lambda. So this is going to take care of our Docker build and push steps for us, which is just awesome. In other infrastructure as code tools, you usually have to manage this yourself. I won't go really into these other settings here, except to say that I would like to build this for arm instead of x86. Let's see, can we do arm64 hopefully platform. This is just for sake of time. I'm going to cheat here. Let's go to infra PY and let's go grab this. Because I'm on a Mac when I build this Docker image, My Mac is going to default to building this Docker image for Army. Whether I'm running on a Mac, whether I'm running my deployment on a Mac or Windows, I don't want that to have to. I don't want that to have to matter. Or I guess based on the cpu, I don't want the developers. When a developer is running this program, I don't want it to be sort of random based on the CPU they have, whether or not this works. So I'm just going to force this Docker image to be built for ARM every time. And that's great because that ARM deployments in AWS are actually cheaper than x86. So we got that. And then we're going to do this. These environment variables are actually not necessary. One thing I will do though, is when our container starts up, when our application starts up, I want it to start up on port 8080. Let's see, is that the right. Yeah, here we go. And I want it to list it on all IP addresses. So that's a little config that I just set in the radio app. So now, now it should play nicely with certain assumptions that our Docker image has to satisfy to run in lambda. So, okay, so quickly going over this, we have a function. It's going to build our Docker file and it's going to register that into a lambda function. We don't need these environment variables. If we wanted to, we could parameterize our app so that maybe you could pass in different model IDs via environment variables. That would be pretty cool, but I won't do that. Now, let's see. Then we actually modify the permissions of the role on our lambda function and we grant it bedrock invokemodel for these resources, like for the foundation models that we want to hit. This could be a source of issues. We may need to tweak this role. When we end up creating it, it's also going to register our application. It's going to create something called a function URL with lambda, which is going to actually give us a URL we can hit in our browser that will turn around and invoke the lambda function. Anyway, the function URL is basically what enables this mode where you can access the lambda function as a website in your browser. That'll be pretty awesome. So what's the best way to proceed with this? All right, I gotta move faster, so I'm gonna cheat and take this Task Runner script. I was gonna tell you all about my opinions on how I like to make task runners, but I'll just have to skip that part for now. So here's a bunch of bash functions that are useful for running our application, and I have a couple of these functions that have to do with cdk. So when you're using AWS cdk, you actually have to. Before you can use CDK to deploy anything, you have to run this command called CDK Bootstrap, which is going to create some essential resources in AWS that CDK relies on. So that's the first thing I'm going to do is run, let's see, run this bootstrap command, do that, run CDK bootstrap unbound variable AWS account ID Yep, Okay, that's going. Another thing I want to do is I want to see if our CDK app is valid even. And so for that we can run this CDK synth command, which is going to try to take our CDK code and produce the CloudFormation JSON out of it. So let's try that. It'll also, I think, build our Docker image as well to make sure our Docker image is valid. So, okay, running CDK synth. Oh, module not found. AWS cdk. All right, let's do UV add, dash dash script and these two. So I'm going to add dependencies directly to our infrastructure. PY file as comments. Awesome feature of uv. I'm going to add these two libraries in CDK needs. Cool. Let's try that again. It's giving me this warning message saying that it doesn't like the version of Node JS I have installed. I think that it CDK wants you to have an exact version of Node JS like exactly 24.0.0 or exactly 22.0.0. So in my experience it's never mattered. If I can't get this deployed very quickly, then we might just have to turn this into a casserole in the oven demo where I just show you the version that I already have deployed. We're on the home stretch. Our synth actually sent successfully. That's pretty nice. At least we know that our generated infrastructure as code is valid. It's well formed. We don't know if it'll work. We have to deploy it for that, but this is definitely promising. I'm going to run CDK deploy. Nice. First thing it's going to do is synthesize our Python code into the synthesis step. Here is what am I trying to say is creating our CloudFormation JSON, then the next thing it's doing it's actually building our Docker image here. That's fantastic. We will see further down if it succeeds to build. I guess we'll give that a sec. Then the next thing we'll see is we'll see CDK pushing our Docker image, which is nice and. Okay, cool. So it looks like the bootstrap process finished. So. Good. Here it is pushing our image. All right, so essentially, just like Terraform has Terraform plan and Pulumi has like an approval step in Pulumi up, CDK won't just go create resources in our account without confirming with us that we like what's about to be done. And so it's highlighting, hey, here's a bunch of permission changes that are about to happen. We're going to create a role. Are you okay with all this? Yeah. Anyway, so I'm just going to say, yes, I approve of these things. And while we're waiting for this, let's go over to the Cloudformation UI and I'll show you what's happening over here. So in the cloudformation UI, you can see stacks. I'm in the wrong region, so go to Oregon. So here's a stack called CDK Toolkit. When we ran CDK Bootstrap, this was created and that gave us things like an S3 bucket and a Docker Docker image registry. So when we ran, when you saw CDK pushing our Docker image, it was actually pushing our Docker image to a repository created in this stack. So this is like a. Like, this is one of the quality of life things that CDK does for you. It makes it so you don't actually have to think about what repository I'm storing my image in that's sort of abstracted away from you. And like under the hood it goes into this repository. And here's our stack with Create in progress. So we can click on the stack and we can see the resources that it's trying to create. So can I. This view is a little annoying. Okay, here we go. So, yeah, it's creating like a. It's creating a AWS lambda function and a lambda URL and a lambda permission and an IAM role and an IAM policy. All this stuff you didn't explicitly see get configured in cdk. And that's because similar to how Terraform has modules that are opinionated, CDK has constructs which are like sets of already opinionated configured resources. So when we created our lambda function under the hood, invisibly, we were also creating an IAM role and a policy as well. So we didn't even have to really think about setting up those permissions, which is fantastic. Ok, so we're close. The create in progress is. The last thing that needs to be created is the function itself. So we are working on that now. Here's a timeline view of each resource in the stack and how long it took each one to be created. Let's see. Create complete, maybe. Are we done? Create complete. Well, moment of truth. Does our app work? If it doesn't, this is where we enter the debug loop, debug our infrastructure's code and see the app. But it would be amazing if we can see it right now. It's taking a long time to load. Why is it taking a long time to load? It's because this is a lambda cold start. So this is the first time we're hitting our lambda function in a little while. And so right now lambda is actually. It's going to take about 10 seconds, I'd say. Lambda has to download our Docker image so that it can run it. Oh, no. Oh, no. What was that? What was that? Download link. That is absolutely not what we want. Okay, we don't have time to debug this because we were really overdue for Q and A time on the stream. So I wish I could. I wish I could say that, you know, we got the lambda function fully deployed. I could probably cheat and go grab. In fact, maybe I'll go do that right now. I think I'll go just deploy the working version.

Yeah, we got to see the working version, so at least deploy that.

Cool, let's do it. I'll go to my answer key. Let's go run cdk deploy. Okay, so while this is running, let's take some questions.

All right, super. Actually, before we get to that, I just want to talk through what's happening with the next few webinars because the count gets slightly tricky. So actually, next week Rhys and I are away, so no sessions for you next week. And I'm very sad you have to watch reruns or get a hobby. Otherwise entertain yourselves. But after that, we've got AI agents, we've got two weeks worth of content purely around agents. We've got nine webinars over two weeks. It's going to be intense, a lot of fun. I think Reese has put a QR code there on screen. So if you want to register that. It's one form to register for all nine sessions. So please do come back for that in the meantime. Yeah, let's go to Some questions. So. All right, so there are few people asking around why we're using Gradio rather than Streamlit or any of the other platforms for this.

Yeah, great question. So Streamlit requires you to run in, like, a container setting, so you have to pay for it around the clock. So Streamlit is a fantastic interface and we use it at work because we're not afraid to pay $30 a month for running a container. The nice thing about Gradio is I believe it's actually built on fast API. So I guess there's this concept of web sockets where Streamlit requires the browser to establish a websocket connection with the Streamlit container. And in order for that to work, the container has to sort of be persistent, running all the time. Grady doesn't rely on WebSockets, and therefore it can be run in this, like, serverless setting.

Okay, so that seems very useful if you're like an individual and $30 a month is like, you know, I mean, it's not nothing. So. Yeah, so if you're only using the thing intermittently, then that seems like a good option, actually. So there's a related question here from. From Evangelo is talking about, like, why using Lambda here? Because the other option is like a low resources vm and so in fact, I know hundreds of services on aws. Can you talk to, like, which services you might want for, like, different use case patterns?

Yeah. So Lambda is like the only one that gets you this serverless. I only pay when I use a thing. Oh, hey, look, it turned on. Let's try setting a message. Hello. None dive has no attribute get. We'll see. What's funny is, what's funny is I think this is actually an issue with Gradio on Lambda, and I think we'll actually see it intermittently work here in a bit. So anyway, we'll give it some time, we'll let it cook.

I feel like there's some lessons around vibe coding here and thoroughly checking code, but it's very tricky situation.

Absolutely.

Okay.

Anyway, you were talking about the different services for different use cases.

Yeah, Lambda is like the one service in AWS that gives you the serverless. Like, I only pay for it when I'm actively making a request. If you spin up a vm, you're gonna be paying for the VM to run all the time do containers. So that's like ecs, App Runner, Elastic Beanstalk. All these things are gonna require you to, like, pay for something around the clock. Yeah, those are sort of the options Kubernetes is of course even worse. You pay for an expensive control plane and containers.

I guess there's a big difference between I'm doing something like as a personal project, it scales right through to I'm running some corporate app with hundreds of thousands of users or whatever. You're going to want very different infrastructure in different cases then.

Yes, yes, exactly. If you're looking to have a persistent app, like easiest way to get started is probably Amazon Elastic Container Service. If you're looking for a serverless app, Lambda is just the only way to go. So.

Okay, all right. Oh, okay. So the question on terminology here, ergonomic design, do you want to talk us through what, what that means in this context?

I just mean like low. Yeah, sorry, like, I mean like low learning curve and simple like, and usually not running in, not bumping into lots of like rough edges that make a tool hard to use. So like I would say ergonomic is the opposite of hard to use.

Okay.

Easy to use. I like that.

Seems like a very useful property of like any software. I think something, something more designers should take. Take notice of. I think.

Yes, please. Yeah.

All right, nice. So someone only known as RJ said, are you actually really using Internet Explorer? I think that logo you showed at the start, it's traumatized a few people. The cutting edge of web browsing technology.

Exactly. I want Microsoft to have all my data. Yeah.

Start using some of those command line only 90s browsers as well. All right. Okay, so next question from RIM. RIM's asking, is CDK the only way to deploy an image on AWS? I guess. Are there any other options here?

Yeah, actually this is a great time to pause and just give plugs to both Data Camp and myself. So we went fast, you know, did we even finish? Like, I don't know, it's maybe a tall order to develop a whole application and deploy it all in 45 minutes. But. But if you want to learn more about aws, there's two places I'd recommend you go. So first I'll plug myself. So I made this course taking Python to production on aws. We talk about a lot of stuff, how to set up AWS accounts, development practices, infrastructure, a lot of the fundamental services. So if you want to just. This will probably answer hundreds of questions and then I don't have it pulled. Yes I do. Yes I do. So people have asked us though, like, will this course help me prepare for a certification? And the answer is really no. Like it's really more of a developer oriented course. Helping you learn how to build stuff. Certifications are sort of more broad. It's like our course is deep and focused, like what a developer maybe would want. But certifications require you to go broad on like a variety of things in aws. And that's where I would actually recommend datacamp's Cloud practitioner, like certification track. So anyway, so sorry, but you asked is it is so. Because I was sort of responding to your question. Is this the only way to deploy an image on aws? No. And I think to answer that question we would have to like talk about what the tools on AWS to do that. But I think the TLDR is if you know about Docker Hub. Like Docker Hub is this website where you can go upload images to AWS and the other clouds let you make your own, like private Docker Hub, like private Docker image registries. So you end up creating one of those and shipping your images there. CDK just saves you multiple steps as you do it.

Okay, nice. Okay, we've got one or two more audience questions and for anyone in the audience, if you do have a question, Last chance to ask, please do. But I've got a question for you. So I'm kind of curious about how you develop this workflow because I know some of this is vicod, some of it is pre prepared. How do you go about structuring these things? Do you do model first, then ui? How do you order all the different things we cover today?

Yeah, I would say local first and then cloud. So the first thing I did was get a Gradio app working locally. And in theory, if you have a good development process, if it works locally, it should work in the cloud. And Docker's supposed to help you out, help out with that. So that's generally the mode I would follow. So maybe do your infrastructure as code second after you get a thing working locally first. Because if you don't, oh hey, look, it's working like we got our chat. Okay, there we go. Yeah. Because if you don't, then you'll end up when you deploy broken code to the cloud, it's just going to be broken code in the cloud and then you're going to have like this confounding situation where it's like o, okay, are my issues coming from the cloud or are they coming from the code itself? So anyway, it's definitely really important to get that set up first. And that was sort of what we did with UV when we started.

Okay. So I like the idea of develop locally first, then Figure out the cloud stuff because it's simplifying problems so you've got less places for errors to happen. Do you develop like the functionality for the application and the user interface separately or do you combine those?

Oh, I mean, for a gradio app. Well, yeah, I feel like UI does come last, I would say, like, you know, like I'm trying to think of an example, like at work, what we have like forecasts and we try to, we try to predict like how many iPhones do we need to order to the warehouse close to us. That two day shipping works and of course the data scientists are spending a lot of time like analyzing that problem and they're creating these bespoke reports and they're, they're playing with data sets. When that's all done and they know it works, then they'll create an interface that people in the business can go play with. Like, okay, what if I want to ask a question about this specific product and hit their app? So it's like, so it's sort of like it's really an interface layer over underlying functionality. And so it doesn't really make sense to do the interface, I would say, until you get the underlying functionality in place. Okay, I like that.

So functionality first, then ui. And one more question from me and then we've got a question from the audience. So you were doing five coding there. So you were getting the code auto generated by the AI. What's your process for ensuring that the code that's generated is what you want?

Yeah, I'm one of those engineers that thinks you actually need to understand the code that's produced. The unfortunate reality is I've written a lot of AWS CDK code and so it was pretty easy for me to just check that. The fact that I could walk you through line by line and tell you what it was doing showed that it was sort of exactly what I was hoping would come out. Um, and one nice thing though is there are tools like Amazon Q. CLI is kind of like Claude code. Amazon Q is pretty good at doing tasks, tasks against aws. And so I've had some really nice wins where I'll be like, Amazon Q, please help me like test this stack. Like if I deployed an endpoint, send up, send off a few requests to it and make sure that that works. And Amazon Q I've noticed within AWS tasks does a pretty decent job at like running my like doing the deployment and then inspecting it to see if it's good and then like iterating from there. So There's a little less supervision I'd say you have to have there.

Okay, so I suppose that does make a lot of sense. Amazon Q good at giving you code for aws. Probably don't want to ask it about Azure code. Probably not the wrong code on purpose.

And I would say it's really actually bad at code, but it's good at taking action against aws. And so that can be really good to, like, help you debug, like, ah, I don't know where to find these logs. Amazon Q. You go find the logs. And it's actually pretty good at that. So. Okay.

I mean, yeah, I presume you train fairly heavily on, like, all the Amazon documentation there.

Exactly, yeah.

All right, wonderful. Okay, so, final question from the audience and then I think we're done. So Matthias asks, can you explain about the advantages of Docker and CDK? Use something called DockerBake.

So, yeah, Docker.

I haven't come across Docker Bake before.

Feature of BuildX lets you define configuration files using a declarative file as opposed to specifying conflict. CLX permission. What? So is this like, that's interesting. I actually don't know anything about Docker.

Bake, but okay, bouncing everything.

Seems like that would interface on top of Docker files. That's kind of cool. Yeah.

So.

So disadvantages and advantages of Docker and cdk. I mean, why did we use cdk? It's because I didn't want to have to go click a whole bunch of buttons in the AWS console to configure something. And of course, if I want to share this with you such that this is a template that you can deploy yourself, it needs to be code in some way. There's a whole discussion you could have on if clicking in the console is better than infrastructure as code is better than just using the SDK yourself. TLDR infrastructure as code rocks. And CDK is like. It is the one you should start with on AWS unless you have strong reasons to do otherwise. So they're all good. But CDK is the one you should start with unless you have strong reasons to do otherwise. And then as far as Docker, you know, it was. We needed to. We needed to package our code somehow. So Lambda doesn't require Docker. So this is just a peculiarity of Lambda. If you don't use Docker, then your whole application and all its libraries has to be smaller than 250 megabytes. I didn't really want to torture myself and try to shave down the libraries. This time. So I just went with a Docker image and honestly, I think. I think that's probably the right choice. So.

Okay.

Yeah, I do like the idea of just having everything in code so you're not then making people go and click in AWS to reconfigure the stuff that you've done already. Just seemed like a useful way of improving sharing. All right, nice with that. We are at time, so thank you so much, Eric. That was very cool stuff. It just sort of works. Just sort of get it working at the end. Always tricky in a live situation. But yeah, I think the process and the idea was very, very cool stuff. I feel like I learned a lot. Wonderful. All right, and thank you to everyone in the audience who asked a question. Thank you to everyone who showed up today. We got a session on Thursday. Maybe see people on Thursday. Otherwise see you at the end of the month for AI agents.