AI Dev 25 | Mike Chambers: Serverless Agentic Workflows with Amazon Bedrock
Mike Chambers walks through building and deploying agents on Amazon Bedrock using serverless AWS tools. He shows the full workflow from creating an agent in a notebook to invoking it in production, covering Lambda, DynamoDB, S3, and Bedrock's guardrails and tracing features.
Chapters
Thank you very much. Thank you, Jeff. And yeah, look, thank you to everybody at the Deep Learning AI team. Seriously. I have and some of my colleagues have been working alongside them for quite a long time putting courses together prior to the short course platform as well. So it is very exciting. Yes. So as was mentioned in the introduction, I want to talk about agents, which is pretty much what everybody's talking about here today. Right. But I want to talk about agents in a slightly different way potentially. I want to talk about moving these agents into production. We see the title up here at the moment, which is Serverless Agentic Flows with Amazon Bedrock. That was a title we very hurriedly came up with actually just before recording the course that this is based off of, which is on the platform. But back then, which was, I don't know how long ago that was a fair few months ago now that was what I think was interesting. It's like we were doing something slightly different. We're getting serverless, we're talking about the infrastructure we're going to be deployed upon. I had this thing in my mind that I wanted to talk about production and that wasn't necessarily something that everybody was talking about back then. But I don't know if you've noticed that today here on this session and to other things I've been into in SF recently particularly, that's what people are talking about. Okay. We've built agentic systems, we've had a go with rag. It all works on my laptop and this is pretty cool. And I'm experimenting with this, that and the other. Now I need to move into production because we're getting a little bit of pressure from up above in the organization we're working in. We're spending a bunch of money on this. We want to actually see some outcome. So that's sort of what I want to talk about today. And that's what the title of this course is. And look, I'm doing that thing where I sort of unplugged. I'm either a stand up comedian or something. I don't know what's going on here. I need to be able to type. And so a little bit about me. Yeah. So as was said, I have produced a number of courses, but some of my colleagues as well on the deep learning AI platform. I'm super happy with this first image here. And if you all stay at the back of the room, you'll see Antia, who is in that picture. I just noticed taking pictures from the back of the room. And this was the first course that I was Involved with. We had courses before that. And this is generative AI with LLMs. It was pretty much the first course that you could get on large language models. Look at the date. 2023. That was a long time ago, right? Yeah. Super exciting. And I don't know if you can tell at the back, but I got Andrew to play Transformers with me because it's all built on Transformers. Right. That was the joke. We had fun since then. Got a couple of other short courses out on the short course platform, which hopefully you've all used. One where we look at just getting into large language models with serverless applications and then getting into this agentic flow, which is of course what we're going to be talking about today. So let me skip ahead. I'm just going to do a brief introduction here. So you never really know where everybody is on their journey. I'm going to make an assumption here that probably most of you are familiar with agentic workflow. Close because of where you're at today. But let's just cover it really quickly. An agent is a system that uses generative AI, as we know, to connect in with systems in the outside world. Something beyond their reach. Sorry, something beyond the internals of the model we connect them in. There's a few more bits and pieces to it than that. We have tools that the agent uses to be able to go and connect in with some external system. That's what these triangles are. I showed this diagram to someone. I said, what's the triangle? It's. It's not a standard thing. These triangles are external systems. And the agent itself, it really is just. It's an orchestrator. It's something that can at the underlying all of this. It's really just doing string manipulation, isn't it? But it needs a large language model or some kind of generative model in order to sort of encapsulate its intelligence. So I put that up the top there and then we have the agent in the middle. One of the sort of core aspects of an agent is that it has this ability to be able to loop in itself. You can ask it something and then it can go and interact with these tools and these external systems and then think about it. Do I need to now go and get something from somewhere else and run another tool and follow this agentic process? This is what we call it. What I want to talk about here is how we can get this into production so off something which is just working on our laptops and is super cool, into something which is in production and will Run at scale, which is of course, what's really important for us. I'm going to step through how to do that. This is, by the way, a workshop. So there will be a QR code shortly. So you will be able to open up the workshop environment and follow along if you wish to do so. If you don't wish to do so, that's totally fine as well. I understand you sort of not got work surfaces to work on, but you got laptops and you've got laps. So you can do that if you want to, but you can just follow along with what I'm showing you and then run the workshop maybe when you're not here, when you go home. So let's look at this now from a sort of more of an AWS perspective, just by show of hands, who sort of deployed anything, production or otherwise, on an AWS platform. Wonderful sea of hands. I'm among friends. This is fantastic. So I'll be talking about some technologies that hopefully you're kind of familiar with as we step through this. So let's look at this then. So we've got this agentic workflow that we sort of are familiar with. Let's look at these tools. So how can we run tools without many headaches? Well, inside of Amazon Web Services, of course, we've got lambda and lambda functions. These are places that we can run snippet functions of code, we can run them in production, we can run them at scale, we can run them securely. This is a perfect fit for an agentic workflow. So we already have a component. This wasn't introduced for agents. This is exactly the right kind of thing that we're looking for. Now let's look at the LLM on the top. So this is large language model. Obviously this term is getting extended beyond just language, but LLMs in general. So we have Amazon Bedrock. Who's heard of Amazon Bedrock? Just by a show of hands, everybody's heard of it. Fantastic. Anybody used Amazon Bedrock? Yeah, a decent number as well. Thank you for that. So Amazon Bedrock for those who haven't, and maybe also for those of you who have. It's not a model. What it is is a collection of different services and components that you can use to build generative powered applications. And so in this particular case, I'm highlighting the foundation models that we have. So Amazon Bedrock has a big list of different models from different model providers. So yes, we have our own models with Amazon Nova models, but we also have anthropic models in there. We have LLAMA from Meta, we Have Mistral and we have a list which is longer than I can remember all of them. So take a look at document documentation. And we have deep seq R1 in there as well. All as managed models, as managed service endpoints that you can just go and use. We also have, and I'll mention this now, it's not a topic of this conversation, but we have the Converse API. It's something that I think we don't talk about enough. What it does is it actually abstracts your code that you're writing from the different models, but not in a huge way. Right. So I know that there are frameworks out there where you can write something and then abstract your way. Plug any model in. This basically gives you pretty low level access to the conversational flow of models through the Converse API. Change the model id, go to different model providers. So question I had today was, and there's many answers to this, by the way, but a question I had today was like, why would I use Claude, for example, on top of Bedrock rather than going to Claude directly? And there are pros and cons in each direction. But one thing I will say is if you're connecting with models through Bedrock, you're one ID of that model away and you're code to go and use a completely different model and to go and try something else. So Deep SEQ comes out and everyone's going losing their minds because it's so amazing, and I know it is pretty amazing, but you can go and change the model provider and go and use something different. So that's foundation models on Bedrock. There's lots more to talk about and there's more to Bedrock than just that. So we also have a bunch of managed components and we'll be looking at quite a few of them today. And the one that we're sort of centering this around is agents. So we actually have a fully managed platform where we can put agent configuration and again, it will scale in production and run at scale. So that's what I want to talk about there. And then of course, because you're running inside of aws, you've got access to the places where your data is and the things that you probably want to connect into. I'm going to highlight some favorites here. So DynamoDB, awesome NoSQL database, which runs serverless inside of AWS S3, I'm sure everybody's aware of, and then the amorphous cloud, of course, anything that a lambda function can connect into, which is pretty much anything you could connect into your agentic workflow Right. So the exciting thing about this is that absolutely everything on here is serverless. And what we mean by that is that there's no infrastructure to manage, there's no patching, and it will scale for you to production scale. So it's a nice way to be able to move your agentic workflows into production. I'm on this setup here, so I can't see where the next slide is. And I have to remind myself, I put this together specially for you. So we're both seeing this for the first time. Oh, yes. So there's other components in there as well. And so RAG retrieval, Augmented generation. Yes, we have that fully managed too. And guardrails as well. So guardrails to sort of run out of band with the model to be able to see what's coming in and out. And making sure that we're keeping your customer service team who are running this agent, they can sleep at night because they're confident that what is happening is going to be in line with policy. So these are managed components as well, which we'll have a look at. Again, potentially running in a serverless way behind the scenes. And every presentation needs some generated image. And I realized I hadn't got one up until this point. So I'm going to replace my person with a secret agent. I don't know. Yeah, okay. I enjoyed putting it in there anyway. And they're going to be there for the remainder of this presentation, so get used to them. Okay. So I have decided that I'm going to litter throughout this presentation. Some QR codes get you used to the fact that they're going to be there. There's going to be a QR code at the end where I've got, obviously, the mandatory session survey kind of thing. There's also going to be $25 AWS credits on that as well, so that you can try out some more of these things outside of the workshop as well. This is the agent's user guide. Maybe don't start reading it now. That would be kind of weird, but bookmark it. We'll see what happens. Okay, let's jump into this a bit more and start to get towards where the code's going to be. So what to expect from this workshop. So, well, feel free to work along with this if you want. We've got a bunch of AWS people in this room right from the very front to the back. We're mingled amongst you. We're watching. No, we're all labeled, so you'll be able to see us and we're here to answer questions for you, if you want. I'll answer the questions if I can up here because that's kind of the stage. We've got. We've got microphones that can go around so we can all hear your question as well. This is your session. I've seen it before. The workshop, that is. So that's cool. This is all for you. If I can't answer your question, we've got experts in the room who I will rely on to answer the question or take a note of it and we'll get it answered for you. So this workshop is built on top of. In fact, it is essentially the Deep Learning AI short course platform. Who's taken a Deep Learning AI short course before? Basically everybody. Excellent. I'm not. Yeah, cool. That's cool. I'm not going to ask you if you've taken my courses because I don't want to know, but you will now take one of my courses. So that's how it's set up. What we've done is we've basically created the same environment, but we removed the video from the side because that would be kind of weird if you're watching a video of me and watching me. So we're just going to have the code component and hey, if you don't like this guy, you can go and take the course for free afterwards as well with my video and play it or not, as you wish. I got the tea mug there because it is a tea mug, by the way, not a coffee mug. Watch the video if you want more background on that. But I have this sort of. You have to come up with a use case for this. So I've got a customer service chatbot which is helping with my mug selling business. Because when I was figuring this out, I think I had a mug in front of me. It was the first thing I saw and it seemed to work. So that's what we're going to do. QR code here, reasonably untested. I don't know if someone tested it before, should help you get into the Deep Learning AI platform. And if you have an account, you can log in and it will take you to the course. Feel free to take a picture of this. You can do it afterwards as well if you want to. But if you do want to do it afterwards, I'll probably point you in the direction of the formal course with the video. That'll make it a little bit more sense. I will wait until I see fewer phones up in the air taking a picture of this Because I know how annoying that can be. What happens here is it should take you into the short course environment. Log in and you probably end up looking at a notebook at some point if you're following along or if you're doing this later. And that notebook should look a little bit like cross fingers everybody, because live code for the rest of this. This. Hey, let me make this a little bit bigger so we can see it. So you're probably very familiar with what this environment looks like. And shrink that down. I'd never thought of doing that before. Basically we're in a jupyter notebook environment. What I want to do here is I don't know if I can get rid of that stuff. You'll be able to see in that little window there. We'll step through the code and have a look at this. What we're going to do is we're going to step through creating the architecture that we were just looking at a second ago. And this first lesson here, if we can call it that, is just to get started. So we're going to create an agent. This agent actually is a little bit unfair. For this agent, it's pretty useless. It doesn't have, if you see, it doesn't have any tools, it doesn't have any access to the outside world. But what it does have is conversational history. It has the connection with a large language model. We can see how these things are created. We can see how we can look at the trace as it's reasoning through its thought processes and we can see how we're going to invoke the model. So that's what we're going to do. Let's step through that and I'll just step through the code that I have here now, as you step through these lessons, there are some cells inside of reconnecting to kernel. Let's hope it does connect to the kernel. That would be excellent. There are some cells in here which are just sort of helpers for this course. It does things like reset the lesson. So if you're going through it and you. The experiment, which you absolutely should, and it all completely collapses and messes up, you can go to the top, reset and try again. So that's how that works. So that's running here. That's nothing much to be bothered about in there. And then we start to jump into the code. All right, so what we're going to do is this is very code centric here. You can do a lot of this in the console. You can do all of this in the console. And we'll talk through that at the very end for various logistical reasons. But I want to talk about how we can use code to build these agents. Because the deep Learning AI course platform is primarily focused on code, which I think is a really awesome thing. So we use the boto3sdk. So the boto3sdk is the AWS SDK for Python. It means that you can use Python to configure and then consume pretty much everything that's inside of aws. So we're going to load that up and then we're going to start getting into the code. So I'm going to run these cells. If I miss a cell, by the way, because I'm busy talking and I forget, just shout out, maybe I'll miss a cell on purpose to see if you're paying attention. Okay, so here we're running and also just hoping. This should have finished a long time ago. I don't know why it's still taking its time. Maybe WI fi, maybe, who knows? There we go. So what we're doing here is we're creating a Bedrock Agent. Bedrock Agent Client Object. So this gives us the ability to be able to configure the agentic side of Bedrock. There are different clients inside of Bedrock. So you've got the ability to be able to just invoke models, configure models, that thing, just the bare bones stuff. What we're doing here is working with agents. So I've got myself a Bedrock agent with that set up. With my Bedrock Agent set up, I can now go ahead and call Create Agent. This is all logical so far. With Create Agent, I can give it a name. So I've got my mugs Customer Support Agent. I can give it access to a particular model. And this whole course actually was actually devised prior to Amazon Nova being a thing. So we'll talk about Amazon Nova a little bit later on because it's something which is released afterwards and it's super cool to talk about. But we've got a foundation model here of just Claude three Haiku. It doesn't matter which model, I suppose, but that's one we've chosen and then some an instruction. Not a prompt, but an instruction. And there's a reason for that. That's because this instruction actually gets inserted into the prompt along with all the tool definitions and all the other stuff. So that's the terminology there. We also give it a role. This is security stuff that's set up for us already inside of this course environment. So I'm going to run that. And the agent should now be being created. Now this is a production environment, so I've got a couple of steps I need to do here. So I went to a hackathon. This is side story. I wasn't planning to tell this. I went to a hackathon and you know what, it's like a hackathon, everyone sort of like smashing away on keyboards to try and get something working on the local laptops. And I was in a team and we were putting an agent together. I was like, don't worry guys, I've got this, I can do this. And so I go away and I just open up bedrock and I'm like, with agents. And I'm tap, tap, tap, tap, tap. And a few minutes later I come back and say, there, I've got it. And they're like, what's that? And I said, it's a fully productionized agent for what we're building. And they're like, what are you doing? We're just hacking code together here. But we ended up doing quite well, actually. And everybody else is like, here's my thing running on my laptop. And I'm like, well, here's our fully productionized thing. Never mind. So this is what we're doing behind the scenes. What we have back from this is we have a create agent response. We can look at that and it'll be like super scary with lots of stuff in there. There's information in here which is useful for us and we're going to extract this out. So we've got things like the agent ID that we care about. And we also have this agent status here as well. And the agent status is creating. And you might say that's why you're padding this out, you're waiting for it to finish creating. But it's not, as my colleagues will attest to. I just talk a lot. So let's keep going. I'm going to grab the agent ID out into this variable here. I have some helper functions if you're interested and you're going to follow along or if you're going to pick apart this when you get home, have a look at the helper functions. They just sort of mean that we can just skip through some of the stuff that we don't need to talk about. That's not a focus for us today. So that's all they are. We'll see those come up from time to time. And one of the helper functions is this one. It's just a function that can wait for us While our agent is being prepared. So this is going to loop through. Oh, no, sorry. While it's getting into the not prepared state. So the first state for the model is not prepared. So we created it. It's not prepared. We now need to go and prepare it. We won't dwell on these things later on in the course. We now need to prepare the agent. So we run this command here, which is directly out of the SDK, to prepare agent. We're giving it the agent id and then we have to wait for the agent to become prepared. And so this is a helper function, just to allow us to run the notebook from start to finish, if you want to. And it should just sit there. It should output something a little bit latency here today. Here we go. The agent is now prepared. So what we've done is we've configured the agent and we've asked it to prepare itself. These are stages we go through because again, we're moving into a production environment. The last thing we have to do is create an alias for it. And so, and alias, it's similar to if you've used some of the things inside of lambda and you've used Lambda in like serious production environments, you have similar kinds of things. And so think of that as this. And so we can see that this is now creating, creating prepared. Now we have a prepared agent. Now there are a few steps that I just went through to go from having nothing at all to having an agent, admittedly with no tools, ready. What we didn't do though, was go to any server. We didn't go to an EC2 instance, we didn't go to a container, we didn't even go to a lambda function. And everything is now ready for us to invoke. Now, again, I want to remind everybody this agent currently has no tools, so it's going to struggle a little bit with our requests. But let's see what happens. I'm going to use my Boto3 client this time, looking at the bedrock runtime or bedrock agent runtime. So one for configuring this, one for actually invoking and using the agent. So I'm going to grab myself one of those. I'm going to include a UUID here. I'm going to send in this session id. This session ID is a unique ID inside of my account for talking to this agent. Conversation history is all inside of the agent. Right? So if I come back to this agent a few minutes later, I want to carry on my conversation. And I can do that by passing in the Same session id. Obviously, if I rerun this code, it's going to reset the session id, so you have to figure that out. But anyway, if I just run this, a few things are going to happen. So I've run here, invoke agent and I'm invoking the particular agent with the alias ID that we just set up. I'm passing in the message and I forgot to review what that message was. There's a bit of a narrative here, but I'm not happy at the moment with my mug because it's broken. So I bought it from the store yesterday. It's broken and I want a refund, which seems kind of reasonable or return. So I've sent that off to the agent again, remembering it has no tools. So at the moment it's going to feel very sympathetic for me, but it's not going to do very much. Let's go and grab the information out of that. So we have something called an event stream that comes back from this agent. So I'm just going to go and grab myself that and then I'm going to iterate through and see what it looks like. Now it probably will look very verbose and believe that it's probably already run. We're waiting for the data to reach to my laptop rather than anything else. Interesting why this is taking so long. Let's give it some other cells to run while we wait. But basically it'll iterate through the event stream which comes back from the response really should come back by now and we should see the various reasoning steps. And those reasoning steps actually will be probably saying things like, I can't really do very much, I don't have any access to functions. I need some more information to do anything moving forward from here, but this might be my time for the demo gods to be not in my favor, which hasn't happened for a while, but it's happening right now. I do have one other function which we have here, which maybe we'll look at in other lessons if they're behaving for us, which is to invoke agent and print. That's a helper function, just to sort of show the reasoning steps as they come out on screen. So run that again. Well, who here has done live coding on stage and who's got lots of sympathy for me right now? Thank you very much for the hands which are coming up. It's working on the lesson. Well, thank you so much for doing the lesson as we went along. Would you mind just coming up here and showing. No, I'm kidding. This is a my laptop, my connection kind of thing. Right? This is clearly the service is working, otherwise all hell would be breaking loose in the rest of the world. So let's move on from this maybe and see what's going on. Let me open up this again. So what we've done here is a couple. See, not even the sidebar is loading for me now this is something more fundamental. Let's do a refresh and see what happens. So what we have been doing there is we put together the configuration for the agent and we connect it with a large language model. Obviously we did that too. Nothing else, but we also enabled Trace. That was one of the things that was inside the code, which is now not anywhere to be seen on my laptop. But in that configuration window we had this thing called Trace Enable. So you don't have to have that on if all you want to see is the output from the agent. And that's great. But as a developer, I absolutely want to see the trace because I want to know what's going on and I want to be able to sort of figure out if I need to do any troubleshooting, such as my agent doesn't know what to do because it's got no tools or I don't know, my Internet. It's having a good go, isn't it? Fast Internet seems to be there. Interesting and lots of fun. I use fun in a loose sense there. We'll try and get back to this. All I can do is refresh that. If anybody's got networking experience and would like to suggest something, then let me know. Congratulations, you unlocked a QR code is what I was going to say here, but I don't feel quite so excited. It's all working for all you guys and that's the most important thing, right? Yeah, not me. I've seen it before, like I said, so I appreciate that it's working. It'd be interesting to know how I can talk through the rest of the code if I can't see it. But we'll figure that out in just a second. Okay. Actually, maybe if anybody who's like from aws, who's like feeling very sympathetic has a laptop on their. If they can see it, then that would be awesome. You can pass it to me. Now they're all scrambling off because they didn't bring their laptops with them. QR code with a. That's very kind, but I feel that's unfair. Let me see if I refresh one more time and I'll back to you if I can, but thank you. So, yeah, QR code here for the link to the Amazon Bedrock API reference. Yeah, it might be worth just in case. Oh yeah, that would be awesome. Thank you. I feel better about taking yours. Although you did tell me before that yours has trouble sometimes. Not because you have.
