How to stream CrewAI Agent steps and thoughts in a Streamlit app [Code Included]

This video is going to be a bit different from what I've been doing before, and there's a reason for that. Right now there's something called CrewAI. CrewAI is a Python framework where you can use agents to do tasks for you, and these agents can do things like trip planning. And these agents coordinate— you give them where you want to go and where you're coming from, and also the vacation time period that you want to go on vacation, and they're going to execute, find the weather, find the best attraction places you should visit, and it's going to generate an itinerary for you all the way up to the dollar amount of the flight tickets and things like that as well. Also like the budget for the entire trip. And I had done this before and I did a very quick screen recording and I posted on Twitter and people really enjoyed it. And I did that using Streamlit as the front end because CrewAI at the moment, there's no like straight UI for it that you can use. So I ended up building one with Streamlit. The main issue with that was the agents take a while to run and generate these responses because they have to go back and forth, search, Generate, pass it to the LLM, give their responses, and they're just kind of collaborating to get you the best itinerary as possible, right? But the issue with that was as it was generating and working together, it was generating this kind of like chain of thought. You're seeing what they were talking about, but it was happening in the REPL, which is in the terminal. It wasn't happening on the application. So you as a user, you were blind. You just saw something spinning and you're like, okay, they must be doing something back there, right? They're gonna give me my details as soon as possible. But that's not really good user interface. So what I've decided now is I want to add, bring those stuff that's happening in the background in the REPL and bring them into the Streamlit app so you can actually track and follow what they're doing as they update so that you can, you know, be aware of what they're doing in the background. First of all, I want to give some thanks to Chris Fang. He did implement the callback agent, the LangChain callback agent to CrewAI, which now makes it easy to get the, this agent processes that are happening in the background and you can call it and use it in your application. Thank you, Chris. So first of all, let me show you the demo and show you how it works. And then we'll jump into the code and then I can show you exactly how you can implement it within your AI agents with Streamlit. So this is the Streamlit application that I'm gonna demo right now. So this is a vacation agent. I just named it some random name, right? And I'm just, let's say, we're just saying, you know, let AI plan your next vacation, blah, blah, blah. Cool. We know that. So on the side here, as you can see, we have where you're currently located. So I'm just gonna put some random place. So let's do San Mateo. Right? And we don't even have to say California. It's going to infer and know where we want to go. And so the city where you're interested in vacationing. So here's what— here's something that I want to try a little bit here. I want to put it my— to my hometown. I was born in Eldoret, Kenya. So let's put Eldoret, Kenya. That way I know— I'll know if it's lying or just bullshitting, right? Um, and that's a good thing to always kind of verify those things. And I'll put the date range at just 3— the next 3 days and see what's happening. And then And let's hit submit. The agent process is starting now, and this is already showing you exactly what's happening. So right now on the— it's showing you action. It's using a tool. The first thing is search the internet, and then the tool input is putting the query into that internet search. Eldoret, Kenya weather forecast for that specific amount— those specific dates that I'm traveling. And so the first thing it's doing is getting me the weather. You want to know the weather so you can dress and pack properly, right? So the log, it says, first I need to gather information about the factors that will influence the selection of the city. This includes weather in Eldoret, Kenya during the travel dates, any cultural season events happening at the same time, and the cost of travel from San Mateo to Eldoret. I'll start by researching the weather forecast for Eldoret, Kenya in late March. And then the action is search the internet. And then the action input, this is the— This is very important. This is the query that is going to put it into the search to get the results. So it's going to say, El Dorado, Kenya weather forecast from March 22nd to 25th of 2024. And then the observation, it's already kind of starting right there. So it's doing the first search and it's using AccuWeather, as you can see, to search the weather. And you can follow the link here and it will take you to that AccuWeather page for that specific period of time that it's looking for. And then the snippet here is saying, get the monthly weather forecast for El Dorado or Sengiju, Kenya, including daily high, low, Historical averages and to you know to help you plan ahead for your trip, and it's also going to climate data org and doing the same thing. It's getting the weather averages, and you got that already. And then it's going to weather twenty five. So you can see it's going through a lot of them to get the averages, so it can give you more more like an accurate assessment of what the weather will be like. So the next thing it's doing is Eldoret events calendar, and it's going to this website to look for events within Eldoret and. The snippet here is saying, explore all the happenings, all the happening events in Eldoret in 2024 with us that best suit your interest. There's theater tickets, comedy festivals, music classes, any, or any adventure really. So trying to find you the, you know, maximize your fun while you're there, or while I'm there, because I'm the one looking this stuff up, right? And then the other title, conferences in Kenya. So it's trying to locate to get you some conferences, some business, technology, science, whatever. And then the next one is endurance races. Duh, you're in Kenya, so you should be running, right? Essentially. But no, so it's looking at that as well to give you some good events that you might find really cool. And the other snippet, it's going through upcoming events in Eldoret, Kenya, and it's going through that link. And as you can see, this is just showing you this thought process that's been programming there to, hey, look for this, find this, aggregate, and so they can create the ultimate itinerary for you, the best itinerary for you. And then so there's snippet here, travel, tourism, youth conferences, Nairobi, gospel concert, mental Health conference, international conference. All right, so many things here. And then they also—it also goes to Eldoret Hub Entertainment Hub Facebook. All right, cool. Also going to some of the media outlets. There's a standard media publication there. It's looking for some things they are publishing on there about Eldoret. ASK—that's ASK—that's a showground thing, agricultural. All right, yeah, it's thinking, saying yeah, maybe I should refine, you know, to get more results. But as you can see. Oh, boom, it's done. So first of all, for example, if we didn't have this events happening and showing up on the application, you would just, you probably would have just seen a spinner spinning on the page and you don't know what's happening really. You're just like, okay, let me just wait. And it could have taken that a long time and you would have waited and finally got this, maybe moved out of this page, go to another page, go do something else. But see, I fell for it as well. I was very engaged that I forgot that this thing was running until I just, it just popped up on the screen that it finished. I didn't feel the time going because it moves so fast because I'm seeing, I'm interacting and seeing what it's thinking, what it's doing. But as you can see, it went and found a whole lot of stuff, tickets, flights, everything. And then it aggregated all these results that you're seeing in here. And I put this in a container so that way you can scroll all the way down, all the way up, and you can see what, What he came up with, and all the way to the end. So observation agent finish, and it's done, and it's giving you all the breakdown. So that is that. And as you can see, I put that in an SD status, and I'll show you the code for that. So the SD status is basically it updates the status. You could before when it was running was saying something else here, like the agents are at work. But now when it's done, it says trip plan ready. And of course, at any point, you can just kind of you know so long as you're still within same page after that specific run, you can open here and inspect. the steps that it took and make sure that it did the right thing. So this is just a working— there's definitely different things that I could do to improve this, but this is a working thing that you can go run with it today and build it yourself. And that's the point of this video really is for me to quickly put this out because I've seen the demand that people are asking for this for Streamlit on there, especially in that Discord channel for CrewAI. And so here you can see the full trip here. So here's your trip plan and it says, here's your detailed 7-day itinerary to Adore Kenya from March 22nd to 25th. And day one, it's breaking down for you. You know arrival and exploration. Arrive at Eldoret, Kenya from San Mateo. Check in at Boma Inn Eldoret. That's a really good hotel. No promotion really, but known for its comfort and excellent service. And then visit East Kone Temple. I've never been there. It's a serene spot for perfect starting your trip on a peaceful note. I guess you gotta pray. You gotta be peaceful, right? You're going for holiday. Have lunch at the local restaurant and try their pilau. Pilau is If you have never had palau, this is your cue. You should find some palau place, and it's a popular Kenyan dish actually. Spend the afternoon exploring local parks and soaking up the Kenyan sun. In the evening, visit Moi Bay Road for some shopping. Perfect. Nature. Day two is nature day, right? So day one is arrival. Day two, nature. Day three, fam family fun and scenic. Aruba Mall's kids corner. Let's just go through this. Chip Keith Ford waterfall. All right. There's All the best city tours and travel for thrilling adventure. Enjoy dinner at local restaurants. Okay, so I just want to see instances where he's naming some specific stuff. So here he's saying visit Rift Valley in the afternoon for an amazing view. Yeah, there's career views. It's amazing. You can just see the Rift Valley. Yeah, so all this seems pretty accurate to me. And day 7, you're checking out the hotel and departing from Eldoret to San Mateo, you know. And the packing suggestions that it's given us here, consider the weather forecast with temperatures ranging 55 to 80 degrees Fahrenheit. And potential rain. Pack light clothing for warm weather, a good pair of walking shoes, raincoat, umbrella, sun protection like hats, sunscreen, a camera, essentials like toiletries, medication. Okay, so breakdown: flight cost, $1,413. That's pretty standard, it's pretty accurate. All right, accommodation, Bomayene, for 7 nights, $700. Yeah, pretty much accurate right there, about $100 a day. Yeah, and then meals per day, uh, for 7 days, $600 Uh, $76. Man, that must be some nice meals. That's a bit expensive for Kenya, but anyway, you're in Bombay, so it makes sense. Hey, um, and inflation too. So I haven't been there in a while, but activities and shopping, $250. Yeah, so you probably want to buy some, uh, some of those ornaments, uh, carvings and wrist stuff. But anyway, so the total for your budget here is saying, uh, $3,039.52, all the way to the cents. Come on. But anyway, that's pretty good. Like you can see, this is a good rubric. It's not like the perfect itinerary that you can ever find out there probably, but it's a good starting point, right? It gives you ideas of what you might maybe should consider. And then at the end it says, you know, enjoy your trip. It's a rich— it's a city rich in culture and scenic beauty, and this itinerary ensures that you experience it all. I don't know if it mentioned— it didn't mention anything about running. Probably did, probably somewhere in there. But anyway, the main piece that I really wanted to show you today was this part, because I know a lot of people struggled trying to implement this. So I want to show you the code now, how you can do this, this magic right here. It's not really magic at this point, but I'll show you how this— you can implement this part. So let's get to the code. All right, so this is the code that I implemented. So like I mentioned to you, right, uh, Chris Pang did, uh, submit the changes that was needed, uh, to do the callback onto the CrewAI Especially on the latest, that was merged in the latest release, if I'm not wrong. So on the latest release, you should have that taken care of and we can just take a look at it real quick. So if you go to the agents, he did implement that. So he added this base callback handler from LangChain that you can use from LangChain to send those messages to the front so the application can, you can use it however you want on the application. And so he added that and he added a bunch of other stuff in here as well. I'm not It's not too much, but it's a good amount to make it work. And that's the thing. You want a working app in this case. So again, thank you, Chris. And so you don't need to worry about this part. The part that you are interested in and probably watching this video for is how do I implement it within my app? And the biggest part where you want to implement this is within trip agents. So within your trip agents, You have to create a callback handler right here. So I used a streamed callback. I just named it, I just named it streamed callback and you can name it whatever you want to name it, right? And so within this trip agents, this is where you're lining up all your agents. You have the city selection agents, you have local experts, you have a travel concierge and all these agents you can imagine as these are different individual people. Think of them as people, right? You have a travel concierge. This is the expert in concierge. For example, for El Dorado, for example, right? And then you have another city selection agent. This person is like—the role is a city selection expert, and their goal is to select the best city based on weather, season, and prices. And the backstory: an expert in analyzing travel data to pick ideal destinations. That's their job. They're supposed to analyze data and pick the ideal destination for you. And they have some tools that they can use to do that, right? Because they're agents; they're not humans. So. You give them a search tool, and that's for searching the internet, for them to go to the internet, get you know search what they need, get that information synthesized, and use it on the next phase of it. And then they also have browser tools to scrape and summarize websites, so they can scrape summarize some websites to get the data for you because the important thing is data here. And verbose true when we're doing step callback, and we're calling that stream it callback handler. And this is the important piece. So I have this. StreamWithCallbackHandler, right? And what I'm doing here, I'm passing it— this function is going to take the step output. And the step output is going to come from whatever the agents are working on and the results that are streaming in or the interactions, what's happening in the background really. So what we're doing here, once we get that, we're iterating through that, whatever the agent is saying or thinking or information that they're gathering. We're iterating through all those outputs that's coming in from the agent. We find the tuple. This might come as a tuple. So we find a step, and if it's a tuple, we split into action and observation. And for that, and then if the instance of action, the action that it's trying to take, if it's a dictionary and there's a tool within that dictionary as well, and also if there's tool input and there's log, those are like it's a dictionary, and those are within that dictionary. What you want to do, you want to pull out the or grab the action, and within that action you get the tool. So first of all, so so in this case, if the instance action is a dictionary, you want to get tool, tool input, and log from that dictionary. And then basically we're taking everything from that dictionary. We're taking the key, and then we matching with the value and displaying it on the screen basically. So We'll take the, for example, here the tool, and we'll add that with, uh, or concatenate that with the, um, action of that tool. So in this case, for example, you know, you have action. If it's action, uh, we get, uh, the search internet and that's the tool, right? We get the search, search the internet. And then just like that to make it look almost close enough to this. So that way you can read as a human, because if we don't parse it like this, the output that we're getting from these agents, it's going to look very jumbled up and you can't read it. It's just a whole bunch of dictionaries and everything's just everywhere. It's really hard to read when it's coming on the screen. And you want something a bit legible that you can read, your eyes can follow pretty well. So this is what I'm doing basically in all this. I'm just parsing through that data that we're getting from the agent steps. And we're just going through it and organizing it in a way that it's readable. And as you saw in that application that we just ran, it's more readable. You can follow along with it. And this is basically what I'm doing with all this. All the way down here, and once this is done, I'm using that streamer callback within this each agent. I'm just passing that to each agent so that each agent has it, and that's all you're doing within this trip agents file. And then I'm saving that. And then when we go to the main application, which is Streamlit app, you don't really have to add anything here because you already imported the trip agents and you've already imported trip tasks. So within trip agents, you've imported everything, and it's going to include all this stuff that. We've we've added here the callback handler, and so the next thing you want to do, you come down here. What I did down here, the only thing that changed here on if submitted, what I'm doing is I'm using the SD status, and SD status basically is that tab is showing you hey agents are running, and so you know, and it's showing the status running and it's expanded so that way the it stays expanded while the agents still working through it, and. As well, I'm just passing that as status so that I can use it down here again when it's done. I'm just using it down here as SD status. I want to update now the the status the status itself because it went from agents are working. I want to update it to trip plan ready. So when it ends, it shows that check mark and it shows trip plan ready and it's complete. And also the status container since it was open when it was running, when it's done, it's closed. So here expanded is equals to false. So it closes that. So it's closed and you see what you actually, what your itinerary looks like. Cause you don't want to see all that, at least for now, but if you want to do it, you click on it and you can scroll through it and see all the chain of thought, what was happening underneath. And so this trip crew, I'm just passing in the location, city, date range, and interest, which was from the application itself when you were filling in the location where you are and the cities where you're going and also the date range that you want to visit and also your interest. Probably maybe you like snorkeling, swimming, and it's going to consider that and also search for those. So you might want to keep that in mind because that might take a little more longer for the agent to finish running. And so there's a result. So once that's finished running, you're just kind of basically running this and instantiating, you know, you pass in, once you pass in those values in there, what you're passing it into, you're running it. And once that's finished, it updates the container and it says, hey, it's ready to go. And it creates a subheader, say, hey, here's your trip plan and it shows your results here. So finally in Markdown. And so that's the quick dirty way for you to implement this within some of the agents you're building, especially if you're using Streamlit. And I highly encourage you to use Streamlit. I also wanna say I've seen a lot of folks there asking for things like, hey, how do I include human feedback into my Streamlit application, for example, right? Like the agents are working on, I add human input. But then Streamlit kind of does this thing that, you know, you interact with any widget within the application, it's going to rerun it from top to bottom each time. So I'm still looking on a way to help resolve that. So stay tuned. If I find something that works really well, I will definitely share that within, in another video. So this is your cue, subscribe and stay tuned for more. I just wanted to get something real quick out. For you to start working on it. And so with that said, I'm going to share the link to my GitHub repo where all this code is going to be— this code that I implemented is going to be hosted and you can fork it, clone it. I don't know. If you have any issues, create an issue there and I'll definitely try to address it for you as well. And with that, hack around, let me know what you think about it, create something, share it with me, and I'll be happy to promote it as well within my socials. So yeah, that's all I wanted to share today. And don't forget to subscribe though. See you around. Bye.