How to build a ROBUST AI Agent stack [CrewAI + YouTube API + Ollama + Groq + AgentOps]
Build an automated AI pipeline that pulls YouTube comments, analyzes them, and generates reports using CrewAI, Groq's Llama-3, and AgentOps. Three agents work in sequence: one fetches comments via YouTube API, another categorizes them into requests and complaints, and a third writes a markdown report. Deploy it as a protected API with CrewAI Plus.
Chapters
In this video, I'm going to show you the new and optimized way of creating CrewAI agents. And on top of that, I'm going to show you how you can add an observability layer on top of these agents. That way you're able to see what's actually happening under the hood with these agents. So with that, let's just jump straight into it. When I'm creating content, I find myself doing research on people that have kind of created similar content to what I'm trying to create. What I usually try to do is I go to the comments after watching the video, Because I think the gold mine is actually in the, in the, in the comments. If you're a content creator, you should always go check the comments of competitor videos and the viewers there will tell you the truth on what they liked, uh, what they didn't like about it. Also what they would've liked the video to cover or what, you know, things should be covered next. So I usually go there, find some good snippets, find some good, amazing, uh, feedback and try to incorporate that into my video or in my next video that kind of is similar. That was a very manual process. And so it took me about 15 to 20 minutes to just go through the comments of one video, especially if it's a very good video that a lot of people left a lot of comments. At that point, I realized I could just use agents to do that work for me because it's a very small task that could be automated. So why not, right? This is what I'm building for this tutorial, actually. So for this tutorial, I have 3 agents up here, as you can see on this architecture diagram. So I have a fetch comments agent. I have an insider analyst Uh, insight analyst, sorry, not insider analyst, uh, insight analyst. And then finally a report writer. This first comments agent, what it does is uses a custom tool which I built, and also it's part of the CrewAI. I'll show you how to set it up, uh, pretty shortly. So this custom YouTube tool, it goes and fetches comments for any given— so you pass it and you give the agent a URL. It just extracts the ID portion of that YouTube URL and uses that combination with the YouTube Data API. To fetch comments for that specific video ID. Once it fetches those comments, it saves the comments in a comments.md file. Then after it fetches those comments, it passes it over to the insights analyst and then it goes through all the comments, figures out, okay, what are the interesting things here that I should be able to pass it on to the report writer who then finally writes an entire report that I'm able to just read really fast and figure out, okay, this is what the viewers in this video are asking for. This is what they didn't like about it in this video. This is what they expect. So that way it kind of informs me on what things I should focus on when creating my content so that the viewers get a better quality content. In this case, this is just a simple architecture and we're going to go dig deeper into how to build this. But the important thing as well is I'll be showing you the new and updated way of doing this with CrewAI. And down here, as you can see, CrewAI of course is what we'll be using as our agentic framework in this case. And then AgentOps, which I haven't talked about, I think I mentioned it in the intro, which is the observability. It gives you an inside look into what the agents are actually doing underneath. And another thing here that we'll be testing as well Is all ammo. So as always, as you know, I try to find ways to find free ways to use it. So we'll be testing with Grok as well. Now that we understand the architecture diagram, let's switch over to the code and let's start with first of all setting up Crew AI the new way. So the first thing you want to do once you're once you're in your terminal is install Crew AI. So we'll do pip install Crew AI. The next thing we want to do is use the new scaffolding method. It's very easy. It sets up all your files that you will need, and all you have to do is just really edit them. And it's Crew AI create. Let me name it as YouTube Yappa track. Traps the yaps. Let's name it that way. It's going to start creating the scaffolding right now. And you'll see it, all those folders that we will need to be able to run the crew. Let's navigate to that and open that folder. Let's walk through each of the directories. And I usually like doing this because it's always good to see what's on the directory side of things first before you even start coding. So in this case, let's see what this scaffolding contains. First of all, it starts with our source file here. And within the source folder, there's the yapper trapper agents that I've already created. And now under the config file, there's agents.yaml and tasks.yaml. And this is actually a new way using Crewe AI where it kind of abstracts the things that you usually include in your main folder. What do you usually have in your main file? You add these things like role, goal, and backstory. So these were things that you would have to bake in into the main code of what you used to do. But now this is abstracted away by using YAML files, which I think is really cool way of abstracting things away so that way it's easy to edit. So if I needed to edit some specific things, I don't have to go to the code. I could just come edit the .yaml file and then save it and it's gonna be parsed into the agent and it's gonna tuck in the new updates. That's the agent one. And this is the tasks. And this is just a boilerplate of what it gives us. I'll be copying code and putting it in here. So that way we can see how it works pretty shortly here. And then the next part here, we have a tools folder. There's a custom_tools.py file here. And this is where you create your custom tool. In my case, I will need a custom tool because my agent will be using the YouTube Data Version 2 API to grab the comments of any given YouTube video. This is the crew.py file. And then inside the crew.py file, this is where you create your agents. This is where you instantiate your agents. And there's new decorators now within CrewAI. There's CrewBase, which is the class that kind of handles all the agents and tasks. And so there's also agent decorator just to show that's an agent, that's an agent. agent. And then there's also a task decorator as well. So that's for decorating the tasks. And also finally, a crew decorator that decorates all the crew together combined. Now that we have that done, let's go to the main file, which is kind of usually the one that kind of runs all the programs within your codebase. And so here it says run and you have to pass in an input. There's also a folder here for tests. So if you're trying to push this to production, you definitely need tests. And then there's the .env file here. This is where you save all your environment variables. These are the secrets actually. And then the .gitignore, of course, this is where you put all the files that you don't need to commit to GitHub. Pyproject. This is also another interesting thing. I hadn't used Poetry myself until I started working on this project. So it was a good learning curve, but it seems pretty easy to implement. I'll show you what I've learned so far to make it work the way I want to do it. Finally, you have the README that's auto-generated as well for you, which is really cool because then it provides you all the steps that are kind of hardcoded to your project. So if you always struggle with READMEs, this is a good time because it does it for you a little bit. Then all you have to do is just customize it a little bit if you need to. That wraps up the directory overview. Now let's get to coding. Also following the same order, we'll start with the config file. I already have this code But everything really generated. So I don't want to code here real time. So I'm just going to copy all the stuff that I've done on the other side and just kind of paste it in here. So we'll paste in here all the agents that I have. And these agents are the ones that we've talked about here, these 3 agents. So I have the comment fetcher and the role I give them, the role is a senior YouTube comments fetcher for tech videos. You can name it really whatever you want. You have to be really explicit with some of this stuff. So fetch all comments from specified YouTube video links. The backstory for it also, just so you're aware, as you can see some of this stuff here, it sounds like AI generator, right? Chat with CrewAI docs. So what I did, I just pasted what I wanted that agent to do. And it did generate a bit of all this text that you're seeing here. Insights analyst here, their role is tech insights analyst. Again, you can name that whatever you want. And then the goal is generate and synthesize insights from YouTube comments on tech videos. And then here, backstory is they're sharp-witted analyst, passion for tech sector, skilled in navigating through noise. And one thing also that you might want to keep in mind is for you to know which ID you wanted to run. So I put this in brackets, video ID, and you'll see that this is what we will pass as input. When we get to the main file, so essentially this video ID here is what they will need to use to grab their YouTube video comments. And so this is just passing it explicitly there for them to remember, so they don't forget because they sometimes tend to forget. Of course, like we all do sometimes, right? And then finally, the report writer here. Their role is tech report writer, and the goal is write comprehensive reports. Of course, of unsynthesized insights from this video ID. You're skilled communicator specializing in tech industry blah blah blah blah blah. And so the agents are saved. Now let's go to task.yaml again. This is what we are referencing, just so you're aware. So fetch comments task, analyze insights task, and then generate report task. And so for fetch comments task, remember we also have to save these comments to .md file. The description here is retrieve all comments from this YouTube video ID and also ensure you've extracted all viewer comments from this video ID. A JSON format list of all retrieved comments, each entry representing a single comment. This is the expected output as well for this agent. The insight task, it's supposed to analyze comments, retrieve actionable insight from the comments as well. And then the expected out JSON list of key insights, categorized themes, And then also categorize them by themes like requests, complaints, suggestions, and other relevant ones they find useful. So I'm giving them some room there to be able to decide what's actually important for me to know from the comments. And so it's going to generate requests that people are requesting, complaints, suggestions, and any other things that I might— I want to be aware when I'm creating my new content. And then reporting, finally, it just kind of takes all these aggregated insights from this and then creates a really full-fledged report for me to be able to digest. The next part now is tools. Let's create a tool that the Fetch Comment agent is going to use to call the API to be able to get the comments for any given video. We're importing the base tool here. And we're using that here within the class that we'll be building. The name of this tool is a YouTube comments fetcher tool. And the description is fetches all the comments from a specified YouTube video URL using the YouTube Data API. So we're creating a comments variable here with a list, empty list that we'll be able to pass in the comments there. Once they fetch, it just kind of gets stored in there temporarily. And then the page token here, it's blank for now. The reason why I'm doing that is because The YouTube API, which is this API right here, v3 comments thread, it only allows you to retrieve 100 comments at a time. So if your video has like over 100 comments on it, it's not going to retrieve all of them in one API call. So it's going to get the first 100. And after each 100, at the end, it's going to have a page token. It could be like a number. It could be, for example, a number like 9, I don't know, 5, 1. And you need that to be able to call the API again with that number to show that, okay, it's more like telling the API there's more. So here's the token. It looks good. Go back and grab the next ones. So if it's maybe like 300, for example, you're going to have a first token for the 100, another token for the 200, and then you're not going to have another token after that just because it knows that this is the end page. So there's no more tokens. I have a self-validate video ID first. So when you pass in the URL, so I wanted to be able to validate that video ID that it's going to extract from that URL. And that video ID is just checking to see, and that I have that function here that does that validation, right? So here, validate video ID. And so it's also another YouTube API and all it's doing is just getting the ID that you passed in and make sure that that video ID is valid. The next part here we're doing is while that is true, while the video ID is good to go, we also want to be able to pass in the YouTube API key. Video ID, of course, we're passing it in. Max results is 100. And this is the max you can actually pass in there. And then we have the page token. If there's a page token, it's just going to put it in there. And then it's going to, of course, if it keeps rerunning it, it's going to rerun it until there's no more page tokens. And here we're just getting the response back from the API. And if the response from the API is good to go, which is status code 200, save save that data in a data variable in a JSON format. And then it's going to iterate through it and append each comment over into our comments variable up here that we had already instantiated. So it's going to save it in there on each comment. And then it's just going to return that. And if there's an error, it's just going to show there's an error. If that doesn't work, if there's no page token at the end of the last page, maybe the comments are done, it's just going to break. If the status code from the API is not 200, which means something failed, it's just going to show the error here. And all the error codes here as well, I have the handle API error here as well function. It details all the possible error codes that could happen in this process, but I wanted to list it out specifically so that way the agent can know exactly what caused it to happen and communicate back to me so that I know exactly what caused it to fail so I can correct it. The next part now is the crew. So the crew part is where we instantiate all the crews now. We create the agents and also the task, try to orchestrate them now. In this page now with the crew.py, first of all, we are importing the tool that we just created up here, which is that custom tool. Our agents will need that. And then from CrewAI, we are importing all the necessary stuff that we need, which is agents, crew, process, and task. And then CrewAI project, we are importing all them as well. And then here, something cool as well, We're importing LangChain chat OpenAI, and we'll use that when we are running the agents using Ollama models. And also I'm importing LangChain Groq, and this is chat Groq. This is when we'll use this when you're using the Groq model for the agents to use to interact here. Finally, here, the observability part that I mentioned earlier, which is using AgentOps, and we'll get deeper a little bit on this when we start going here. And so the AgentOps portion here as well, we're importing AgentOps as well. And it's very easy to set up AgentOps. It's very automatic in this way. So all you have to do is import AgentOps. You, of course, you have to install Install it first and then import it and then initialize it, agentops.init. And it's going to automatically track all your agents so you don't have to do anything else. And if you want to be a little bit granular with letting AgentOps track your agents, you can also track agent name here. And so you can put track agent and then the name, you put the name of the agent on top of each agent. What we are doing first is we are gathering all the YAML files that we already created up here for agents and tasks. So for agents config, we're grabbing the agents and then tasks as well. We are instantiating the LLMs. So for Groq, we're just saying, Grok LLM and then chat Grok. We're using that. We're setting the temperature to 0. We don't want it to be super creative here because the higher the number, that means it's more creative. Use the data that we've already given you, which is the YouTube comments. So don't make up anything. We just want you to use those ones and provide us insight based just on those. Grok API key, which is grabbing the API key from the .env file here. And we haven't added that yet. We'll add those in here shortly. The model name that we'll be using here, I'm using Llama 3, which is the new model. I'm using the 70B. You can use whatever one you want here. We also insert here Ollama. And then I have Ollama LLM chat OpenAI. And in this case, I'm passing Mistral as the model. Also just passing in the base URL and the API key. This is just really, you can put something random. It's required to have something in here, but it's not going to be used. It doesn't have an API key. We're also setting the temperature to 0 on this one as well. We have the track agent. And I mentioned to you, this is the one AgentOps will use to track this specific agent. When we get to the dashboard later on, we'll see we can pick by even the agent and see what that agent specifically did. And this is what we are doing. So we are doing that for the agents. We're doing that for Track Insights agent. And then also we're tracking Report Writer agent. We're also having the decorator as agent here. And then agent responsible for fetching YouTube comments. Very simple, right? So all we're doing, config, we're getting the config, which we've already imported from our YAML file here. So this is the agent config file. And that's our agent. We're putting that here as comment fetcher. And then the tools, it's going to be that YouTube comments tool that we just built in here to call the YouTube API. And then allow delegation, we're telling it false. So we don't want it to delegate any tasks to anybody else because that's his job. It's just going to do its job. Nobody else can do its job. And then here, LLM, we've instantiated using Ollama LLM. But in this case, I want to use Groq first because Groq is a little bit faster. It will allow us to do things a little bit faster. So let's do that first. And then we'll test Ollama as well. For verbose true, that means we just wanted to print out everything that it's doing. For track agent, for this next agent, Insights Analyst, we are tracking this Insights Analyst. And then again, we're instantiating it at decorator for agent. And what it's doing is agent responsible for analyzing comments and generating insights. We're also returning an agent here and the config file as well. It's insights analyst. We're just passing that to him and then, or her or it. And then LLM here as well. Ollama. I just want to change that to Groq as well. Delegation again falls verbose true on this one. And then the last agent here, track agent name, report writer agent, report writer. Again, it's just supposed to write a detailed report and analysis. And so agent again doing the same instantiation. I'm just going to change this again to Groq and now to the tasks. So for the tasks again here, what we are doing is fetch comment task. We're starting with the first comment task. And again, we're just passing in that config Task comment search. And right here, so this task config, we're just passing that now to fetch comment tasks. Agent that it's going to be responsible for doing this is also the comment fetcher, passing that in. And then the output file, what I mentioned earlier, that it's supposed to grab all the comments, put it in a .md file, which is called .comments.md, and also pass these comments to the next agent. And now let's go to analyze insights tasks, analyze comments and generate insights. We also given it the task.yaml file. And then the agent is insight analyst that is responsible for this role. And then also human input. Yeah. Very interesting part about CrewAI as well. As the insights are being generated, it'll show you exactly what it's generated, like a preliminary report of what it's generated, what it thinks it's going to give you. And it'll pause first of all before it passes that to the report writer. And it'll ask, do you have any feedback? And you'll read through it. If you have any feedback, maybe that report doesn't sound good to you or you want something tweaked, you can just tell it, this doesn't look good. I want you to change this and that. And it's going to self-autocorrect in this case. Generate report task to generate create the final report. And again, we pass in that config report writer, who is the agent that's going to be in charge of this role. And then the output file is report.md, which is the final report that I'm going to be able to use to read and create more content, right? Better content. In this case, we're just putting everything together now. We've instantiated the agents, we've the tasks, we're just putting all the tasks in a list here. And then the process we're doing sequential. And sequential, what it means basically here is this has to run first and then the next one has to run and then the final one has to run. So you can't run them parallel because this depends on this and this depends on this. So they're all very interdependent. So the comments have to be there first before the insight analyst analyzes the insights and then the insights have to be there before the report writer writes the final report. We have also memory, which I set false. There's so many other parameters you can pass in here, but in my case I put it as false. I don't want it to retain any memory here. And then the max RPM here and this is really throttling it. So you can do 1 or 2. We also have verbose to 2 verbose, just printing out everything on the screen as well. So this is our main file and what we are doing here, first of all, we are importing all the YouTube comments crew, which created here, .env, loading the secrets, which are the API keys that we'll put in here after we finish going through this code right here. And then also we are instantiating agentops.init. And so import agentops and then agentops.init. And so we have this extract video ID function here. And what it does, it takes in a URL and it's going to use regex to be able to extract just the video ID out of that URL. And this is what I mean by video ID in a URL. So on a YouTube URL, you have this URL. And then at the end here, you have mostly the video ID. All we're doing is The YouTube API will need that video ID only. It doesn't need the full URL, just a video ID. It's going to need that to be able to generate, to grab the comments for that specific video. And so all we're doing is we're extracting that video ID from the URL. And it could be a URL that's normal, shortened, or embedded video ID. Once we've extracted that video ID here, it's going to take in here as an input. You have a dictionary here where you're going to pass in your input here. So in this case, it's just the video ID here. And now we're going to create a crew, which is from the crew that we already imported up here, YouTube comments crew. The result here, we're just going to do a kickoff of the crew, entire crew. And then we're going to pass in the input. Input that we already have here into this agent to kick off. And it's going to do its thing. And finally, when it's done, it's just going to print all the results here on the REPL as well. Now we go to the cool part as well, which is using API keys now. So this is where we save the API keys. It's under your .env file here. Since we're not going to be using OpenAI because we're very cheap, we try not to pay for nothing, we're going to delete that. And don't worry, I know somebody's going to in the comments will tell me about how I need to delete all these API keys after I use. Yes, I'm going to delete this before this video goes up. These are the API keys that I'm going to be using. There's one here for AgentOps. You're going to need one from AgentOps. It's free for now. So everything that I'm trying to show you is pretty free at this time. If anything changes, you'll definitely find out when you get to their website. The important thing is you can get started with AgentOps for free. If you click that, you're able to then come in. Once you log in, you're able to come in and find your API keys very easily here. So if you come to settings, you're able to come to your keys and secret and you're able to create your API key here. And you can also delete it or revoke it if you wanted to. So for YouTube as well, it's a very easy process that you can look at YouTube data API, and then it will show you exactly where you can go find your API key and be able to load it in and be able to use it. So I don't want to get into that because it's going to make this video way longer than it should be. If you're still struggling with setting up the YouTube portion part of it, just let me know in the comments and I'll be able to— I'll try to help you out as much or give you pointers where— how you can do that. Now this part is set up. Let's save the .env file and let's get to the point where we actually run this code and get to see what it outputs. And also keep in mind that we passed in the Groq, which is the Llama 3 using Groq. And that's what we are using here. I need to set the environment first, which I didn't. So let's go ahead and create a Python virtual environment here real quick. And as you can see, now that we have a virtual environment, we are getting a bunch of errors here that we need to import a bunch of stuff here. So we need to get that set up. pyproject.toml here, which is for Poetry. And you can see that some of the stuff that have already been installed here or things that we need to install. There's a CrewAI here with extra and tools version there. And then this is our project right here. So the first thing we need to do is install Poetry. You're still feeling lost, you go to your README, you'll see Exactly what you need to do here as well. So we don't want to do lock first because we're missing AgentOps, LangChain, Grok as well, and some stuff here to get started. So we'll do poetry langchain grok because we're missing that, and also we are missing AgentOps. We should see them getting added here shortly once you do add. It's been added in here, so you can see there's CrewAI that's been added, LangChain has been added, and then AgentOps has been added. Now that's done. The next thing you want to do is lock the dependencies. So we do poetry lock. I'm just copying it from the README. And then we'll do poetry install when that is complete to install our dependencies. For us to run this, we need a YouTube URL, right? Let's see videos about CrewAI because there's so many of them that have been done before. I'm trying to find a medium-sized one. So this one was a recent one by Matthew Berman. All right. So we're going to grab this URL for Matthew Berman and he has 276 comments. So that's quite a lot actually. Let's go over to our terminal and run poetry run. And then we just need to name it youtube-yappetrapper. Then we tell it run. So it's going to kick off and ask us to input the URL, which should be coming up pretty shortly here. Cool. So we paste it in there and hit run. And now the agent fetch comments is going to start fetching the comments. And there it is. It just fetched all the comments real fast. While we're waiting for it to pass these comments to the agent and create the comments.md file, let's go up here and see some of the output that's happened here. But now here, AgentOps has given us a URL that we'll need to go to their dashboard and be able to see what's happening with our agents that we just created. This is the dashboard. As you can see, This is a session drill down. We're drilling down to this session that we're running. And so right now nothing is happening. While that's going, let's go back and go over the AgentOps platform. So AgentOps essentially is an observability platform like I mentioned earlier. It gives you insights into what's happening with your agents. It also gives you your run environment, which is the name of my system here. Here are the agents. So we have an error here and it says rate limit reached, error code 429 with our Groq API here. This is very common, especially if you have a lot of comments and it has to like parse through all of them and be able to get insights from all of them. So it's going to hit that rate limiting with the tokens. But for now, I'm just going to need to pick another YouTube video that maybe doesn't have a lot of comments. And in this case, I'll just use mine, right? Because I don't think mine has like a lot of comments. So let's go to this one. And this one has 150 comments. This should be good, right? So let's use this instead. Let's go here and rerun this program again. It seems like it has gone through. So I tried using the same input. I might stop. So you can see here, it's just saying, hey, I need to fetch all comments from the specified video ID. And it shows the video ID that it extracted from that URL. Well. And then thought. And now we have another error. Debug working agent Tech Insight Analyst. Starting task analyze comments you receive. Generate actionable insight. So it passed those comments to the Insight Analyst. But now we're getting another rate limit. Now let me even use a much smaller YouTube video that has very less YouTube comments. I have 41 comments. So it better work on this one. Let's try this one. This should be small enough that it should pass, you would think. So let's see. So that went by fast. So I'm guessing this is good. And it already passed all the comments over to the tech insight analyst agent. And the insight analyst agent is saying, here it is. So I think we're good. So it did gather all the comments and I mean showing, all right, and here you're saying task, analyze the comments, blah, blah, blah. What exactly I told it earlier what to do. And then so it's doing, it's entering here. This is the agent's final answer. Here's the JSON formatted list of insights of key insight characterized by themes. And here's the human input that I mentioned earlier. So it says, please provide a feedback. So in my case here, I'm just saying it looks good to go. Right. So now that I've told it, I've gave it feedback, I told it it looks good to go. It's gonna get all these insights and pass it over to the report writer. And the report writer is gonna generate a report for us. If I had an issue with it, I would've stated there, hey, maybe restate this word this way or do this or fetch more insight. And maybe it could have done another iterative work there to fetch maybe better insights. It's already passed it to the tech report writer and it's generating report. And here's the full, the final report now. So we have the comments first. So if you see here, let's just take this down. So these are the comments that it retrieved for this video. This is the full report now that it generated. So here it's just saying, saying production, this report provides a comprehensive analysis of the comments for this video. And it's also giving that video there for me to reference if I have this report and I forgot what video it was. Following requests were made by viewers. Combine this video with previous video on local PDF. So excellent, possibly in future. And then add a PDF uploader that can learn from documents. This makes offline models. Okay, so these are actual comments, the insight and the actual comment. There's a claim and a proof. And then complaints. One complaint was made by a viewer and it says emojis are unnecessary. That's Funny. Yeah, this is somebody that I ended up chatting a little bit, and he said, "I appreciate the video, but emojis don't make anything cool." Haha. Yes, it doesn't make anything cool unless it's mine, right? Anyway, but this was a streamlit in relation to a streamlit app that I created, and this streamlit app basically I added Notion emojis that made the app look a little bit cooler in my opinion at the top, and then there's a title. And the person jokingly said this; he didn't actually mean it that you know it doesn't make anything cool. He just meant like, "Hey, you know it doesn't make it cool." It was more. Funny. And you can see the interaction as well on it. So that was pretty cool that it did catch on that. And then suggestions, show people how to use Conda and provide sample connects in Git. Yeah, pretty good. And it cited that and that's correct. And I'm saying that because I've read those comments and I responded to them. So I know exactly if it's stating something that's wrong. So these are exactly the comments. I don't want to read through all this, but you can see this is like pretty cool. Like this is good insights. I didn't have to spend 20 minutes reading through a YouTube video, right? All I did was go find that YouTube video, get the URL, paste it in there. Boom. It did a good report here for me. And I can save these reports somewhere that I can reference when I'm creating that Eventually. So if I have like 6 videos that I'm trying to reference, I can just grab all those YouTube videos, paste them in, let it run through it real fast, generate a report, save it somewhere while I'm doing something else. That's the key thing. It's agents. I'm supposed to be doing something else and not worrying about this. All right, so on the AgentOps platform, I'm just going to show you what I ran earlier and a few days ago as well. Here the cost, there's a 0.30 cost. I use GPT-4 here, which is the default for CrewAI. So if you don't add your LLMs like we did with adding Groq and Ollama as well, it'll just default to GPT-4 if you don't add any models there. And it's going to ask you for your OpenAI API key too. To put it in environment. And here is the average cost per session. So that's how much is costing you per session of agents running. And so here's showing you kind of like all the end session reasons. So that was 429 for the rate limiting and then signal surgeon detected. That was more of like if you hit Ctrl+C and kind of like cancel the process running. Yeah, these are just some of like the debugging errors that I was getting here early on. And so let's go ahead and now run using Ollama since we've run through here using Groq and we saw how fast it was. It did run pretty fast other than the rate limiting. It seems like everything is good. So all I'm going to do here, I'm just going to change all the LLMs that I'm passing to the agents here to Ollama. I've changed all of them to Ollama. I'm going to hit save. And for Ollama, we are using Mistral. Find another video that I have created as well. Let's use this one. This one has 27 comments. Let's grab that one in this case. We also noticed that it takes quite a while for the Ollama models to run on the system because of course there's a lot of things that's competing here for memory. Mistral is going to be taken over. And here's the AgentOps link that we are getting as well. You can see there's 2 events running. And here you can see it's pretty cool because We have the two events. So one count, it ran for four to eight seconds, four to five seconds here. Call summary, you can see here. It's first of all starting with the user, and this is what it passed in as the user prompt: "Your senior comments fetcher blah blah blah." This is what you need to do. Basically instructions. And then the assistant said, which is the agent, "I need to retrieve all comments from the YouTube video specified." So it did find an invalid syntax with an escape character that's causing the problem there. I don't have time to go fix that portion as well. Well, so let's find another video. Tech team has 88 comments. It should work since this is Ollama anyway. So let's copy tech team's video here, video ID, and just pass that to our agents. Passing that URL this time, it should work. And I'm glad I'm getting all these errors. So you, as you're watching this and you're building, you're like, okay, at least it's not like a clean tutorial where everything just magically works, right? I never hit any errors. So if you hit these errors as well, at least you can see how I troubleshoot it and work through some of them. Stuff myself. So we should not get rate limited on this one because we're running on Ollama. While that is running, let's go ahead and go to that drill down session. It's already finished the first process, which it only ran one count so far, which was just passing that video ID, which it grabbed that from that. And you can see that's the one process it ran. It's Mistral again. And these are the results that it's— this is the composition call that's happening right now. And so while we wait for that, let's refresh the page again and let's even do a drill down on the agent. So we do the first agent here is the common fetch agent. And you can see this is the one that's done the most work right at the moment. All right. So everything is running so far. It's just going to take a long time. Like I mentioned, Ollama models are pretty slow at this point. And it's very dependent on your system as well, mostly. So my system right now is not super optimized. I don't have a GPU. Everything is running just on my CPU. And it's probably why it's slow. So we're just going to take a minute. I'm going to cut to where everything has run through and it's good to go. All right. So It's already finished to the portion where we are at the Tech Insight Analyst. And so the Tech Insight Analyst has generated some themes here. And we have the request here. It seems clear instructions on installation usage. And then request examples, templates for processing data, troubleshooting help with specific errors. And then complaints, complexity of setup and usage and lack of documentation resources, slow response times with Ollama. And this is reference to Tim's video, maybe based on the comments here. And then suggestions, integration with popular IDEs or text editors here. Here. And the creation of community forum or support group. And then the person saying a community forum or support group where users can ask questions and share solutions would be really helpful. And popular tech queries, popular tech queries here, we have differences between LlamaIndex or Llama Multi LLM. And then the queries there. So feedback here, I don't see a section for praise. So let's ask it to add, can you add a section on praise. I think it was mentioned as theme. All right, let's see if it's going to take that input feedback that I've given it and execute on it and provide that section for us. So it's a good test for it. But as you can see already from what it's doing here, I mean, it's good, but it's not quality like the GPT-4 or Llama 3 that we ran earlier with Groq. I mean, it's okay. It's not bad. All right, so it seems it has added that feedback and created the praise section. So thoughts here, as it said, I've reviewed the feedback provided and come to the conclusion that adding a praises theme would be valuable as there were several comments expressing appreciation for different aspects of the video or the tools discussed. Now let's go see those praises that it's generated. So here it says praises theme, praises sub-themes, praises. Here it says appreciation for the clear explanation of concepts. It says, I really enjoyed this video and found explanation of LlamaIndex or Llama Multi LLM to be very clear. And then here, positive feedback on the presentation style. The presenter did an excellent job explaining these tools. I learned a lot. Boom. So you can see that it did a pretty good job of taking that feedback. Now let's see if it's already passed that to the report writer. Now that is the next step. Now here it's passing that to the report writer. Again, I'm just going to fast forward here to the point where the report writer has generated a report. Also, while the report writer is doing its job of writing, let's come here and review the steps of what's happened so far. So here on AgentOps, you can see there's already a number events and no errors so far, which is amazing. That's what you want to see at all times. If you come here on the event distribution, you have all these different runs, which is 5 runs. Then the event types, which is LLM 5 count, then repeat thoughts. Then these are the thoughts that have been repeated throughout the entire process here. This is the full chat, which we've already seen in the REPL when it's running, and we've been reviewing that as well. This is the entire conversation here that's happening. As you can see, this is where I gave it the feedback and it went ahead and generated that feedback and generated better content. Here you can see the first replay, if you see session replay. So this was the first session here it generated. And you can, once you click on it here, you are able then able to see it here exactly what was going on. And of course, the model is still Mistral. And so it's showing you exactly what happened there. And then if you come to the next step here, it's showing you the next portion of what happened, it fetched all the comments there. And then we see here kind of reviewing it and sending that over. And then finally getting that over to the insight analyst. And now to the report writer here in case, who's kind of generating the report right now. So that one is still going. You can refresh again. It's still 5. So let's go to the report and see if the report is ready. All right. The report writer has finished writing the report. And you can see here, report Tech Inside, and these are the results. It's already generated a report and it's printed out here on the screen. And it's also saved it as a markdown file as a final report. One thing you can see here from AgenOps, we're getting this run cost $0. This is the report that it generated now. So we have the request here, Clear Instruction installation and usage, and exactly Exactly. It's tagging the exact citation of where that comment relates to that insight it generated. You can see examples of templates processing and all this right here. There's complaints and you can see the complaints here. Complexity setup and usage, all that stuff, link to documentation resources. Then there's suggestions here as well, showing the suggestion there. Then the praises, which we did ask for it to be able to add that as feedback. It's showing you the feedback here, praises that it got from the comments. Then the popular tech queries, it's showing you that here as well. This looks pretty good, although it's It's not like as detailed as we want it to be because then it should have like a conclusion here with like a full list of summary. But I don't think I was super explicit about that in our YAML file. So that's something else that I also want you to note. Be very explicit when you're writing this YAML files. Be very explicit. Tell it exactly what you want it to do. Otherwise, it's just going to generate some random stuff for you. And if you don't provide it extra exact feedback, then it won't do that. Let's do a quick test here of using, and we can just use the same video ID, I think the video URL. Yeah. So let's use the same video URL just to test it out as well. And so it's going to start here. The agent is starting Of course. Now we have Llama 3 running this and Llama 3 is trying to do the exact same thing that Mistral did. So we're going to compare the output here and see if it's going to do a better job in terms of the content itself. If it sounds good, if it sounds like it synthesized the information better than Mistral. And this is just a good benchmark for the same exact same report and see how that does. And so I'm just going to skip this video to the point where it's already completed this so we can do a comparison of this. Now this is the output we're getting from Llama 3 learning locally on Ollama. And this is the report that it generated. So like we went through earlier, it does look the same really. And the formatting is not as the best as Mistral did in my opinion. You can see it did some just decent work, like just like the other one really. So there's not much difference here. So I also want to bring to your attention that CrewAI now has CrewAI Plus, which essentially allows you to turn any crew that you've created and uploaded to GitHub into an API within seconds. So you can connect to your apps using webhooks, REST APIs, gRPC, and more. And recently I got access to it. And this is what my account looks like. When you log on, you'll see your account looking like this. And the first thing you would do then is connect your GitHub to your CrewAI account. And your GitHub repository is where you have your crew already. And this is all my GitHub repositories. I gave it access to all my GitHub repositories. So you can also change that into just giving access to one specific GitHub repo account. And in my case, I created this example crew, which I then deployed here as well. And as you can see the deployment here, I have a URL which I can use to call it and a bearer token, which I can use to interact with the API. This crew agent and pass them an input, and then they're able to process in the background and give me a result. And so if you come here to crew info, there's a status here showing you the crew is online, and then the URL of course, and then the date you deploy the crew as well, and the bearer token that you'll need to be able to call that API. And also you're passing in your variables, which we used during the development here or in our demo as well. We have the AgentOps API key, Groq API key, YouTube API key, and you can in bulk APIs just this way or individual ones as well. Once you're done with that, you would then deploy them and it would look like this when you get to it. You can delete it, you can reset it. So I won't go over this today, but I'll create another video that specializes specific on this. And so stay tuned for that. That's all I want to showcase there. And just keep in mind all this code that I've used to create this crew, which is all this, I'm going to load them to GitHub. If you want to kind of have a starting point where you can start off with, I'll load that there so you can start with that. There's something adjacent to this that I talked about, or you want some clarification on some parts, please let me know, and I might just turn that into our next video as well. So with that, thank you again for watching, and I'll see you in the next one. Goodbye.
More from Tony
Summarize Videos and Research Companies with CrewAI + Gemini Models [Quickstart Guide]
How to Make Your AI Agents More Reliable with CrewAI Task Guardrails (Step-by-Step Tutorial)