SmolAgents: A Smol Library to Build Agents

Hugging Face just released Small Agents, which is a library to build great agents. This is supposed to be the simplest framework out there to build powerful agents. Now you have other options like Crewai, Autogen, Langgraph, which are extremely powerful, but this one is only about thousand lines of code and their goal is to keep abstractions to their minimal only for the most important things, which is exactly what you want if you're building a production ready system. Now what sets this apart is that Small Agents writes all the actions in code. So essentially this is a code agent that can generate code on the fly, execute it, and then if there are mistakes, it can recover from it and rewrite the code, which are basically the actions that it's taking. Unlike other agentic frameworks, this approach of writing actions code seems to be better than the existing approaches. And since you are directly taking actions in code, this also to reduce the number of API calls that you're going to be making. They claim that the reduction could be up to 30%, which is pretty substantial. And also you could get additional performance out of your agents. Now, since it is executing code when it's taking action, Small Agent provides a secure Python interpreter to run the code, or there is a sandbox environment that you can use through the EBIT package. Small Agents does not mean it's using small models. It's only that the package itself is relatively small with the least amount of abstractions. It has support for a number of open models, or open weight models to be precise, which are hosted on Hugging Face, but it also has support for proprietary models through external APIs. You can build your first agent only in three lines of code. I'll show you how to get started and I'll also show you how you can build an agent RAC system using Small Agent. Since all actions are taken in code, if there are issues it is able to recover from those and then rewrite codes which will be able to run. They're really upfront that you often don't need agents for most of your work, which is actually true based on my experience as well. If you're working with LLMs and you think you need an agentic system, you want to read this blog post Building Effective Agents from Anthropic. Here they talk about workflows or state machines. If you are familiar with traditional software engineering, these are different patterns that you can use to accomplish most of the tasks that you are going to come across. I have a dedicated video on the topic, I'll put a link to that. But in general you want to avoid using Agents as much as you can. But if you still want to learn about small agents, I'm going to show you how to get started. I'm going to link this jupyter notebook in the video description. So we first need to install small agents. If you want to use the models available on Hugging Face, need to also install Hugging Face Hub. And you will need to get an API key from Hugging Face to access some of the free models that are available on Hugging Face for inference. Next you will need to log in using your Hugging Face token if you're using a model that is hosted on. After that, let's first look at an agent that will be able to do web search. So for this we will need to first import the agent class. That's class that we will be using for implementing our agent. Then there is a tool for doing online search using the web search engine. And since we want to use a model that is hosted on Hugging Face, we will also need to import the Hugging Face API model. Defining your agent is extremely simple. So you create a code agent in tools. You provide the list of tools that you want to use. In this case we want to use only a single tool. You can also define your own custom tools. I'll show you how to do that later in the video. And then you want to provide the model that you want to use. So by default it's using the Qin 2.5 coder 32 billion instruct model. But if you want to provide another model, you just need to provide the model name from Hugging Face and set it as the model underscore ID variable when you're calling this Hugging Face API model function. This will create our agent and the agent has access to a single tool in our case which is to do web search. Let's say we want to do web query and we want to figure out how long it's going to take for a leapard in seconds to run through this bridge. The agent first decomposes our original query into multiple different parts. And here you can see that all the actions that are being taken are in the form of code. So here's the original query. Then it says executing this code. So it figured out that we want to first need to find out what is the actual length of this bridge. So it's going to do a web sesh to figure out the length of the bridge. And then it's going to figure out the top running speed of a leopard by using our web search tool. It does the web search to figure out the length. First you can see Some of these are relevant to the bridge in question, and some of them are irrelevant search queries. But the model is smart enough to only focus on those when it's doing the final generation. The second part is how fast a leopard can run. So it's actually figuring that out by running different search queries again and combining the results to generate a final outcome. Based on the results, it figured out that the distance is about 155 meters. And since there are a number of different values for the top speed of leepird, it is taking the the range and then figuring out the average speed and kilometers per hour. Again, it's doing all these actions in the form of code. So you have the speed, you have the distance, and now you can figure out the time it's going to take for Leaper to cover that bridge. And as a result, here's the final outcome. Another query that I wanted to run was this. How does Deep seq3 model size compares to Deep sequel 2? And what's the percentage increase or decrease from one version to the other one? It does web query for both of them and then writes a code to print those results based on that information. It writes in step number one, another Python code example, the number of parameters in billions for Deep sequel 2 and 3, right? Then the percentage increase is calculated. So we get about 184% increase in terms of the number of parameters. So that's the answer that we get. Now I wanted to complicate this a little bit more, so I added this. How does it Compare to llama 3.3? So that 3.3 part is a little bit ambiguous and you can see that it says new run. So here's the original user query. Then it does a web search which is V3 model size, V2 model size and then llama 3.3 model size. Now here's an interesting thing. That thing that happened, so it got the search query results. Now it's a single string, but it thought that it's going to be a list, right? And that's why when I tried to execute that code, it ran into this issue. So it says you are trying to subscript a string with string index, which is impossible. So then it rewrote the code and now it's processing this as a string and rather than as a list. So this is pretty neat that you actually can see the failures and then how the agent is able to recover from those failures. Now, I think based on the initial query, it wasn't able to figure out the llama 3.3 size. So it wrote some code. Again, it failed, but then it was able to rewrite that code and get the answer that it was looking for. For some reason, the model size for V2 that it got from the search results is incorrect. I think it probably has to do with the amount of information that it got from the web and then parsing that information. The model seems to have confused results from a different model. So if you use a bigger model like GPT4O or clause on it, we probably will be able to take care of this issue. But based on those results, it basically again wrote a code to answer our initial query, although it got this wrong because of this model size that it got wrong from the search result. But I'm really impressed the way it's able to recover from errors when it's executing the code and then it's able to rewrite the code and make sure that the code is actually correct. Next we're going to look at how you can build Agentic Rack system with small agents. Now, why do you want Agentic Rack system? So in a vanilla or standard Rack system, you usually perform retrieval only once. If the results are bad, the generation will turn out to be bad. But in the Agentix rag system, just the way we saw, if the results are bad, you can actually modify your query and then do retrieval again. And based on this new retrieval, you can do the generation again and you can repeat this process multiple time. And based on the the modified query, you might be able to retrieve the proper context that the LLM can use to generate proper and accurate responses. Now, although it's going to be more accurate, but keep in mind that there are going to be multiple calls to the LLM and multiple retrievals. This can add up pretty quickly, so you need to be careful with that aspect as well. They recommend to modify the user query because the user query is used as a reference to retrieve documents, but user query is going to be in the form of a question. However, the documents that you usually retrieve are in affirmative voice, so they are not in the form of question. And as a result, sometimes what can happen is that the match between the the retrieved documents and the reference query can be bad because tone of the query and these reference documents is different. They recommend to change the user query to affirmative voice so that instead of using an interrogative form, it will have similar tone to the documents that it's supposed to retrieve. Now this is a very interesting insight and I wanted to highlight it in this video because I think this can help with some RAC systems. So definitely check it out. If your documents are in affirmative voice and your questions are in interrogative form, you want to keep them in the same form. Okay, so the example data set is going to be hugging face documentation specifically related to the transformer package. So here we have the data set package, we download all the documentation and then we only are extracting the documentation related to our transformers package. So for each one of them it gets the original text, which is markdown format and the corresponding metadata. Here it's using recursive character text splitter. If you want to understand how to use this better, I have a couple of videos. Also I have a detailed course on building advanced RAC systems called Rack Beyond Basics. So if you want to learn more about RAC systems, I highly recommend to check out the course. In this case, we're going to be building a very naive RAC system, but the idea is just to show you how an agentic RAC system can be built using the Small agents package or framework. So we take our documents, we pass them through this recursive character text version, and as a result you're going to have multiple different chunks. Now, one common mistake which I'm also making here, because I'm using the example directly from hugging faces that they are using a chunk size of 500 characters, not tokens. So the chunks that you are going to see are going to be extremely small and usually they are broken at inappropriate places. So you want to make sure that you look at the structure of your documents when you are generating these chunks. And as I said, I cover this topic in very details in my course. So here is an example of the first chunk. So this is the chunk, you can see that it just breaks the sentence right in the middle. So it basically loses a lot of context. And you don't want to do this, but this is just a quick example. We also have the corresponding metadata. Then you want to create a tool because we want this rag system to be used as a tool in the small agents framework. Now here is an example of how you can build the tool. And in order to interact with the external systems, you will need to create your own tools. This small agents framework comes with a very limited number of tools at the moment, but creating tools is extremely easy. So first you need to have a name. We're calling it Retriever because it's going to be doing Document Retriever. Then what is the description? So this uses semantic search to retrieve the parts of the transformer documentation that could be most relevant to Answer your query. Now after that, what is the input? So the input is going to be a string and this is the information that goes into the LLM. So the query to perform this should be semantically close to your target documents. Use affirmative form rather than a question. And that's based on the discussion that we just had. Now, in the first step, we want to create the index. Now for this simple example, we're just using a BM25 keyword based search index. For more advanced systems, you want to probably use embeddings plus keyword. So that's going to be a hybrid approach. But the retriever in this case is just keyword based search and we're going to be retrieving the top 10 chunks. So you take all the documents, you create a retriever or essentially an index based on TF idf. In the forward pass, we're going to take the query, run that on the retrieval, get the documents, and then just return those retrieved documents to our agent to generate the final response. So we pass this through the Retriever tool, the actual documents that we want to process. This is the tool that we created. Now, if you just run a query directly on the retriever tool and call the forward function, it's supposed to return the top 10 documents. So I asked how to download a model from Hugging Face and you can see that it returned document zero. Then I think there is document one here and it returns a total of 10 different documents. So this is just to show you that the retriever part actually works. Now, in order to create an agentic Rack system, all we need to do is just pass that tool in the list of tools to our core agent. By default it's again using the Quinn 2.5 and if you want to change it, you can do that by just providing the name of the model. So we want for our agent to try four times if it's not able to retrieve the proper context for LLM and we want it to show us its progress. Now here I asked the same question, but now this time the agent is supposed to use the retrieval tool and then generate answer. So the question is how to download a model from Hugging Face. So here's the thought process. It says to download a model from Hugging Face, I will use the Retriever tool to find the relevant document and instructions. Once I have the necessary information, I will provide the final answer. And then you can see that here's the Python code that it wrote. So all the actions that the agent is taking are through code. So it is providing the query directly to our retriever and it will get the instructions. And the instructions are basically the documents that are returned by Audrey Retriever. So if you see chunk number one is extremely small and the chunk is using the Hugging Face widgets, it's not relevant at all. This is chunk number two, chunk number three, and so on and so forth. Now when it goes through this process, it figures out that the provided documents don't contain the instructions for downloading a model from Hugging Face. I'll refine my query to directly get the relevant information. Now in this case, it basically modified the query from a question to more of a statement. It says downloading a model from Hugging Face. Now when we modified our query, you see a lot different responses that the model is getting. So this is the first retrieved chunk which looks very different from what we saw in the previous example. Here are the second, third, fourth and up to 10 different chunks. So it goes to the next step. It says the relevant information for downloading a model from Hugging Face is found in document 4. I'll extract and format that information as the final answer. Now chunk number four talks about how to load different checkpoints. So not really specifically related to downloading a model, but if you're loading a checkpoint, so it will first download that for you and then it will load it from disk in subsequent calls. So not the best answer, but still it's pretty close. And here's the final answer that it gets. Now here's another query which is for transformer transformers of model training, which is slower either forward or the backward pass. Now again, let's use the same model. In this case it says the to determine whether the forward or backward pass is slowed in transformers model training, I'll use the retriever tool. So the first thing it did was it modified the original user query. And based on that it does retrieval. But then it says based on the retrieval documents, it seems the backward pass tends to be slower than the forward pass. Specifically document 3 mentions that for convolutional and linear layers. So here's the answer that it got. Now it says to confirm this with more specific information, I will retrieve documents that specifically discuss the performance comparison between the forward and backward pass. So again you can see that it's smart enough to modify the query again because now it's looking for more specific information. And here is, here are the retrieved documents. And it only take three steps to generate the final response in this case. Now next thing which I did was that I put together two different tools. So I provided the retriever tool and the search tool as well. And I asked it how to download a model from Hugging Face. So it says to download a model from Hugging Face, I need to find the appropriate instructions and follow them. Now it decided to use the search tool to get the necessary information from Hugging Face documentation because it's able to perform search now. So sometimes you will need to actually instruct it and tell it which tools to use specifically. So there might be a way in which we can tell it that, okay, if you're looking for Hugging Face documentation, only do retrieval. But we didn't tell it right. So it picked one of the tools and, and search seems to be an appropriate tool because once you create an index, there might be some updates online based on that. It looks for download model from Hugging Face. Now here, based on the search results, it is actually able to find the appropriate Python function that you can use to download a model and you can look at the thought process. So it says the most relevant information for for downloading a model from Hugging Face is in document number one. It provides clear example of how to use this function from the Hugging Face hub. Now something which was interesting that happened was it wrote code in order to process or in order to perform an action because it's supposed to write code to perform action, but it ran into an error and the error is coming from this sub process module. So it says in the next step, since the sub process process module is not allowed, I can't use it to install Hugging Face Hub. Keep in mind, like for this specific use case or this specific prompt, it is not supposed to actually try to download a model. We are only looking for that instruction. But since it's running code, so I think the model decided to download a file from Hugging Face and that's why you want to make sure that you run this in a sandbox closed environment. Otherwise it could generate code and it will try to run that code and it sometimes it can have unintended consequences. So you need to be very careful by limiting the code execution to a secure, airtight environment. Okay. Then I tried to use the snapshot download function. Again, it's running into very similar issues. And later on I think it figured out that it can do it without downloading those models. IT agents are probabilistic in nature, so as a result sometimes it will get into this loop of trying different things. Right. But in production systems you really need to be careful about this kind of behavior. Here you can see that it's taking multiple steps. We limit it to take only four steps, but These API calls can quickly add up and when you're building any system, whether it's agentic or non agentic, it's very critical to log all the intermediate steps because if you look at these intermediate steps, it really gives you an idea of what exactly was happening and what went wrong. In some cases you can use these intermediate steps and logs as a feedback mechanism to improve your system. This was a quick introduction to small agents. Now apart from the search and the retrieval tool, you can also build a lot of your own tools and so there is a guide which talks about how to build your own tools and how to integrate them so that small agents can start using it. I will put a link to this in the video description if you are interested in learning how to build your own tools that the agent can use. So here is for example a quick image generation tool which uses the Flux model. Then also there's a blog post on how to build good agents. It's worth reading if you're interested in building agentic system. Whether you use small agents or any other framework, Small Agents works with a number of different LLMs. You can even use a local LLM using Ollama or a proprietary LLM like OpenAI or Claude. Let me know if you are interested. I can make a follow up tutorial showing how to use both OLLAMA as well as these API providers. So 2025 is going to be year of agents when they actually become useful Again. Just a reminder, you're often better off without agents, so if your application does not need an agent, do not use them. That will save you a lot of time in debugging and development. You can thank me later for that. I hope you found this video useful. Thanks for watching and as always, see you in the next one.