Anthropic’s Blueprint for Building Lean, Powerful AI Agents

If you are a developer building agentic systems, this is a must read blog post from Anthropic. There is a lot of hype around frameworks for building agentic systems and agents themselves. We have some really great frameworks like Crewai, Langgraph, Autogen, but in most cases you don't need them. The most successful LLM implementations for agents use simple composable patterns rather than complex frameworks and specialized libraries. And this is kind of the main takeaway of this blog post as well. You want to start with simplest possible solution and only add complexity when needed. Most of the frameworks have a lot of abstractions which you don't want in your production systems. In this video we are going to look at the blog post also some of the architectures that they have suggested and I'll also show you code examples of how to implement those. In the process we will explore different patterns, understand when to use them, and most importantly learn how to avoid unnecessary complexities in our agent designs. First, let's clarify what exactly is meant by an agent, because this term gets thrown around a lot in AI world. Now for me the definition of agent is extremely simple. You have an LLM that has access to a number of tools and it can decide when to use which tool. This meme really captures the essence of agents. But as the blog post states, some developer thinks of agents as fully autonomous systems that operate independently for long periods using various tools to accomplish complex tasks. Others see them as more structured system following predefined workflows. Anthropic put this into two categories. One is workflows and the other one is agents. Workflows are systems where LLM and tools are orchestrated through predefined code paths. These steps are clear, ordered and predictable. So think of this as a traditional state machine where you have the current state of the system. Based on some external input or feedback, the system move to another state. Now these states plus the paths that the system can take are well defined well in advance and most of the systems that you are going to build using LLMs today will actually fall into this category. And that's exactly what I usually recommend when I'm working with companies. Now agents on the other hand are more like autonomous problem solvers. So they dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks. They can adapt their approaches based on the situation. So this is more probabilistic in nature where the system can plan what it's going to do, which tool is going to use rather than following a predefined path. Now, the problem with agents defined in this way is that if something goes wrong, it's extremely hard to debug. On the other hand, workflows can capture really complex systems and they are relatively easier to debug if something goes wrong. So in general your first option should be to use workflows rather than agents. But there is a place for agents in our current systems or applications, and we will talk about those later in the video. Now the next section is when and when not to use agents. And I really like this part, which really resonates with my own experience. So the main recommendation should be not to build agentic systems at all if you don't need them. And most of the people and companies actually don't really need probabilistic agentic systems. A simple workflow is more than enough. If you are building agentic systems, they will add additional latency and cost. That is going to be a trade off with performance. So these trade offs needs to be considered when you are working in a business setup. Workflows on the other hand, can be used to build very complex systems, especially if you need predictability and consistency for well defined tasks. Now, agents are extremely flexible and if you need model driven decision making then they are definitely an option. But as they say, for many applications, optimizing single LLM calls with retrieval and in context examples are usually enough. So just to sum it up, don't use agents if you don't need them. And if you think you need them, you need to make sure that you exhaust the possibility that a simple state machine is not enough for your application. But what about all these different frameworks you can use? Crewai, Langgraph, Autogen to build very complex systems. Now in general, if you are just starting to experiment, these are great frameworks. You should use them for experimentation purposes because these frameworks make it easy to get started by simplifying standard low level tasks like calling LLMs, defining and parsing tools and chaining calls. Together they also introduced the extra layers of abstractions and hide the underlying prompts and responses, which makes a huge difference in the performance of your agentix systems. But if you want to put these systems in production, you actually need to understand what is going on under the hood. And in most of the cases you don't really need this abstraction or the additional complexity that frameworks like Langgraph or Crewai would actually add in your application. And that's why their recommendation is to start using the LLM APIs directly. Many patterns can be implemented in few lines of code and we'll look at some of the examples. It's a lot easier to do than what you would think. Another problem that I have seen with these frameworks is the breaking changes. Because things are moving so fast, none of them are mature. There's new research, new techniques. Sometimes additional changes or updates will break your existing code bases. So it's even hard to keep up or maintain your code bases if you rely on these frameworks. But keep in mind, this recommendation is for developers. If you're a non coder or you don't really have a lot of coding experience, then my recommendation would be to start with frameworks because it will make it easy to begin. But later on as you get experience, the recommendation would be to start directly using these different APIs from the LLM providers and build system around them. A good example is swarm from OpenAI or the transformer Agents. Because if you go through the code, these are repeated composable patterns that they use and the abstraction is kept to minimum. In the rest of the video. We're going to look at different common patterns that the anthropic team recommends or they have seen in production. The first thing that they talk about is the augmented LLM. So when you're using or building these agentic systems, either workflows or agents, the way they describe it, you cannot use a simple LLM itself. You need to augment the LLM with different abilities. And these abilities are retrieval tools in memory. Now here's an input to the LLM. The LLM can use something like a retrieval system, for example, a RAG system, to get factual information. Or it can interact with external systems using tools or API calls. So these are going to be function calls. And then in order to keep track of everything that it has seen so far, all the action that it has taken, the LLM also needs to have access to some sort of memory. And that is why they are calling it the augmented LLM. So that is a very important building block when it comes to building agentic systems. Something they highlighted when you're implementing the augmented LLM is that you need to tailor these capabilities or augmentation for your specific use case and ensuring that they provide an easy, well documented interface to your LLM. The way I recommend to think about it is implementation of functions or methods in a programming language. So the best practice is that you want to make sure that your function performs a single operation. If you add a lot of complexity in a function, it kind of beats the purpose. Similarly, if you are building an agentic system, each Agent is supposed to perform a single function, and the LLM needs to have access to well defined tools which will help it to execute that single function or a single specific task, rather than give it a number of generalized tools. So this is extremely important to keep in mind when you are building agentic systems. These are not magical systems and you want to make sure that the LLM or the agent has to make the least amount of decisions possible. Now let's talk about different patterns. So the first pattern is a workflow or a state machine. And this is called prompt chaining. So prompt chaining decomposes the task into a sequence of steps where each LLM call processes the output of the previous one. So essentially the output of the first becomes input to the second, and so on and so forth. Now sometimes you can have these programmatic checks or gates, so you can actually check the output of a subsystem and make sure that it is actually executing what you want. And if it fails, you can repeat that task. Think of this like breaking down a complex essay into an outline. Then based on that outline, you generate a rough draft. And then based on that rough draft, you will generate the final version. So each step kind of builds on the previous one. So when to use this workflow, it's ideal for situations where a task can be easily and cleanly decomposed into fixed subtasks. The workflows follow paths in order to execute or accomplish a task. Now the main goal is going to be to trade off latency for higher accuracy by making each LLM call an easier task. Examples use cases would be content generation. As I said, first generating an outline, then expanding each section. Code generation is another example. You can create a high level design then implementing specific components. Translation is another example where essentially you can first translate it literally by then refine the translation for language flow. The second pattern is routing. So this pattern is like having a smart receptionist who knows exactly which specialist should handle each request. In the front you have a router LLM which is extremely smart, and then it passes on the task to a specialized LLM. Depending on the nature of the task, these could be different LLMs with different system prompts. And depending on the user input, this router selects which LLM and which system prompt to use. So routing works well for complex tasks where there are distinct categories and different LLMs or different models are better for each of those categories. Now the router itself can either be an LLM or a traditional rule based classification system, so it doesn't really need to be a Generative AI system here. Some examples again are customer service. So you can direct different types of queries to appropriate departments. Content moderation is another example. So you can route different types of content to specialized checkers. Code reviews is another example. So you have different aspects of the codes which are reviewed by different experts. And then the router can decide which expert to choose. The next one is parallelization. The idea here is you execute the task in parallel. So either you kind of break down the task into sub tasks that are independent. This will be an example of sectioning, which breaks the task into independent subtasks. You run Those through independent LLMs and then you aggregate the results. Another example would be voting. So you run the same task through multiple different LLMs. And the idea is you get diverse perspectives or outputs, and you aggregate them to get the final output. Now, the voting example kind of relates to ensemble models in traditional machine learning systems. So if you're familiar with ensemble systems, the random forest is a perfect example of a system like this, where you take your input, run it through multiple different trees or models, and then you combine those results. And the idea is these different models are looking at different aspects which aggregates the results. And usually ensembles gives you better results compared to a single LLM or a single model. The problem is that you are making parallel calls which will add cost when you are running them. So when to use this? It's effective when you can divide your tasks into subtasks that can be parallelized and that will increase the speed. Or if you want to have multiple perspective to solve your problem. Now, sometimes it actually helps a lot if you provide a lot of information to an LLM, it can hallucinate. But if you break it down into subtasks and independent LLMs are looking at these subtasks, then you can potentially get better results. A closely related pattern is orchestrator and workers. In the orchestrator worker workflow, a central LLM dynamically breaks down tasks, then delegate them to workers, and then synthesize the results. So if you look at this pattern, it looks very similar to parallelization, except there's this orchestrator right in front of the parallel section of your pattern. And you want to use this pattern when there are no predefined subtasks. So if the orchestrator has to dynamically come up with the number of subtasks that needs to be executed in order to achieve our goal, then you want to use this setup rather than something like parallelization, because it expects that you will be able to have predefined subtasks that you can run in parallel or you run the same thing through multiple LLMs. It's a lot more flexible compared to simple parallelization because the subtasks are not predefined but determined by the orchestrator based on this specific input. This is like having a project manager who breaks down complex tasks depending on the input, assign work to specialized team members, and then synthesize results into a cohesive solution. Examples again, coding products that make complex changes to multiple files each time, so you don't know which files to change ahead of time, but depending on the input you will change them. And search tasks that involve gathering and analyzing information from multiple sources for possible relevant information. The last one is evaluator optimizer. So in this workflow or pattern, one LLM call generates a response while other provides evaluation and feedback in a loop. Think of this as having a writer and an editor working together. So the writer writes something, editor will give it a feedback, and based on that feedback the writer has to improve it. And the process goes on for a predefined number of steps. In Traditional computer vision, GANs, or generative adversarial Networks was a classical example of this evaluator optimizer workflow where one system will generate an image, the other was figure out whether it's fake or not or assess the quality. And based on that feedback, the generator will try to generate better images. Now when do you want to use workflows? Particularly the effective when you have clear evaluation criteria. So think of this. If you can break down your task into predefined paths that the system can take in order to accomplish a task, then workflows is a perfect example. And workflows can also contain a hybrid approach. So it doesn't really have to be all agentic system within LLM. And it can have a traditional machine learning component or even a traditional software component along with this generative aspect. So this should be your first thing to consider whenever you're building any agentix system. You want to make sure that there are deterministic steps involved. And if you can come up with a state machine to accomplish that, you should use workflows. So far we've been talking about workflows of state machines, but then there's agents. There are definitely use cases in which you want to use these probabilistic systems which can make independent and automated decisions. Now for these systems to work, you want to monitor their progress and usually you want to have human in the loop because those are the best kind of agentic systems that you can deploy in production. So essentially this is what it looks like. Human. The LLM LLM does an action. Action is executed in the environment, it gets feedback from the environment. An action is executed through tools and then the LLM will get to decide whether it needs to stop or sometimes, as I said, human in the loop. So it will show the results to the human, the human observe those and kind of provide a feedback if needed or step in if it needs to stop the execution. But the main thing is that the LLM is autonomous and its abilities to make a plan of when to use certain tools or when to take some certain action and when to use them. So they can be used for open ended problems where it's difficult or impossible to predict the required number of steps or where you can't hard code a fixed path. And this is where it's kind of really hard to evaluate as well. Sometimes they will just fail and you won't even know the reason of failure. Since the LLM has to make a number of different calls in order to execute a given objective, it could mean higher costs and potential compounding errors. Now I am describing these issues. It doesn't mean that you can't use autonomous agents. There are already a number of different applications, but I still feel like we are in the process of having these systems matured. So you really need to keep an eye on these systems if you're going to be running them in production. And I'm talking about it based on my own experience. Here are a couple of examples where they show that agents can be helpful or useful. A coding agent to resolve suitebench tasks involved editing many files based on the task description and then it has to run them, get feedback from the environment and make the edits again if it doesn't pass the test. And they also refer to their computer use reference implementation where Claude uses a computer to accomplish tasks. Different aspects of an Agentix system. You have Human, which interfaces with the LLM. You get the query from that. Sometimes the LLM might refine the query based on the user feedback, right? Then it starts interacting with the environment itself. Here's a coding example in which it has to search for files, modify files and things like that. But it could be anything. It might be like accessing weather information or stock market data. So it goes and get feedback from the environment, refine its tasks, executes it again. And when the LLM thinks that it has completed the task, it will come back and show it back to the user. Okay, so these are different patterns that anthropic recommends to look at. If you're building agentic systems, you can combine these to build more complex systems. And this is not an exhaustive list, but let's look at some code examples of how you would potentially implement some of these workflows without the need of a specialized libraries or complex frameworks. Now for this, we're going to look at this cookbook from Anthropic, which goes at basic workflows, then evaluator and optimizer and orchestrator workflows that we discussed. Now, the main file every one of these notebooks is using is this. So this is basically calling an LLM, which is the Anthropic API client, that's the main LLM execution call and and then there's another function which extract the XML tags, it generates structures output. But these are the two utility functions that each of the notebook is using. So let me walk you through the example code in the notebook which makes it extremely simple of how you can implement these workflows in your own setup. And if you want to run them, you will just need to clone the anthropic cookbook repo and you have access to these notebooks. Now, in terms of implementation, this basic multi LLM workflows implements prompt chaining, parallelization and routing. For prompt chaining, you provide your system prompts and then you execute them one by one. So as we looked here, you have the first LLM call, second LLM call, and third LLM call. And the implementation is essentially a for loop where you have system prompts for each of the LLM calls. You run the first LLM call, you get the result, you feed that result back as an input to the second LLM call, and so on and so forth. Now for the parallelization you use multi threading, where you get the user prompt, the user prompt can be decomposed into subtasks and run those, or it can be the same prompt which runs through different LLM calls in parallel. So this is an implementation of how you can potentially implement or parallel workflow. And the last one is the routing. So in the routing, the main selector prompt, which basically gets the input and then depending on the specific needs of the user, it can select a specialized, pass those to that specialized LLM and then get the results from that LLM and generate the final response. So we're essentially looking at this router LLM, which is the selector prompt that I just showed you. And then you have these parallel pathways, but actually not parallel pathways because the router will select one of the available LLMs to generate the final output. This notebook also goes through some example use cases. I'll highly recommend to look at this notebook to get a better sense of how those simple implementations can be used for specific use cases. Now, this second notebook shows an example implementation of Orchestrator and workers workflow. You want to use this specific pattern if you don't have well defined subtasks and you need to create those subtasks dynamically at runtime. In this case they have the orchestrator prompt, which is a bigger LLM that will generate a number of subtasks depending on the user input. Then the worker prompts, which are parallel executioner which will execute those subtasks. You also have this parsing function for subtasks that are generated by the orchestrator. Whenever a user query comes in, you first pass this to the orchestrator. The orchestrator will generate a number of subtasks, then they will go through the workers and each of the workers will execute those subtasks to generate their own results. And at the end you will synthesize everything using a specialized synthesizer. Now the example use case in this notebook is marketing variation generation. So here's the orchestrator prompt. You want to make sure that you customize your orchestrator for the specific task that it's supposed to do. You also want to customize your workers because those are the ones that are going to be executing those subtasks. This last notebook implements the evaluator optimizer workflow. You want to use this when you have a clear evaluation criteria and you have a value from iterative refinement. In order to execute this or implement this, there's a generator function, then there is an evaluator. You need to have a very clear idea of what the evaluator is supposed to do and and then you just have a loop which will go through this loop of generation, evaluation, feedback, and whenever we pass based on the evaluation criteria, that's when it's going to stop. The example use case that they have presented is iterative coding loop. So you have the evaluator prompt which will evaluate the code, and then there is a generator prompt which will generate the code for a given user task. The takeaway from this blog post and my own experience is that you want to start simple and only add complexity when needed. Frameworks are amazingly helpful if you're building a proof of concept, but usually you end up customizing a lot of the code for your own application. Sometimes it's better to just start with a green field implementation and build your own custom solutions. This will save you a lot of engineering efforts, a lot of time and a lot of compute costs as well. Now this suggestion or recommendation is specifically for developers. If you are a non developer and you are looking for a quick way to implement, frameworks are the best way to go. But again if you are going to deploy anything in production, you need to understand how they work, what is the underlying implementation because that is going to save you a lot of headache later on. Anyways, I hope you found this video useful. Thanks for watching and as always, see you in the next one.