Agents, Agents and more Agents 🤖🤖🤖

It's kind of crazy to see that a single picture of Strawberries from Sam Altman has everybody on X speculating more on this later in the video. As a developer you probably have seen frameworks like Crewai, Autogen, Transformer, Agents from Hugging Face and Lang Graph from LangChain. These frameworks enable you to create agentic workflows. But I see another trend has started to emerge and that is foundation model creators such as OpenAI, Mistral and even Meta with the Llama 3.1 series are working towards making their own agentic systems, which I think is a great news for developer because this takes away a lot of abstraction offered by these other folks and now you will be able to directly build on the API provider. During the last couple of days there were two interesting announcements that I want to look in this video. First is going to be the introduction of structured outputs in API from OpenAI and the second one is Agents released by the Mistral theme. I'm going to show you a quick demo of how to get started with this new agentic framework. First, let's talk about the structured output in the new GPT4O API. This feature was announced last year during the dev day and was just released yesterday and it's probably one of the most important features when it comes to building real world applications which goes beyond chat. Consistent valid JSON output is critical for something like function calling that enables the LLM to interact with real world and proper function calling is the basis of agentic workflows. So if you want to build apps beyond chance, you definitely want function calling and agent capabilities. So what exactly is this? This enables the developer to provide JSON schema and the model is going to return outputs in that schema provided by the developer. And this new model seems to be pretty good at it because according to their own testing, 100% of the time when the developer asks for JSON output, the new model can actually provide that. A great example of this is creating user interfaces depending on the user need and you can automatically generate those. So here is an example that the model is able to generate structured JSON outputs and you can use that to render different types of pages. So for example, here is an example of a landing page. Now the same model can be used to generate a signup screen for an app and stock price widget. I'm going to be creating a dedicated video on this because I think this is one of the most important features announced by OpenAI in recent days. The second major news was today, so the Mistral team announced the new agentic workflows or Agent Builder in a blog post titled Build, Tweak, Repeat, Making it Easier to Develop and Share Generative AI Applications. There were two things announced. One is fine tuning of the models. And if you combine fine tuning with this alpha release of agents, I think it makes Mistral a very powerful framework. And this new agent framework goes very well with their Mistral Launch 2 model, which was released a week ago. So let me show you how you can build custom agents using their Agent Builder and then how you can call this through the API as well as use it within their leadchat platform. Here's the Agent Builder. It's a graphical user interface, so it makes it very easy to start building agents and you can build pretty complex agents here. In this example we're going to look at a very simple agent which I'm going to call Python Agent. And this is based on one of the examples. The idea is that it will receive an input from the user or a user query and will generate Python code without any comments or any explanation. Next, we need to select a model that we want to use for the agent. They recommend to use the Mr. Launch 2 model. They also gives you the input and output cost per million tokens, which is pretty neat. So we will just go with the Mistral match 2 since the task is to generate Python code, so I'm going to set the randomness to zero. We don't want any randomness in the output that is going to be generated. Next, you need to provide system instructions. Think of this as custom instructions that are going to be passed on to your agent, and these instructions will control the behavior of your agent in this case. Here are the system instructions. So you are a Python coding assistant that only outputs Python code without any explanation or comments. Return one Python function for given query and one test case using assertion. So return Python code with two sections. The first is going to be a Python function based on the user query and the second one is going to be a test case. So you want to provide detailed instructions that that will control the behavior of your agent. They also suggest to provide few short prompts or examples. So here you can add some examples for the agent to learn from. So we're going to add just one example and that is going to be a relatively incomplete example. So the example is how can I sort a list of numbers in ascending order? And the expected output is supposed to be Python code. But in my example I'm not providing the test case because I think the Mistral Nash 2 model is smart enough that it can come up with that. But in real world applications, if you have some complex workflow, you definitely want to add more complete and concrete examples. Now we can test this agent and see how good it's going to be performing. So here is a playground that you can test it. So I'm going to ask it a very simple code. So the code is going to be code to download files from an S3 bucket and let's say using the Boto3 library, and let's see if it actually follows the instructions that we provided in here. So you can see that it's only generating Python code and the code has two sections. One is the Python function and the second section is a test case. So this is pretty neat. It's actually following exactly the instructions that we have provided. Here's a second prompt. Sort a list using bogosort. Now, it doesn't specifically ask for code, but since we're using this Python agent that we just created, so it actually knows that it's supposed to return a function and a test case. So this is pretty good. But the main power is that not only you can create your agent in here, you can deploy it both to their Lichat platform. So you can use a graphical user interface or deploy it to an API endpoint. Now, in order to deploy it, just come click on this Deployance button. Here you can enable deployment both for API and the Lichat. And if we click on the chat, we see this message. Deploy agent to lichat. Deploy your agent to lichat so that everyone in your organization can use it. And using this agent in the chat will be free and will not be built like API calls. If you're accessing this through the API, they will bill you now till the end of the month, which is August. If you sign up for the platform on Mistral AI, you will get substantial free credits, which I'll highly recommend to do. So we're going to click on deploy and that will deploy this model for us. Now I'll show you how to use this both using the platform as well as to the API. So you can click on the Share button. This will take you to the Leap platform. And here you can see that apart from the normal models that are available on the platform, there is a specific section for the AI agents as well. So you can just select that Python agent and start interacting with it right away. And if I ask it how can I sort a list of numbers in the descending order using Bogo sort? Then if the agent Works correctly. We should see two different sections. The first one is the Python function and the second one is a test case. Now, it's actually pretty smart. When I said can you do that in C, it simply returned the Python code. So it's actually sticking to the behavior that we ask it to adopt, which is pretty great. Next I'm going to show you how you can use this agent in your own code. So this is the example that they have provided. And we're going to see that this codebook has four different parts. One is user query, the second is code and test case generation. This is basically performed by the agent. Then you execute that code and validate it and if there are any issues we will be able to retry and then finally we'll be able to see see the results. So first you want to download the latest version of Mistral AI Python SDK, then import those packages and set the API key as your environment variable. The only difference compared to the normal Python SDK calls to the Mistral API is going to be that you need to provide your Agent ID now and you can go to the Agent Builder platform and just copy that API, that agent ID from here. Here they have provided a few functions which is basically run the Python agent, extract code, check the code and run the whole workflow. Now, in order to create the builder, we will be using the Mistral Python SDK and as I said, instead of the model, now you just need to provide your Agent id. Now this is great because you can create specialized agents for specialized tasks in your application, then select those agents rather than making calls to the general Mistral models. So here, if you provide your query to the main Python workflow function, here are the steps. So it basically tries it first, then pass that on to the agent. The agent will generate the code as well as the test case, and the code that is in here will basically extract both of those separately. So it's a very simple example. Link is going to be in the video description. Now they also have more complicated multi agent workflows and this agent is able to do data analysis, planning, code generation and execution, and then analysis report summarization. Here in this notebook you can see that they provide three different system prompts or instructions for the agents. One is for the planning agent, the second one is for the Python agent, and the third one is the summarization agent. And now you have IDs for different agents that you can get from your own platform when you build these agents. This is an example of a complete workflow or a complex example or application that you can build on top of agents. Now each agent has one specialized task and that is being controlled by the user at the end. Using the summarization agent along with the other agents it is able to provide a very comprehensive report. Now do check this out. I think it's extremely powerful. I'll be creating more content and if you're interested in that make sure to subscribe to the channel. One thing which I really like about these new agent frameworks from the LLM providers is that there is no abstraction or any extra code. You can just use the API providers directly and build on top of them. Apart from the proprietary models, you also have Llama Agentix system this is an Agentix system that the LLAMA team is creating and is currently available. It has built in support for different tools like search or code interpreter so you can actually run code directly. Very similar to the code interpreter or code execution API from Gemini. Also this system is able to do tool calls to new tools that it hasn't previously seen using the zero shot learning abilities which is pretty amazing. And some other models. Providers like Quinn, which is yet another open source project, also has their own agentic framework. Now coming back to the Strawberry that has everybody talking. So it seems like Q Star or Strawberry, which is an internal project of OpenAI is potentially an agentic framework, but we don't really know much about it yet. Anyways, it seems like everybody is moving towards creating agents which is pretty exciting. We're going to see a lot more of these. If you are a developer who is working with LLMs, I highly recommend to pay very close attention to agents and agentic workflows. These are very early days for agentic frameworks and agentic systems and in all honesty most of them don't really work at the moment, but this is the worst they ever will be. It's going to improve from here, so you really want to pay attention to agentic frameworks or you should be able to build your own agents based on the LLM providers. If you are interested in agents, make sure to subscribe to the channel. I'm going to be creating a lot more content on this. I hope you found this video useful. Thanks for watching and as always see you in the next one.