AutoGen Studio UI 2.0: Easiest Way to Create Custom Agents

Autogen from Microsoft now has a pretty nice user interface called Autogen Studio. You can use this user interface to create skills, agents and workflows for building multi agent capabilities. Autogen Studio is open source and you can run this locally on your own machine. And that's exactly what we are going to be doing today. In this video I'll show you how to set this up locally on your own machine and then we will look at a few examples. In order to install Autogen on your local machine, you have two options. First is to install the Python package via PIP and the second option is to build it from source. In this video we're going to go with the easier option. So we will install the Python package for that. First we need to create a condo virtual environment. I'm calling the virtual environment R210 and we're going to be using Python 3.10. First just create the virtual environment. Then we need to activate the virtual environment using conda Activate and provide the virtual environment name. You can see that the virtual environment has been activated. Now we need to set up our OpenAI API key. So here I'm providing my OpenAI API key. I will revoke this after this video. You can also use open source LLMs with autogen Studio. I show you that in another video. Next we need to install the auto gen Studio package. So we're going to use PIP install autogen Studio that will install the package for us. You can see that it is already installed on my local machine. Next, in order to run the auto gen Studio, we're going to use the auto gen Studio command with UI and we will need to run this on a specific port. In this case, we are running this on 8081, but you can use any port that you want. When we run this, this will show us a URL. This is running on localhost port 8081. You need to copy this and paste it in your browser. And here we have our Autogen Studio running. There are three high level sections. The first one is called Build, the second one is Playground and the third one is called Gallery. Let's go over them one by one. The first section is Build that focuses on defining the properties of agents and agent workflows. This section contains the following three concepts. The first one is Skills, second is Agents and the third one is workflows. Let's look at them one by one. The first one is Skills. These are basically Python functions that describe how the agent can solve a given task. So here are a few examples. They are basically Python functions. Now each skill is supposed to have a name of the function, then a descriptive doc string which describes how exactly it's supposed to perform a specific task. And this description is going to be used by the agents when they are introduced as part of the workflows. At the inference time, all these skills are going to be available to the agents. You can also create your own new skills. So click on new skill, then provide the name of the skill that you want. And here you need to replace this code with your own implementation. The second concept is agents. This provides an interface to declaratively specify property of autogen agents. So it's very similar to what you probably have seen in their python code. Now each agent is supposed to have a name. So you can provide a name here. Then description of the agent, what exactly this agent is supposed to do. You also need to provide a system message which will control the behavior of a given agent. After that you can specify which different models or LLMs this agent can use. So here the default list has GPT4, GPT 3.5 Turbo and some open source models as well. You can add and remove models here. And the last thing is that you need to specify which skills are available to this model. So for the time being, the find paper from archive X as well as generate images, these are the two skills that are defined by default. And you can also add new skills if you want. So in this case I'm going to just use the GPT4 preview. So I will remove these other two models and then click OK. So by default there are two agents. One is the primary assistant, the other one is user proxy. So user proxy agent that execute codes. If you look at it, this one doesn't really have any system message, but you can add one here. And you can also add skills. Now similar to skills, you can also define your own agents and you will need to provide the name description whether it can accept human input or not. And the most important part is system message plus the skills that you want the agent to have. In another video we're going to look at how to build example agents. The last section is workflows. An agent workflow is a specification of a set of agents that can work together to accomplish a task. So here they have provided three different example workflows. Let's look at this general agent workflow. Now if you click on it, first you need to provide the name of the workflow, then the description of what the workflow is supposed to do. So this is used for general purpose Tasks. Then the next section is summary method. So define the method to summarize the conversation. In agentic frameworks like Autogen, multiple agents are interacting with each other to accomplish a task, right? So you can summarize their conversation. For example, this is just the last conversation. If you don't want any summary, you set it to none. Or if you want the LLM to generate summary of all the conversation, you can send that as well. Now, in this case, the sender is going to be the user proxy agent that simply execute code and the receiver is the primary assistant that we define in here. You can add more agents in here depending on the complexity of the task that you want to achieve. Just like Skills and Agents, you can also create your workflows here. The next section is Playground. So here you can test different workflows that you have created. So for example, we can create a new session. Let's click on this. Next you need to select the workflow that you want to use. So we will just use the general agent workflow. Click create. And now you can start interacting with this workflow if you go back to skills. So we had the generate images skill and find papers from archive. So let's test this agent. My message is going to be create an image of an astronaut on a horse and let's see what happens. Okay, pretty cool. We have an image of an astronaut on a horse, and I think it was using Dall e to create these images. Now let's look at what exactly happened in here. So here is the Python code that it created to execute our request. So first it created a prompt or description based on our initial prompt. And then it used the generate image skill to generate the image and save it. You can see the actual interactions between the agents. So there are a total of four different messages that were exchanged. Here's the initial prompt. So this is basically the user proxy agent that accepts the user prompt. Then the primary assistant agent in this case says to create an image of an astronaut on a horse. I'll use the generate and save images function from the skills PY file. This function will generate the image based on the provided description and save it to disk. So let's proceed with generating the image. And here's the Python code it used. Here's the file name for the Python code it's supposed to execute that is provided to the user proxy agent and it executes that code. And as a result, we get our image since the agent has the ability to find papers on archive. So here I was experimenting with another prompt so in the same conversation I said find papers related to Flash Attention and it's able to actually pull up the papers related to Flash attention. Now the first one is related to LLMs. The second one doesn't seem to have anything to do with LLMs. It seems to be more of a material science related paper. The third one also seems to be related to nlp. Now we can look at the internal interaction of the agents that we created. So user proxy agent receives our input prompt or query. The primary assistant says to find papers related to Flash attention, I'll use the search archive function from the Skills py. So this is another skill that we give to the primary assistant and this function will search the archive database for papers that match the query Flash attention. After that it writes the code it's supposed to execute in order to get search results. So here are the search results that the user proxy agent got and those are then followed to our primary assistant to generate the final response. Now, as you can see, both the agents are working collaboratively to generate responses or achieve certain tasks that we are defining. Okay, now let's look at the last section which is Gallery. So here you can save different workflows, agents and skills that you want to share. Now the way this works is you can simply click on Publish and then you're going to see that workflow is saved in here that you can share with other people. Anyways, this was a quick overview of Autogen Studio. We looked at how to set it up on your local machine, we looked at different functionalities that Autogen Studio offers and we looked at a couple of examples. I will be creating more content on this and we are going to look at some more complex workflows. I hope you found this video useful. If you did, consider liking the video and subscribe to the channel. Thanks for watching and as always, see you in the next one.