Gorilla: An API Appstore for LLMs

In this video we will be looking at this amazing project called Gorilla Large Language Model connected with massive APIs. In summary, it's a language model that is trained on API calls rather than language, and therefore they're calling it an API app store for LLMs. On making correct API calls, it's able to beat GPT4 by 20%, which is massive. First, let's talk about why this paper is even important. Language models by themselves do not have the ability to interact with physical world. They need external tools. So for example, ChatGPT has access to plugins which are essentially just making API calls to external tools. If you are familiar with LangChain, it also has a concept of agents where agents are able to use tools to interact with the external world and these tools are accessed through the API calls. However, if you notice the number of tools inside LangChain is very limited. That's where this paper comes in. Using their approach, a platform like LangChain will have access to an order of magnitude more API or tools than it previously had. For the rest of the video, we'll quickly go over the paper, then we'll check out the GitHub page and also show you how it works. Now, the great thing is that some of the models they present are completely open source and you can use them for commercial purposes. First, let's look at an example use case where this could be helpful. So for example, you can simply provide a prompt to the model saying that Help me find an API to convert spoken language in a recorded audio to text using Torch Hub. Now, there are a few steps that the model has to take. First, it needs to understand the natural language that you are using. In order to deduce which API to use from all available APIs, it has to pick an API which can perform the operation that you are requesting. Next, the model has to accurately figure out what are the inputs to this API call. And this is when it becomes very tricky because in a lot of cases you have very similar APIs with different inputs. And this is where most of the current LLMs fail. Just to summarize the whole process, they have fine tuned a llama 7 billion parameter model and now they are calling it a gorilla 7 billion parameter model. The model is fine tuned. Specifically on making API calls. For that, they curated a new data set which contains around 1600 API calls from three different platforms. Now there is a filtering process of how they selected these different APIs. They looked at the model card, had complete information. Then for each of these API calls, they created instruction API pairs Basically, Gorilla is an instruct fine tune model using the instruction and the corresponding API pairs. Now interestingly, this data set was generated using GPT4. For the rest of the process they used Information Retrieval System. So it's very similar to retrieval of information from your PDF files. Watch this video if you want to learn about that. So what you do is when the user provides an input query, it simply looks at all the APIs and then picks the one that is closest to the user query. In this case they tried different retrievers, including one based on GPT4. Now you get the API and then the Gorilla model, based on its knowledge creates inputs for that API call. So in summary, that's how the whole thing works. Now based on the results, they are showing that this approach can give you better results than GPT4. So it's around 20% better than GPT4 and surprisingly it's 10% better than ChatGPT. So it seems like ChatGPT is much better at making these API calls compared to GPT4. Now that's a quick summary of how these things work. Let's look at a working demo. First let's quickly look at the GitHub repo. So there's a Google Colab demo that we are going to be looking at in a bit. You can also run this locally and they're providing five different models. So the first one is Gorilla 7B hugging face V0, which lets you interface with hugging face API. Then there are a couple of other models which is this TF and TH and they are specifically for tensorflow and Torch respectively. Now these three models are based on Llama so you can't really use them for commercial purposes. However, they are also releasing these two other models. One is MPT based and the other one is Falcon based. Both of them are Apache 2.0 license and that's why you can use them commercially in terms of the FAQs. So we talked about it can use. You can use this model commercially and you will also be able to actually interface this with LangChain. So as I showed you in the beginning of the video, there is this concept of agents. So you can simply take Gorilla model and use it with the agents to perform different operations. For the sake of time, we are not going to be covering that in this video, but I will create another dedicated video for that. So let's see this in action. So first you need to go and open the Google Colab. As always my recommendation is to go to file and then save a copy in your Google Drive or in your GitHub account. Now next we have the code to set it up. You will need to install OpenAI, but you don't really need the OpenAI API key, and that's why it's set to empty. Now simply run this cell and you are all set to go. Okay, so next we will look at how to use this. So you simply need to call this function called get guerrilla response, provide prompt and then the model that you want to use. So for this specific example, they're using the hugging face model and the prompt is I would like to translate from English to Chinese. So let's run this. So here's what it came up with. So first it's showing you the API call. So you are going to be using transformer package and then pipeline from transformer and then there are some explanations. So hugging face transformers that the API provider and explanation, it says import the necessary library, which is pipeline from transformers, Then use the pipeline function from transformer to create a translation model that can translate from English to Chinese. So it also specified the model to use, which is pretty nice. And next it's saying that pass the English text from the created translator and it will return the translated Chinese text. So this is pretty awesome. It also gives us a code example. So basically, if you use this as a part of Agents or tools in LangChain or any other package, you can automatically make these API calls using the code that they have provided. Now the big question is going to be does it actually work or not? So for that, let's copy this. So I already installed the transformer package because that is going to be required. Okay, so we got the translated text and here is the translation. So overall, the API call was correct, the code is correct. Now we don't know whether the returned translation is correct or not. So we can use Google Translate for that, but this has nothing to do with the model itself. Okay, so according to Google Translate, the translation is pretty close. I said, how was your day today? But it's saying, how is your day today? So that's, that's pretty good. Here's another example. In this case the prompt is, I want to build a robot that can detect objects in an image. This means we need an object detection model. So it figured out the domain. So computer vision, object detection, Then it figured out the models to use. So this is the API call that we're going to be using and then which is the API provider. So hugging face transformers and then it provides step by step implementation. I have noticed that in some cases it provides Example code. So for example, here this was the code, but in some other cases it doesn't do that. But keep in mind it's the first iteration and it's a very small model. If you have a much bigger model, it will be more consistent. In the next two examples we are going to be looking at PyTorch and TensorFlow, specific models that they have trained. So the first one is I would like to translate from English to Chinese and it did a pretty good job because it provided all the information as well as a code example. So here's the code example. In the second case it also I think provided a code example. So this is what I was saying that in some cases it does provide code examples and in other cases it will simply ignore it. The last example is using the MPT model, which is open source and commercially viable. So in this case when I ran that translation prompt through, it gives us this response. Please input the text you want to translate and for some reason it actually responds to itself. So translate to Chinese. How old are you now? Similar to the previous case, it came up with an implementation and then it uses that phrase or sentence that is provided as an input to the model. So it's a decent model that works pretty good for the API calls. Now to summarize, this is a great project because it opens us so much different possibilities. Now these LLMs will have access to so many different tools because it's be able to make these API calls independently. The overall concept is very similar to Agents, both in Hugging face as well as LangChain. I hope you found this video useful. Let me know in the comment section what you're going to be using this for. Consider subscribing to the channel if you like this video. We also have a growing Discord community, so come join us there if you want to talk about everything, AI and LLM. Thanks for watching and see you in the next one.