Advanced Function Calling with Mistral-7B - Multi function and Nested Tool Usage

In a previous video, we looked at Mistral 7b, version 3, which has now native support for function calling, which is critical for these models to interact with real world and to be useful beyond a simple chat assistant. We looked at an example of how to do function calling, but in that case we were using a single function. In this video, I want to test this model's ability to do multi function calls and also do nested function calls, where the output of one function call is the input to another function call. We're going to be using this notebook created by uncle code. More on that later in the video. Before looking at the code examples, let's understand how function calling works and why do we need that. So a lot of times the model may not be able to do certain things based on its internal knowledge. For example, if you want the current weather conditions, the LLM will not have the ability to give you the current weather conditions. However, it can use external APIs to retrieve the weather information and then show it to you. Now, similarly, you can do that for stock market analysis or if you're interacting with different devices that you want to control using this LLM. So for that you need to have separate API calls or functions that will implement those API calls. Now, the flow is something like this. When the user query comes in, the LLM first determines whether it can do that operation based on its internal training knowledge, or it needs to have some external connectors that it needs to use. So first it will determine whether it needs to use an external tool or not. If it doesn't need any external tools, then it makes a second call to itself based on the initial user query and will generate a response. Now, if the LLM is smart enough and it determines that it actually needs to use an external tool, then it will look at the user query and also look at all the available external tools that are available to it. And based on the description of each of the tools, it will pick which tool to use. These tools are basically functions. Now, that's the second step, and in the third step, then it will use the Python compiler to actually make function calls. So basically run those functions, get the results from these functions, and then feed those results along with the initial query to the LLM again to get a response. Okay, so this is how it works in a nutshell. I have covered this in a lot more details in my other videos. I'll put links to a couple of videos in which you can actually see how these function executions work. Okay, so in this video we're going to test the Model ability to do multifunction calling and nested function calling. So let me explain what I mean by that. So let's say the user query comes in and it needs two separate operations to be performed, right? And those operations are implemented using two different tools or functions. So is this model able to pick multiple tools in order to do those operations and give the final output? So that's going to be the first test, which is pretty easy to do with most of the models that have these function calling abilities. Now the second test is going to be more complex. This is what I'm calling multi stage or nested function calling. So in this case, let's say based on the user query, it needs to do sequential function calling in which it will pick one function. So this is going to be the level one out of all the available functions. It will just pick one of them. Then the output of that function will determine second level function calling, which basically means that the output of this function call is going to be input to another tool or function. Then the result of that second stage function calling is going to be fed back into the LLM. Now in this video we are going to be using a Google Colab notebook provided by Uncle Code. He works a lot in function calling and fine tuning models specifically for function callings. So I'll put his Twitter link as well as YouTube link in the video description. Make sure to check it and check out his work. Okay, so here's the Colab notebook. I'm going to put a link to this. But rather than running this on Google Colab, I'm going to be running this on a a6000 GPU that has 48 gigabytes of VRAM. I'm running this on a VM from Mast Compute. If you want 50% off of the instances, make sure to check out the link in the video description. Okay, so first we are installing the Mistral inference package. That's the package that they recommend to use for inferencing the Mistral models. Then I'm downloading the Hugging Face hub and basically we are going to need to log in because these models are now gated. So make sure to get your access token from Hugging Face. Use that to log in and after that you can start using this model. So let me quickly walk you through the code. We are going to be using the snapshot download function from Hugging Face. So we just provide the model name and where we want to store this model. So we create that specific directory. Then we use the snapshot download function to download the model. So you need to provide the repo ID and we will also need to download some assistive tools that we will need. Right, so this will basically download the model for you. Depending on your Internet speed, it might some take. It might take some time. Okay, so next we'll import all the required packages for chat completion. Now we need to load our tokenizer along with the model, right? So these are the steps that you will need whether you're doing function calling or you're just doing chat completion with Mistral. Okay, so let's look at a function calling example that we covered in the previous video. This is a single function or a single tool. So we define the tool here, which is basically get current weather and, and this will get you the current weather based on the city name that you provide. And there are two different inputs that you can have. One is the location and the second one is the format in which you want the weather to be provided. So whether Celsius or Fahrenheit. And the model has to basically infer this based on the user location. So this is an example that I showed you in a previous video. So we take the input message along with the function call, we tokenize it, right? We use those tokens to generate responses and we show those responses back to the user, right? So in this case it actually makes the correct function call because there is only one function and it's going to use the get current weather. The arguments are going to be location. So that's basically Paris. And the format that it decided to use was Celsius. Now in this video I'm not going to show you how to make the action function call. I have covered that in my previous videos. So I'm going to link a couple of videos in which we go into how to actually do the function calling and get the results from the functions. But just to show you like how the available tools list looks like. So we're going to take the tokenizer, encode this whole input and then I'm looking at the text that is going to be coming out. So if you look at the available tools, it actually has this get current weather as a tool available, right? And it also figures out what are the inputs and outputs. So you can look at this whenever you generate a response and see if there are any available tools. If there are tools available, then you will need to make calls or execute those functions yourself. So the LLM is not able to do that. But I have covered this in previous videos. Okay, now let's look at an example of multi functions. So in this case we have multiple functions in here. The one is get current weather. The second one is get current time. Okay, so the input in this case is what's weather like today in Paris? So it has to make a function call to get current weather. The second one is also tell me the time in San Francisco. Right, so now the model has to decompose this query into two parts and based on those two parts make separate function calls. So when we decode everything, here's what we get. So the first function call is actually correct. So this is going to use get current weather. The location is going to be Paris. Format is Celsius. And for the time it uses get current time. And the arguments are just the location, which is San Francisco. And here we are just looking at the location. Right? So now the model is able to pick two functions and these are going to be potentially parallel function call. There is no interdependencies on these function calls. So the good thing is that it can actually pick those two separate functions that it want to use. Now look, let's look at the case of nested function calls. So basically the output of the first function call is going to become the input to the second function call. Okay, so here we have two other functions. One is get current temperatures. So very similar to what we had before. Okay, so the second function is convert temperature. Now in this case the inputs are one is going to be the actual temperature value, then what unit the temperature is in currently and then what unit we want the temperature to be as a result of this function call. Right? So if you say what's the weather like today in Paris? Right. Now in this case it just makes a single function call to get current temperature. But the output also said this will return the current temperature in Paris, France in Celsius, because it has to make a deduction whether to use Fahrenheit and or Celsius. Now it says if you want to convert the temperature to Fahrenheit, you can use the convert temperature function. Now what happens if we say what's the weather like today in Paris? Also convert it into Fahrenheit. Now in this case it's going to have to make two stages or nested function calls. So in the first one it gets the correct function call or to use it in which it's going to get the current temperature using the get current temperature function. The input is going to be Paris. Then it's going to make a subsequent function call to convert temperature. And in that case the temperature is going to be that you need to get the value of the previous function call. So that's why it's using this get current temperature dot results and then it also makes a deduction that the input is Celsius. So the temperature is in Celsius and it has to convert it into Fahrenheit. Now Uncle Code actually recommends to use a system message to make sure that the results that it gets from the first function call is in the correct format. And it's actually a very brilliant approach, so you definitely want to play around with it and in this case the system messages. You're a helpful assistant. Your job is to select tools relevant to the user query. In the case of multiple tools, if the tools are dependent on each other and one tools input parameter come from another function, use at followed by the function name for the parameter value. So this ensures that the value that you get is going to be correctly formatted and then you can use partial functions in Python to actually execute these function calls. Now we run the same prompt through the function call and again it will get the temperature similar to what we saw before. But now the value that you are getting is going to be basically correct value from the intermediary function call. So this is pretty good. So you always want to make sure that you use a system message to ensure that the output is going to be valid. I also wanted to just experiment with what happens if we do multifunction calling and one of of them has a nested function call. So in this case I added this function. So basically get current time and the user prompt is a combination of everything we have seen so far. So what's the weather like today in Paris? Also converted to Fahrenheit. And what's the current time in San Francisco Now? Interestingly enough, it makes three function calls. It gets the current temperature, then it uses the convert temperature to convert that temperature into Fahrenheit for Paris and then it's also able to call this current time function for San Francisco. So Overall this Mistral V3 is pretty smart when it comes to function calling. And if you want to execute these functions, make sure to check out the video description. There are going to be links to other videos on how to do the execution of the function that are called or the tool usage. Hope you found this video useful. Make sure to subscribe to the channel for more technical content. I do a lot on fine tuning as well as rag, so if you're interested in those topics, make sure to subscribe to the channel. Thanks for watching and as always see you in the next one.