Open Interpreter: Run ChatGPT Code Interpreter locally with CodeLlama

But in this video, I really want to focus on how to set up Code Llama with Open Interpreter. If you have access to GPT plus, you probably have seen Code Interpreter, which is now called Advanced Data Analysis. With the help of this, you can simply upload files to GPT4 and analyze those files. But there is a new open source project called Open Interpreter that basically lets you run Code Interpreter from your own local terminal. It's open source and completely free to use. You don't have to be a paying member of GPT Plus. Now it does use GPT4, so you'll have to pay for GPT4 API. But the great thing is it has support for Code Llama, which is an open source model.

And I'll show you how to use Code Llama with Open Interpreter in this video, we are going to be setting.

This up on a Mac machine, but the steps will remain the same for all different operating systems.

Okay, so first we need to set.

Up a new CONDA environment. So I'm going to use CONDA Create, then dash n and then we need to define the Python version. So I like to use Python 3.10.0. So this will basically set up a new CONDA environment for us. Okay, so I'm going to just hit Enter that will install all the required packages. So our CONDA environment is set up. Now we need to activate this. So in order to activate the virtual environment, I'm going to simply copy this and then paste it here, hit Enter. That will activate the virtual environment. Next we need to install Open Interpreter. So for that we're going to go back to the GitHub repo. Simply copy this command. Let me just clear up everything and.

We are going to just paste the.

Command that we copied. So this will install Open Interpreter on our local machine. Okay, so the installation process is complete. I'll simply clear everything. Now, in order to run Open Interpreter, you simply need to type interpreter and that will basically initiate the interpreter. So you have two different options. First is to use GPT4, which is recommended, but you need to provide the OpenAI API key. The second option is to use Code Llama. It's free, but it's less capable. So later in the video I'm going to show you how to set this up as well. Now, first we need to provide our OpenAI API key. So I'm going to paste my OpenAI API key. So here it is, and I will.

Remove this right after this video.

But you can simply hit enter. So it tells you that you can actually use this export Command to set the OpenAI API key environment variable. If you want to do this on Linux or Mac. So you're going to use Export command. If you want to set the OpenAI API key variable on your Windows, then you need to use Set X command. Now, since it runs Python code just like the code interpreter on GPT4, so every time you run this, it will actually ask you for approval before running the code. But you can bypass this behavior by using hypen y with the initial command, so that in that case it will automatically execute all the codes. Now, if you want to run this with Code llama, you will use interpreter local and that will use code llama. We're going to look at this later in the video now to see whether.

This works or not.

I'm going to run a very simple test. Analyze this file, and then I'll simply drag a file. So when you drag a file to your terminal, it will simply put a path of that file in here automatically. So if we simply hit Enter, it starts analyzing the file. So it says, sure, I can help you with that. And it also defines the steps that it is going to execute.

So it says that read the content.

Of the file, analyze the content of the files, and as I said before, so it will come up with the Python code, but then it waits for you to let it run the code. So we are going to type Y and it will start executing that code. So then it says that it appears to be a Python script. So it's actually pretty smart. Now again, it's writing some more python code in order to do further analysis. So if I hit enter, then it goes on to say that, okay, so there are different components. So for example, there's logging, there's a click torch. I'm using AutoGPT Q in there, downloading files using Hugging Face hub, right? So it's able to analyze everything, just like code interpreter in GPT4. So if there's interest, I will create a subsequent video where we will look at some more applications. But in this video I really want to focus on how to set up Code llama with Open Interpreter. Now, for that, I'm going to exit the current session. Now, in order to use Code llama, you can do it in two ways. You can type in dash dash local and hit enter, and then it lets you directly select which model you want to use. Or the other way that you can do it is you can simply use the same command. So, for example, I'm going to use the interpreter command, Then it loads, and if you simply hit Enter without providing the OpenAI API key. Then it will go and start using Code llama. Now here you have three different options. So it has 7B, 13B and 34B. So for the sake of this video, we are going to install the smallest model, which is also the least capable one, right? So if you simply click the arrow keys, you can change the selection. So let's start with 7B. So I'm going to hit enter and then Even for the 7B, there are, I think different quantization level is like low, medium, high and the higher one has the highest requirement in terms of both the size as well as RAM usage. So let's select the medium one, hit Enter. Then it asks us, do you Want to use GPU? Since I'm running this on my MacBook. So Mac has MacBook has an internal GPU and I want to use this. So I'm going to type Y. Then it says Code Llama inference package not found. Install Llama CPP Python. So we're going to type Y. So this is the issue probably you will also run into. It was not able to install LLAMA cpp, but we can do this manually. So the way we're going to do it is we're going to end the session again. Okay, Then we'll go to this website where we're going to be using this command to manually install Llama cpp. So I'm going to put the command that you need to use both for Linux as well as for Mac OS.

In the description of the video.

Okay, so we are back to the terminal. I'm going to paste that command. Let's run that command. So this will basically manually download Llama CPP with GPU support for Apple Silicon. Okay, so the installation is complete. I'm going to clear everything again. Okay, so after installing Llama cpp, let's run this again and this time I'm not going to provide the OpenAI API key. So we will just hit Enter. It says switching to code llama and now it's asking us which model we want to use. So I'm going to go with the 34 billion model this time and actually let's use the high model this time. So again it's checking whether we want to use the gpu. Then this instance of Core llama was not found. Would you like to download it? So I'm going to hit Enter again and it starts downloading the file. Now, based on the RAM usage, this seems to be downloading the 8 bit version of code llama 34 billion model. Now, this is going to take some time, so I'm going to be back after the download is complete. But one thing to keep in mind is the even though it's a 34 billion model, it's not as capable as GPT4, so the responses that you get will not be as good as GPT4.

Okay, so the installation is complete and now we're using code llama instead of GPT4. So now we'll use the same command that we used with Open Interpreter when we were using GPT4. So analyze the content of this file and we provide our Python file and after that you simply hit enter. Okay, so it seems to be working with Codenamer. So I got a response back. For smaller models 7B and 13B, it wasn't really working, but in this case it identifies that it's a Python script that uses local GPT models to generate text based on a prompt. Now, it was able to identify some of the functions that I'm using within the code. So for example torch transformers, but it added this function generate text, which is actually not present in the actual code. Now, for some reason it repeated itself again now in order to see whether it was actually reading the file or not. So I asked it what are different functions that are defined within the file? So it correctly identified there is a main function, there is a load model function that loads a GPT model. However it also says there is a generate text function and parse arguments which are actually not present within the file.

But in general it seems to be working.

Probably it will need better prompting. We also need to look at the internal commands that are used.

So probably I will have a much.

Deeper dive into the code of the open interpreter to see what can be improved. But overall, at least it works with codenamer. So that is a great news for the open source community.

So my final thoughts.

It's an absolutely amazing project and that's why it's actually trending right now on GitHub, which is absolutely great. Now you can do a lot with the interpreter. If there is interest, I will make subsequent videos showing different applications. For example, how to control your computer using Open Interpreter. If you found the video useful, consider liking the video and subscribe to the channel. In order to support the channel, you can check out My Patreon link is in the description of the video. Thanks for watching and as always, see you in the next one.