EASIET Way to Install LLaVA - Free and Open-Source Alternative to GPT-4 Vision

In the previous video I showed you an open source alternative to GPT4 with vision. So just like GPT4, this model is able to understand and see images. In that video we looked at the online demo. But in this video I want to show you the easiest way to run this, either locally or in a Google colab on the GitHub repo. The authors of Lava, which stands for Large Language and Vision Assistant, has provided detailed instructions for installation. However, I'm going to show you a much easier way of doing this. Now instead of the official GitHub repo installation instructions, we are going to be using yet another GitHub repo which is called Lava Colab. And the Author of this GitHub repo is commendoro. Now it makes running Lava very easy. All you have to do is just open the code. In a Google Colab there are just three different sections and you just need to click and run them. If you look here, there are two different options. So the first one is Lava 7B 8bit Google Collab. This you can run in the free Google Collab. For the 16 bit version you will need the high RAM version. So you probably need to have Google Collab Pro version. So for the purpose of this video I'm going to simply run this in the free version. Okay, so let me quickly walk you through this Google Collab. But before that I also got access to GPT4 with vision. So now I can upload images and can talk to images within GPT4. So in subsequent radio going to be comparing this with LAVA with Vision. That will, I think will be a very interesting comparison. Okay, so I'm going to put a link to this Google Colab in the description. First thing you want to do is you want to click on Save a copy in Google Drive. So it's always good to create a copy of Google Colab notebooks that you find online. Now next we need to go to runtime. Click on Change runtime. Here I'm selecting T4 GPU. If you are on a free plan you should be able to see a T4 GPU. So let's connect to that and after that we will simply connect. So let me quickly walk you through what is happening in the code. So first we are changing the working directory to content, right? Then using this Git clone command, we're cloning the directory. This is basically the GitHub repo that we saw. And then it's installing Ninja Flash. Attention, you will need this if you want to actually do some sort of training of the model and after that we are installing all the packages or all the requirements within this repo. So let me run this code. Okay, so here all the installation is complete. Now we need to run the second cell. So this basically will start the lava controller and for that we are using the sub process module from Python. And now the third one is the most important bit. Okay, but before running this we are going to also look at the resources that it's using. So right now you can see that the system RAM is under like 2 gigabytes usage. We're not using any GPU ramp yet because we haven't loaded the model. Now in this code, essentially what is happening is it will load the model worker, then it's going to be hosting it at this specific port number and we are loading the model in eight bit. Right, and here's the model that we're going to be using. Now when you start this process you will have to wait for a little bit. Now according to the authors of this notebook, when you run this, you will need for about five minutes for it to load the model. And you can actually see that the RAM usage went up, the disk usage is also going up and it just also started to use the GPU ram. So essentially it's simply loading the model right now. And you can see that they recommend that after clicking the third cell, please wait for the model to load. So for the 16 bit model it's going to go up to around 14 gigabytes of RAM and for the 8 bit model it's going to go UP to 8 gigabytes of RAM. So we will have to wait a little bit for the model to load and after that we are going to run the web server. Okay. Since we are using the model in 8 bit, so we actually have got to around 8 gigabytes of VRAM usage and it has flattened out. Right. So this is basically the time when you can run the last part of the Google Colab notebook. So we'll start the server, the graduate server now. Okay, so this is going to take sometime, but you can actually see what is happening in here. So it's basically hosting the grid, your web server at this specific port number and it also gives you a public IP address. So this is the one that you can share and people will be able to access your app over the Internet. Right. And the link is going to remain active for 72 hours. This is great thing about gradient self. So let's open this up. All right, so here's the web server and if you have seen my previous video. This is the exact interface that I showed you in the last video. Now, one thing you want to make sure is that the Google Colab is running. So you can actually look at this cell and it's running. And that's why like, you will be able to use this web app. Right, so first let's look at a simple demo example that they have provided. Right? So here we have only one model. So I'm going to select this example and now the prompt is what is unusual about this image. If you want, you can change this prompt as well, but let's send this prompt and see what the model comes up with. Okay, so here is the response the model is generating. The unusual aspect of this image is that the man is sitting on a folding chair which is attached to the back of a yellow taxi. This is not typical sight as people usually do not sit on folding chairs while riding in taxi. Right? And then it goes on to say the man is also holding clothesline which with clothes on it, which adds to the unconventional nature of the scene. Right? So it shows you that lava 1.57 billion model is actually working right now. Now, since I have access to GPT4 with vision, so I use the exact same prompt and same image. And the answer is the image shows a man ironing clothes on an ironing board that is set up on the back of a yellow, moving yellow taxi. This is unusual because typically ironing is done indoors in a stationary setting and not in a moving vehicle, especially not in the middle of a busy street. So the answer is pretty good. Although LAVA model was able to provide, I think, a relatively detailed response. Now I'll be making a very detailed comparison video between GPT4 with vision and the LAVA Large Language and Vision Assistant. The goal of this video was just to show you how to set it up quickly and run it either on Google Colab or if you have a powerful machine with an Nvidia GPU such as 3090 or 4090, you can use the same steps to set it up on your local machine. Now, one thing which I have noticed that this currently works on Ubuntu only. I think there are some pull requests for making it work on Windows and Mac, so those updates will be coming very soon. Now, another thing which I want to say is that GPT4 with vision is definitely a much better and more generalized model. But it's great to see that what these customized and specialized small open source large language models are capable of doing now. Now, if you like what you are seeing on the channel and would like to support. Check out my Patreon or now we also have membership through YouTube. I will be adding some member specific perks, so if you would like to join, simply click that join button and support my channel. I hope you found this video useful. Thanks for watching and as always, see you in the next one.