LLaMA & Alpaca: Install "ChatGPT" ON Your LOCAL Computer

In this video, I'm going to show you how to run large language models as powerful as ChatGPT on your local machine. And they're completely unrestricted. So in this video I'm going to show you how to run these models. These models will be able to respond to our queries, just like ChatGPT, and they will be able to follow our instructions. So we will explore two sets of models. The first models are called LLAMA models. They are part of Meta's research group, formerly known as Facebook Research. And the second model is Alpaca model which are created by Stanford. So I'll walk you through a step by step instructions of how to get them, how to install them and then how to utilize them. Before looking at the installation instructions, let's look at what these large language models are and how do they compare to ChatGPT. The first one we are looking at is the LLAMA model, which is from Meta AI, which was formerly known as Facebook AI. So according to the abstract, we introduced Llama, a collection of foundation language models ranging from 7 billion to 65 billion parameters. So it's not just one model. There are four different models with different number of parameters, and in comparison, ChatGPT has 175 billion parameters. But when they perform the comparison, they said that in particular Llama 13 billion parameters outperforms ChatGPT, Chat and GPT3, which is essentially the ChatGPT model on most benchmarks. And the Llama 65 is competitive with the best model, Chinchilla 70 billion and Palm, which is 540 billion parameters model from Google. Okay, so now we know that these are language models, but you may ask, what exactly is a language model? So according to them, language models are probability distributions or sequence of words, tokens or characters. This task, often framed as next token prediction, has long been considered a core problem in natural language processing. So basically a language model takes in a set of words or characters. So so for example, a sentence, and then it simply predicts the next word in that sentence based on all the previous words that it has seen. That's exactly how ChatGPT works as well. You give it a query and based on that query, it simply generates the text that it thinks is most appropriate. So that's exactly what Facebook AI did. They train these seven different models with different number of parameters on, on a large corpus of text to make prediction of the next word or next sentence. Now, unlike OpenAI, Facebook actually or Meta AI made it partially open. By that, what I mean is they actually opened it to a group of researchers to experiment with this. But someone on the Internet actually raised the weights of the model. So essentially everybody has no access to the models and they can use this on their own computers. Well, you might be thinking what do I do if I want to run it on my computer? Well, there is a simpler answer. Somebody put up this GitHub repo called delay where they made it available to public so you can actually simply download this, configure it and you will be able to run the llama models. And the great part is you don't need a powerful GPU to actually run these. Now, before we proceed with installing the llama models on your computer, let's talk about the second group of model, the alpaca model and see why it's so significant. From the description of their page, we introduce Alpaca 7B, a model fine tuned from Llama 7B model on 25K instructions extension following demonstrations. So what they did was they took the llama 7 billion parameter model, fine tuned it on only 52,000 instructions and they say that on a preliminary evolution of a single turn instruction following alpaca behaves qualitatively similarly to OpenAI's text DaVinci3 models, which is actually the ChatGPT model, while being surprisingly small and easy slash cheap to reproduce. So that's under $600. So let me unpack this for you. ChatGPT is, is trained on billions of instruction following pairs. So they give it billions of instructions and then they train the model. In this case the Alpaca 7B models, which is very similar in behavior to ChatGPT, it was trained only on 52,000 instructions, which is crazy to think about. This means that anybody can literally gather the data and train a model like ChatGPT. And keep in mind that they use the smallest of the llama model. So it's not, it's not like they use the 65 billion parameter model. Instead they use the 7 billion parameter model. So if you train it on 65 billion parameter, it's going to be a lot more powerful. And this whole process took less than $600. So to just understand this, let's look at this tweet. So this is somebody from ARK Investment group and this Tweet is from December 3, right? 2022. So AI training cost declining at 60% year on year it cost around $5 million to train chat GPT3 in 2022. By 2030 it will cost around $500 to train a model at the same level of performance. So he is Simply off by only seven years, because the Stanford group in 2023, early 2023, they were able to do this just for $600. So this shows you that how far we have come in such a short time. So now let's go to the part of the video that probably everybody's interested in how to run this locally. Now, to run it locally you just need these three instructions. So the first one will install the llama model and it actually installs both the 7 billion and 13 billion parameter models. The second instruction installs Alpaca models, and you don't have to actually execute the first one. If you simply need the alpaca models and you don't want to install the llama models, then the third instruction is you simply run a server which launches this web UI for you. Now you can do this on Linux, Mac and Windows. You just need to be careful about the memory requirement. So first, to run the 7 billion parameter, you need at least 4GB of RAM. Similar for 13 you need 8GB, and for 64 you need 32GB of RAM. For in terms of the disk requirement. So for 7 billion model, the Alpaca model you need around 4 gigabytes space on your hard disk. And for llama models, since there are like multiple models. So for 7B, you need around 31 gigabytes for the full model and if you look at the quantized one, so you need around 4 gigabytes and then the disk space usage changes for higher models. So for example, for 65 billion parameters you will need around half a terabyte almost for the full model and for quantized you need around 14 gigabytes. But I would recommend like if you are simply interested in the latest model, just run the alpaca model and don't even think about the llama models. You don't need them. So based on the operating system, you can following the corresponding instructions. I'm running this on a Windows machine, so I followed these instructions. So first you need to download Microsoft Visual Studio. So keep in mind it's not Visual Code Studio. That's the difference. This is Microsoft Visual Studio. So if you click on this link there is this Community Edition, which is free. So simply click on Free Download and then follow instructions to install it. I have already installed it, so I'm not going to do this part now. Once you start the installation, make sure that you select Python Development Node, JS Development and Desktop Development for C, you can uncheck the rest because that's going to, I think, add up around 12 gigabytes of space, but you don't need that. One thing which I noticed while even after completing the installation, the Node JS was not working. So I actually had to go and download Node JS from the website. I'm going to put a link to it. So if you are still facing issues after installing Visual Studio, just go and download it from the Node JS website and that should work if you are on a Windows machine. Just one thing to keep in mind, especially for Windows 11, so there is a CMD, the command line interface and then there is PowerShell. So you need to run these instructions in the command line, not the PowerShell. So you need to be very careful about that. Okay, so let's say once the Visual code Studio and Node JS installed, if you want to install the alpaca model. So all you have to do is NPX, that's the Node TS, Dely Alpaca and install 7B. Right? I have already installed it, so I'm not going to run this command. But if you run it, you and you face some errors. Just re execute this command and it should work. Okay. Once the installation is complete, you're all set. So all you have to do is run this NPX deny serve. Right. If you run this for the first time, it will actually install a few things, but it will just show you that server running on localhost at 3000 port number. Right. So I'm going to copy this, just come to your web browser and execute this. Okay, so I executed the. I actually accessed the localhost at that address. And here you are going to be presented with this simple interface. So right now I have only the alpaca model. If you have the llama model as well, you will be able to access them from here. And here are some other parameters at the top that you can set. I will go over this in more details in another video. Just a note when you run this for the first time. So like in this case, it simply checks for the path and it found the alpaca model here in this path. So I have the 7B for the Lambo model since I didn't install anything. So it's going to be an empty string. Okay, so let's ask some questions and see how it responds. So the first question I'm going to ask is what is the most important innovation made by humans? And let's see what the response is. Okay, so it says the invention of fire. Fire was a tool that allows us to cook food, create warmth and light our ways at night. Times, I think it's actually true. I would agree with that. Now for comparison, let's ask the same questions, the same question from Chat GPT as well. And let's see what Chat GPT answers. So it starts with the typical. As a language model, I don't hold the personal opinions and beliefs, but I would argue that most important innovation made by human is the development of written language, which is actually true. I think I would agree with both of them. Let's see if it knows the difference between alpaca and llama. I don't think they will know the difference between these language models, but alpaca and llama are actually two different species. Okay, so alpaca asylum, which means that they are related to the Nema, but they're not closely, genetically speaking. They have been domesticated for thousands of years in Peru and Bolivia where it is still used today primarily for. Primarily birds who. So that's. It's talking about alpaca first and then llama, which means that they are related to alpaca, but they're not closely, genetically speaking, they share about half of their DNA. Okay, so it actually give a pretty good answer. Let's ask the same thing from ChatGPT. Right. ChatGPT is definitely taking its time to come up with responses and I'm on the free version, so that's why it's a bit slow. Okay, ChatGPT is taking too long, so I'm going to go to the next question. So this is an example on the Stanford website. So write an email to congratulate new Stanford admits and mention that you're excited about meeting all of them in person. Let's see if alpaca can come up with a nice email. Like I had to run this a couple of times because it was giving me some unusual outputs. But here, here's the email that it wrote. Dear New Admits, I'm so happy for your acceptance into the prestigious university. This is a great honor indeed. I've been looking forward to seeing everyone at Stanford and getting acquainted with you guys soon. Then wishing all of us an exciting journey ahead. It's actually pretty neat. I would say it's good. It's not bad at all. Okay, I wanted to make a comparison between this and ChatGPT, but for some reason Chat GPT today is not letting me do anything. So sorry for this, guys. So in this video we looked at how to locally install the alpaca and Lamo models and run them in our local machine. I hope you found this video useful. If you did, don't forget to subscribe to the channel and like the video. This really helps with the algorithm and keep me motivated. In the next video I'm going to show you how to actually fine tune these models on your own data set. So that would mean that you are going to have a model, a chat GPT like model on your own data set. So I think that's going to be very exciting. So keep an eye out for that video. Thanks for watching. See you in the next one.