Llama is Finally OPEN - Commercial Use is Welcome!

If you have been paying attention to open source large language models, you probably have noticed there has been an explosion of new models every day in the last few months. It all started with Alpaca. Then there were models like Vicuna GPT4ALL and Open Assistant. Now for all of these models there is an Unsung Hero and that's Meta or Facebook. They were the ones who released the initial Llama models, although the weights are still not publicly available. But they got leaked and as a result we have all these amazing large language models. Now there has been efforts to recreate llama models because llama models cannot be used for commercial purposes. Now one of the earlier approach was to recreate the data set. So if you remember, there was a project called Arit Pajama from 2gether who actually assembled a data set of 1.2 trillion tokens. So now so far the only thing missing is the llama weights that can be used commercially. That's where this new project comes into play. So today I'm going to show you this new project called Open Llama and it's an open reproduction of Llama for Llama model. We already know the architecture, but somebody had to actually go and retrain it. That's what this project is doing, right? So in this work they are releasing a 7 billion parameter open llama model that has been trained on 200 billion tokens. You might be thinking, wait, llama was originally trained on 1.3 trillion tokens. And that will be correct. That's why they are saying it's an early release of a public preview. So they are actually retraining the model with much larger data set and it will be hopefully available pretty soon. And this project is already releasing the model with synpytorch and tax format. So you can actually use this if you want. So let's go over the training data set and the training process. So in this case they are actually trying to replicate the training process and the training hyperparameters, just like the original Llama paper. So in for example, they're using exactly the same architecture, the same context length, same number of training steps, the learning read schedule and optimize it. The only difference is since they don't really have access to the original data. So they are using a part of the red pajama data set. As I showed you, red pajama is 1.2 trillion tokens. But this is just using a subset for the initial training. Now as a part of the report they're actually including the their training loss which actually Smoothly goes down. Now let's look at the evaluation that they have provided. Just one thing, I'm not going to be showing you any demos yet for this project because it was just released and I don't really have the hardware yet to run this locally. But what they have done is they use this language model evaluation harness. So it's basically a protocol to evaluate the performance of language models and then they compared the results with the original LLAMA paper. Apart from the llama comparison, they are also providing comparison with GPT J which is a 6 billion parameter model trained by Eleuther AI. So before looking at the evaluation, we need to keep something in mind. The LLAMA model was trained on 1 trillion tokens. GPT J was trained on 5 billion or 500 billion tokens, whereas open Nammer in the current form is trained on 200 billion tokens. The training data set comparison is critical when we are going to be looking at the results. Now here's the table of comparison. So in the first column is a number of benchmark data sets on which the performance was evaluated. And then you have GPT J, the llama original llama 7B and now the open llama 7B preview model. Now if you look at the table, overall the Results of the Llama 7B and Open Llama are very comparable. There are definitely a few cases in which it doesn't perform as good as allowance llama 7b. So for example, if you look at this model or this model, for those two data set, the original LLAMA performs better than the open llama. However, there are several cases. For example, on this ARC Easy data set that the open LLAMA model actually outperforms the original LLAMA model. There are a few other cases as well where you can see that the performance of this model is definitely better than the original model. So let's say here's another example. Now this is significant because if you think about it, the model is being trained with only 20% of the data that was used for training the original LLAMA model. Now this shows you that with well crafted and carefully curated data set, these models can actually perform really good, even if the training data side is much smaller. Now the great thing about this project is they are already releasing the weights of the model. So you don't really have to request anything. They are on Hugging Face Hub. So if you go to Hugging Face Hub, uh, it's under the Open ML Research, I believe it's a project out of UC Berkeley. Now if you go to files and versions, so there are two different model files. One is this EasyML format and the other one is the hugging face normal Pytorch format. So you can see them here. Now, if you're using the EasyML platform, then you don't really need the Llama tokenizer and weights because they have retrained the whole thing from scratch for using weights in the transformer library, which is on the hacking phase using Pytorch. So you can actually use the transformer package. So in this case, from the transformer package you will actually need the LLAMA for causal LM and the Llama tokenizer. But instead of giving it the weights or delta weights of the original llama model, you will simply provide these new weights to this model. Right? And similar is for the tokenizer. The only difference, the only thing that you need to be careful about is that they use beginning of a sentence token during training. So ID equal to one is actually a reserve token. Now so you need to append that. But other than that, the configuration will remain exactly the same because it's using the exactly the same architecture. You don't really need to change anything at all. You will be just loading different weights. Now, as I alluded to in the beginning, this is actually a preview model, so it's not a fully trained model yet. They plan to train this model on the entire red pajama data set. So that is going to be 1.2 trillion trillion tokens. And the best part about this whole project, which I am really excited about, that they are also training us much smaller 3 billion model in hope for facilitating large language model usage in low resource use cases. So with a 3 billion parameter model, you probably will be able to run this on a consumer hardware without the need for expensive GPUs. So if that actually works, I see a lot of different use cases where this model can be actually used. So this video is a little different because I don't really have a demo to show you or to compare its performance with other models out there. But I think it's significant because with this model now you can actually have a baseline model where other models can be trained on top of without worrying whether you can use it for commercial purposes or not. It's a pretty open license so you can actually build commercial products on top of it. I have been covering a lot of open source large language models lately and one of the comments that I see repeatedly is that these models are not as good as ChatGPT or ChatGPT or ChatGPT 4 are the best models out there. I don't think anybody actually dispute that. But the importance of these open source large models models is twofold. One, if you are concerned about data privacy, you can run them locally. The second, which I think is more exciting is you can retrain these large language models or fine tune them on your very specific business use case and they will actually outperform these much bigger models because these smaller models would be task specific. And that is, I think the most exciting part about them. If you guys have any questions or comments, put them in the comment section below and I'll try my best to respond to them. As always, if you like the content, please consider subscribing to the channel. If you haven't. If you have subscribed, please consider pushing that bell notification button so that you get a ping every time I upload a video. As always, thanks for watching. See you in the next one.