Meta released Llama 2, a free open-source AI model available for commercial use. The 70B version doubles the context window to 4096 tokens and was trained on a trillion tokens. It outperforms other open-source models at the same size and comes with built-in safety guardrails.
Questions this video answers
July 18, 2023
7 billion, 13 billion, and 70 billion parameters
4096 tokens
Trained on ~40% more data than Llama 1
Approximately 1,000,000 examples
Chapters
But it was actually able to figure out the answer. Okay, so this is huge. Meta or Facebook just released Llama 2, the second version of the Llama model, which basically started this whole revolution of open source large language models. One of the biggest limitation of the original version was that it was only for research use, but this time it's available for free for both research and commercial use. So this is going to be huge. Now you can actually request to download both the weights as well as the corresponding inference code. So you will have to fill out a form and then they will send you the code and the model weights. So what is actually new about this second version? So they are releasing three different models. One is the 7 billion parameter model, then 13 billion and then the biggest one is 70 billion parameter model. If you remember in the initial release of Llama There was a 65 billion parameter model which was the biggest model that they had. Now according to them it's trained on 40% more data than the llama one and the context size is now doubled. So instead of 2048 tokens, now the context window is 4096 tokens which is great. So now this is going to be a lot more useful than the current version. Now apart from the base model that is trained on true trillion tokens, they are also releasing a fine tuned version for chat and that is trained with 1 million examples using the RLHF method. So it's trained or fine tuned with reinforcement learning with human feedback. This is the same technique that OpenAI is using for training their models with. Okay, they also have shown some benchmarks and I think this is the most important part. So on the left the column you see different data sets or benchmarks, data set and then performance of different open source large language models. So let's first look at the 7 billion parameter model. They're comparing it with the MPT7B and the Falcon 7B. Now if you look at this specific coin column, it actually blows everything else out of the water for a 7 billion parameter model. Now if you are looking at the 7 billion parameter model, so this new LLAMA 2 model consistently outperforms the other 7 billion parameter models and actually by a large margin, except I think this specific case, the human evolve. So in that case the MPT7B model is performing better than the Llama 27B. But overall if you see for the rest of the benchmarks, it's actually outperforming the previous 7 billion parameter models. Another interesting trend is to notice when it comes to the comparison with the previous 65 billion parameter model. And again you can see that it easily outperforms the existing model for the 65 billion and 60, 70 billion. In most of the cases, this new model outperforms the previous 65 billion parameter model. Now if you compare the 7 billion to 13 billion parameter model, there is a huge difference between the performance. So I'm assuming that most of the people will be either trying the 7 billion model or the 13 billion parameter model and both of them gives huge improvements compared to the existing open source models. Now from the description it seems like there is a chat version of this Llama 2 model that is using reinforcement learning from human feedback to ensure the safety and usefulness. You probably recall that RLHF is a technique that was suggested by OpenAI. That's how both the ChatGPT as well as GPT4 models are trained. And it seems like Meta is also adopting the same technique for training the second version of the Llama models. Okay, so Meta is also highlighting the safety aspect of this model as well as there is Llama impact challenge that you can apply for. Even though if you do not have access to the model weights and the entrance code yet, the open source community was quick to put together an app that you can use to try it out. Someone actually put together this streamlit app that you can run two different versions of the llama 2. So the one is the 13 billion parameter model and I think there's an option of running 7 billion model as well. So I use this just to play around with the model and it actually is pretty good. So on the left hand side there are some parameters that you can change, but I'm going to keep everything to the default values. Okay, so I'm using this test prompt. A glassdoor has push on it in middle writing. Should you push or pull it, please think out. Think out loud, step by step. And this is a prompt that I have used for other open source models and this seems to struggle with it, but it was actually able to figure out the answer. So it says first I would like to point out that the word prompt push is written in the middle writing, which means it's reversed from how we normally read it, right? So if you were to follow the normal convention of reading from left to right, right. So it's going to look like this, right? However, since the word is written in the middle writing, it's possible that the intended intended meaning is the opposite of what you would normally expect. So in this case this would mean that you need to pull it right and that's actually the correct answer. This is pretty great. In a quick experiment, it seems to be working pretty nicely. Okay, so here is another prompt. Explain the correct usage of homophones in the following sentences. The principle is your pol and the site at the site was quite a sight, right? And again, it actually figures out both of them accurately. I kind of like that it's putting these emojis in its answers and probably it has to do with the system prompt that is being used within this streamlit app. So in this case it gave us what homophones are. Right. And it's able to figure out that principal and polar homophones. So that's pretty nice. In the second sentence, it figured out that site and sight or homophones. Okay, now it does seem that there are some guardrails or safety mechanisms, so it's not going to give you answers for every questions that you ask. Now overall, it's great to see the release of this second version. This was a much anticipated model and everybody was looking forward to it because Meta said that it's going to be available for commercial use. I hope this will really push the big giants like Microsoft, Google and OpenAI to open source some of their models. And it's kind of surprising to see that Meta or Facebook is the one that is actually releasing all these open source tools. And we need to thank Meta for their initial Llama model because of that. We have all these different flavors of open source models. So once I have access to the weights and inference code, I will be doing more thorough tests on this Llama 2 model and we'll also look at integrating this as a part of the local GPT project. Okay, so this was a quick video. Just wanted to cover the release of this new model and I'm excited to see what people will build with it. I hope you found the video useful. If you did consider subscribing to the channel. Thanks for watching and see you in the next one.
