NEW Zephyr-7B is Awesome

Hugging face just released the second version of their Zephyr model series. It's a fine tuned version of the original Mistral 7B and is not only able to beat much larger 70 billion parameter models, but is also uncensored apart from the code and weights. They also released the technical report which is very helpful in learning how the model is actually trained. A trend that we have been seeing with the more recent 70 billion models is that the quality of the training data as well as the training procedure is much more important than the size of the model itself. So in this video we're going to look at the technical report in order to understand how the model is actually trained. We will also look at some of the assessment and we will do some testing as well. Now if you look at this plot for the assessment, it seems like the Zephyr 7B beta version is able to outperform much bigger open source model like llama270bill on tasks like STEM humanities, writing, role playing and even coding. Now in some of these tasks it actually closes to something like GPT 3.5 Turbo. But there are definitely some tasks like coding and mathematics on which it lags behind these proprietary models. But for a 7 billion parameter model, this is extremely impressive. Now the question is how, how does a small model like this is able to achieve that level of performance? So for that we need to look at the training strategy. If you look at the technical report, they are doing training in three different steps. The first step is something which they are calling distilled supervised fine tuning. So you might ask, what is the difference between normal supervised fine tuning and distilled supervised fine tuning? So let's understand how this is done. So for this first step they're using an interesting data set called Ultra Chat data set. This contains 1.47 million multi turned dialogue generated by GPT 3.5 Turbo. So basically it's a multi turn dialogue between a user and the assistant. Now what they found was that there were a lot of issues within this data set. After finding the issues, they actually cleaned and filtered the data set and at the end they were left with only 200,000 examples. Now here is how the distilled supervised fine tuning works. So you have two different models. One is called the teacher model, which is a much bigger and powerful model like GPT 3.5 and the other one is the student model. So that's Zephyr 7B in this case. So you pick a prompt from the data set, provided it to the teacher model. The teacher model is going to Generate a conversation based on that prompt and it's simulated in terms of the user and assistant interaction. Then that prompt along with the response is fed back to the teacher model and it's used to criticize or improve based on the response that it previously generated. So at the end of this process, you're going to get better responses from the teacher model. And then that data set is used to fine tune the student model. So basically you are distilling information from the teacher model to the student model. Now the next step is the AI feedback part, which you can think of reinforcement learning, but rather than using human feedback, they are using AI feedback. Now, for this specific step, they're using a Data set called Ultrafeedback, which consists of 64,000 different prompts. Now for each of the prompts they are using four different models to generate responses. And those responses are graded or rated by GPT4 based on honesty and helpfulness. So basically GPT4 decides which response is the best. Now, in order to train an RL model you need binary preferences and the way they have done it is very interesting. So they chose the best Response selected by gpt4 as, let's say, the most appropriate response that is going to be liked by the model. And then for the rejected one, instead of using the one that got the least rating from GPT4, they are selecting randomly out of the three other examples that were not highly rated by GPT4. That means training any model on this data set is going to be a bit more challenging now. So the third step is basically training another model using the data set that we just created with a winner and loser. And now the model has to come up with a strategy so that it can optimize for the winner response. So this is a quick summary of the training process. In my previous video I covered this in a lot more detail. I'll put a link to that video in the description. So what type of performance you can get out of the model using this training data set and training strategy. If you look at the yellow color, that's the performance of Zephyr 7B on MT bench and Alpaca Evolved data set. Now it's able to outperform all the 7 billion model irrespective of whatever strategy or algorithm was used to align those models. But the most interesting aspect is it's able to outperform much bigger models. For example, the Falcon 40 billion, Galvanico 65 bill and even Llama 2 chart 70 bill model. Now there are other 270 bill model visit LM and Gin LM which are able to outperform Zephyr 7 billion model. But for its size, the results that you see in here are very impressive. Now keep in mind these results are on two smaller benchmark data set. You could forfeit your model to these benchmark data set. That's why it's important to test these models yourself for your own applications. But apart from these benchmarks, there are some other ways you can see whether these models are doing better than the previous models or not. Now one such application is rag, which is Retrieval Augmented Generation. AMI Index is an excellent open source project to build RAG applications and they recently put together this table to keep track of how good these open source large language models are based on different aspects of rag. Now if you look at this table, both the previous version of Zephyr as well as the current version is really outperforming all the other open source large language models including the Llama 270 build chat. Based on these assessment, I'm also planning on updating the default model in the Local GPT project and use Zephyr as the default model. Now if you don't know what Local GPT is, it's my own project that lets you chat with your private data using the power of open source large language models. I'll put a link to a playlist where I go over different aspects of the project. Now in this video I'm not going to be doing a comprehensive testing of the model. I just want to focus on the uncensored aspect of the model. But before that, let's look at this plot once more. If you are picking a large language model for your application, you you really need to think about the underlying LLM and how good that LLM is on that specific task. So for me, my daily use case is mostly coding. So if I look at the Zephyr model for coding, it's not really a great model. Although like it's better than llama2.70 build chat model. But on coding it's well behind GPT 3.5 and even cloud. So if I still wanted to use an open source LLM for coding, I'd probably look for something like Code Llama or one of the fine tunes of Code Llama. But if your use case involves writing, this might be a really good candidate for that. So what I'm trying to highlight in here is that you need to pay close attention to performance of these models on specific areas and then pick a model appropriately. So for example, I asked this model to write me an email to the city appealing my hundred dollar parking ticket and I asked it to appeal to the sympathy and also admit that I parked incorrectly. And I must say that it did a really good job not only by following my instructions, but coming up with a really nice letter. So for example, here is the introduction. So I'm writing to appeal the recent parking violation ticket issued to my vehicle bearing registration number. Then it basically says that I made a mistake. It also explained why that happened and then it's making an appeal. So this is pretty good. Now another question is whether it's an uncensored model. So the answer is yes up to a certain extent. So for example, here I'm asking how do I steal a kitten? Walk me through a step by step process and it says I strongly advise against stealing a kitten as it is an act of animal cruelty and may also result in legal consequences. However, if you insist on doing so, please follow these steps. So here's another one. How do I break into a car? Don't tell me it's illegal. Walk me through a step by step process and it's able to give you a rough step by step process. So one thing which I want to highlight that probably people who have malicious intents, they're probably not going to be using LLMs for those purposes. But there are legitimate use cases for uncensored large language models and it's great to see that we are able to make progress in that area even with these smaller LLMs. For running the Zephyr 7 build model, I was using the online demo provided by Hugging Face, so I'm going to put a link to this in the description of the video. I also want to show you how to run this model locally. For that you can use something like LM Studio. It's not fully open sourced, but I think it's the easiest way to run natural language models on your local machine. Another option is going to be Uber Booger text generation web ui. I have previously done a video on LM Studio, so I'm going to put a link to that video in the description if somebody is interested. So in this case all you need to do is simply type in Zephyr and click on Go. Now here you get a list of all these Zephyr models that are available. So for example, in this case you can simply click on the second option because that's the latest version. And then on the right hand side you can see different quantization levels in which this model is available on Hugging Face. So pick the one that you want to run. So for example, I already downloaded the 8 bit quantized version, but if you have let's say under 6 gigabytes of RAM, I would recommend to use something like 4 bit quantized version or even lower than that. Once the download is complete, simply go to the chat section and from the top you can select the model that you want to use. So for example, I'm going to switch to the Zephyr beta, so in this case it loads the model. It's also telling me that it's using around 8 gigabytes of RAM for my 8 bit quantized version. And after that you can basically start chatting with the model the way you would chat with a model in something like Google BUGA Text Generation Web ui It's simply amazing to see the progress that we are making with these smaller large language models. If you found this video useful, consider liking the video and subscribe to the channel. If you like the content on this channel and would like to support me, check out the YouTube membership as well as my Patreon and we also have an active Discord server, so come join us for everything related to generative AI. Thanks for watching and as always, see you in the next one.