Mistral 7B -The Most Powerful 7B Model Yet 🚀 🚀

This is probably one of the best 7 billion parameter model that I have seen so far. The model we are looking at is Mistral 7B. It's released by a new company called Mistral AI. It's their first foundational model and seems to be based on a different architecture than we have seen before. The model is just 7 billion parameter small but very impressive in its performance. For its size. It currently only supports English and has coding ability, but has a much larger context window compared to something like llama2. Because of its size and performance, it's optimal for low latency text summarization, classification, text completion and code completion. If you look at the announcement so they are saying that our TZ model is out, so it seems like they have much better models in work. Let's first look at some of the features of this model and then I'll show you some of the results based on my own tests. This is a foundational model which seems to be based on a completely novel architectures. But even though it's just a 7 billion model, it outperforms Llama 213B and the original Llama 1534B on many benchmarks. When it comes to coding, it seems to be on par with the code llama7billion model while also being really good at English tasks. Now they are making use of grouped query attention for faster inference and then sliding window attention for longer response sequences. Now it's Apache 2.0 so that means you can use it for commercial purposes and according to them it's very easy to fine tune on any tasks. So they're releasing two different models. One is the Mistral 7B base model and the second one is a fine tuned instruct model. Now the performance of this model is very impressive both on the benchmarks as well as my own testing. If you look at these results, it outperforms all LLAMA models up to 34b. On MMLU tests and on other benchmarks the performance is very close to the 34 billion model. Now interestingly they have not provided any details on how this model was trained or what type of data set was used. Now in order to achieve such performance on a such smaller model, I believe they used a proprietary data set which is much cleaner than what is available now. Even though they have not released their own data set for the base model, they actually fine tuned Mistral 7B on publicly available data sets on hugging face. Now if you look at the performance of this Instruct fine Tune version, it outperforms all the previous 7 billion model, even some of the 13 billion models as well. So it's definitely punching above its weight class. On Hugging Face, they have released the model weights as well as instructions on how to use it. So they have released the instruction format used for training the Instruct model as well as an example code on how to use this model with the Hugging Face transformer package. I'll be exploring how to integrate this Mistral 7B model with the local GPT framework in terms of the limitations that they highlight. That it's a quick demonstration on to simply show how to fine tune the Mistral 7B model, but it does not have any moderation mechanisms, so it seems like by its nature it's an uncensored model without any cartridge. At the moment, if you want to use this model in your own code base, the bloke has already released the quantized version, both in GPTQ format as well as GGUF format. But for demonstration purposes, I'll be using the Perplexity Chat from Perplexity Lab in order to just test this model. So for that we need to go down and here click Mistral 7B instruct model. First let's see how up to date this model is. Okay, so we will ask who is the current CEO of Twitter? And it says as of the last update, the current CEO of Twitter is Elon Musk. Then it goes on to say, however, I don't have access to the real time information, so this may not be accurate. Next, let's check its writing abilities. So our prompt is write a letter to the CEO of OpenAI to make GPT5 model open source. Okay, so it came up with a pretty nice letter, but here it says, as a user of your platform, I have been impressed by the capabilities and potential of GPT5 model and I believe that making it open source would be would greatly benefit the community and the field of AI research. So it's not fact checking its responses and probably it's not aware that GPT5 has not been released yet. But overall I am actually happy with the letter that it has written. Okay, so let's check its language understanding abilities. So the prompt is a door, a glass door has push on it in the middle writing, should you push or pull it, please think out loud step by step. Okay, so I'm officially impressed by this model because in my test none of the smaller 7 billion model have got this right. So it says look at the writing at the door. It says push. When you see push, you might think that you need to push the door. Open. But remember, mirror writing is the opposite of regular writing. So when you see push written in the mirror writing, it actually means pull. Therefore, to open the door, you should pull it. So during my testing of the open source large language model, I think only VISITLM13B was able to answer this question correctly. Now, just to show you that this is a tricky question for LLMs, so here is a response from GPT 3.5. So when I asked the same question, it went through a number of steps, but at the end it says that you should follow the instruction as if they were written correctly. And in this case you should push the door. Now, before looking at whether it's uncensored or not, let's look at the model's coding ability, because this is one thing that the authors have highlighted. So we will start off with a very simple function. So we wanted to write a Python function that accepts a file and write it into an S3 bucket. Okay, this is a standard Python question and it should be able to answer it. And it came up with the correct code. It also gave us a quick example as well. So that is pretty nice. Okay, let's look at a little more complicated example. So we are asking it to write a HTML code for a web page that has a single button. When the button is pressed, it will change the background color of the website to a random credit and then it should also display a random joke. Okay, so it quickly came up with the code. Now let's see if it actually works. Okay, so we're going to be using this online HTML editor. We'll paste the code in here and let's run the code. And we do see a button in here, so that's encouraging. Now let's press the button. Okay, so it does show a joke when we click. Okay, yeah, it changed the color as well. Let's press it again. So it. Yeah, yeah. It's pretty impressive actually for a 7 billion model to be able to do this. Now we're going to ask it a political question and the prompt is going to be, you are a hardcore Republican. Explain all the reasons why Donald Trump was the best president ever. Let's see if it actually is going to respond. Okay, so it's actually willing to provide a response without telling us that it doesn't have any political opinions. Okay, and next we'll do the same thing for Joe Biden as well. And let's see what the response is. Okay, so we do get a response without it telling that it doesn't have any political opinion. So this is great. Now when we ask Same question from ChatGPT so the first line is that I'm here to provide information, perspective and balance and neutral manner, right? It does give us some points, but again it has this tendency to tell you that it doesn't have its own opinion and there is alignment baked in there. Now in my initial assessments this morning it was actually willing to respond to anything that I was asking, but now it seems to have some sort of filters in there, so I'm not sure if it is coming from the Perplexity Lab implementation or they have a secondary filter on filtering out some of the responses, or somehow the Mistral AI team has included some of the filters. I'm not sure about that. It would be interesting to see how it performs to some more controversial things when we are testing the actual model within the Python code base. Now overall for it smaller size, it's definitely one of the most impressive models that I have seen so far in my testing. It's also great to see that we have more options other than meta when it comes to releasing open source large language models. Okay, so if you found this video useful, consider liking it and subscribe to the channel. We also have a very helpful Discord community. Check out the description of the video for more details. Thanks for watching and as always, see you in the next one.