Stable-Vicuna: The Most Powerful Open-Source LLM Yet?

Stability AI, the company behind Stable Diffusion, just released their second large language model. Now this one is called Stable Vircunia and they claim it to be the world's best open source RLHF LLM. If you recall, a few days ago they released stablenm. It was their first large language model. So how is this new Stable Vicunia different from stablelm? So this model was trained with reinforced learning from human feedback, which is a very similar approach to what OpenAI is using for their models including ChatGPT and GPT4. The base model itself is Vocunia v0.13 billion parameter model. It was further instruction fine tuned on a large data set and then it was trained using this RLHF. Vocunia itself is a fine tuned version of llama 13B model. Now that makes it very tricky for commercial use because Llama's original weights are not open source. Theoretically you still need to request Meta or Facebook to get access to those weights. Now the main question is, does using RLHF even help the model or not? So in this work Stability I actually provided a comparison of this model with similarly sized models on different benchmark data sets. We will be conducting our own tests in this video and I'm also going to show you how can you potentially use this on your local machine if you have a powerful enough gpu. But based on these results, in some of the tests or benchmarks it actually performs pretty good. So for example, this bool queue, it actually outperforms all the rest of the models. Similarly on this data set mnli, it is able to outperform the rest of the models. However it's not really consistent. For example, if you look at this truthful QA then it lags behind and same is the case in some other cases. So let's say in this case there's ARC challenge. So it lags behind, but overall it's up in the first or second position. It's not too far off. Now in most of the cases the appropriate comparison is going to be with Vicuna 13B parameter model rather than the rest of the models. So in all the cases, if you compare this stable vicuna with vicuna 13B, at least on these benchmarks it outperforms Vicuna except in this truthful QA benchmark. So this is very significant because as far as I know, Vokunia is probably the best open source model which could be compared with ChatGPT in some cases. Now how did they achieve this? So they actually did two steps fine tuning. In the first Case they used a newer data set. So this is a combination of three different data sets to fine tune the Vicunia 13B model. The data set that we used is open assistant conversation data set which is a human generated human annotated assistant style conversation. Next data set is GPT for all prompt generation. This data set is prompts and responses generated by ChatGPT. And the last one is the alpaca data set which is again structure and demonstration generated by OpenAI's DaVinci model. Now a couple of things to consider here. With the fine tuning, two out of the three data set are relying on ChatGPT. So the answers or responses from this model could be skewed towards responses that you would expect from ChatGPT. The second stage is where they do the reinforcement learning with human feedback. Again, they're using three data sets. So one is the open assistant conversation data set containing around 7,000 preferences samples. Then there's an entropic reinforcement learning dataset and then there is the one from Stanford human preferences dataset. As the great thing about these data sets is all of them are open source and you can actually look at how they look like. So for example, we are looking at this anthropic data set. So in this case, human was given two responses from the assistant for the same prompt and then the human chose which response is good. So in this case, for example, you see that the prompt is the same but the assistant gave two responses. And then on the left hand side you can see the one that was chosen by human on the right hand side, that's the one that was rejected. So this is how they trained it using this reinforcement learning technique. Now apart from this model, they also announced another feature which we're going to talk about in a second. But let's talk about how you can actually get access to this model. As I said, it's an open source model based on vocunia 13 billion parameter which is based on the number of weights. They have made the delta weights available and you can even play with this locally if you have a powerful enough machine. We're going to be looking at hugging face spaces where we will be experimenting with the model. But at the end of the video I will show you how you can potentially run this locally or in a Google Colab if you have powerful enough machine. So if you recall a few days ago I showed you this hugging chat interface from Hugging Face. Now Stability AI is also announcing the upcoming Chatbot interface. This is great because this shows that we're going to have Multiple options. So it's really awesome if you don't want to be sending your data to OpenAI. Now let's experiment with the model itself. To access the model you can go to Hugging Face and then click on spaces and then you want to search for stable Lacuna. I am going to put the link to the model itself. So here we have the link from Carpool AI. So let's click on this. This is their chat interface. So let's first test whether it's any good for ideas generation. So the prompt is create A list of three startup ideas in the enterprise B2B SaaS. The startup ideas should have a strong and compelling mission and also use AI in some way and we want it to avoid cryptocurrencies and blockchain. So it came up with three ideas. One thing is that at least the names are original. I asked the same question from other open source anyway chatgpt and most of the time it comes up with product names which already exist. But in these, in this case all the three the names that it came up with are actually original. So that's pretty nice. And the second part which I really liked is especially this insightful dairy. This I haven't seen before at all in any of my other interactions with these large language models. So it says a B2B SaaS platform that uses AI to predict the optimum time for dairy performers to schedule their milking. The platform can then automatically schedule and optimize milking to maximize efficiency and minimize waste. The mission is to help dairy farmers increase production, productivity and profitability while reducing environmental impact. This is pretty cool. As I said, I haven't seen this before at all. The next two is what you would expect. Probably you guys all have already seen it. So the second one is generate custom tailored financial forecast for small to medium sized businesses. I think this is already being done. And the last one, Mindful Manufacturing. It's a AI platform to automate and optimize manufacturing processes. So that's also something which is already available. But I really like this insightful dating idea so I'll give it a next I went and found this article three ways AI can help you build your course outline hint for you guys and copied this all text and ask Stable Vicuna to actually summarize it. And I said things step by step. So here is the summary and it's actually it did a pretty good job. It was able to get the three main points that the author is trying to convey. So author is basically talking about how you can leverage AI in course creation and it came up with a pretty nice summary. One of the example that they have shown in their blog post is that it's able to do basic math. So for example, they are asking you to add 2564. The answer is 89. So I thought I'm going to replicate this. But it didn't go as planned. So in this case I asked, can you do math? It said, yes, I can help you with math. Right? So I said, what is 26 plus 9? The answer is 93. That doesn't seem to be right. Right. So I asked it, can you explain step by step? So it says you add two numbers and the answer should be 93. Then I checked it, what is one plus one? It's giving me the correct answer. Then I asked it again, what is the sum of 16 plus 9? So this time it got it right. So it really depends because I asked the same question again and this time it gets the correct answer. So another test was the square root of 81. It is able to figure out it's 9. But based on my test, I think you can't really rely on it for mathematics. It's all over the place. The answers are not consistent. It will just make up numbers sometimes. So probably best to avoid it for mathematics. Now, just to be fair to a stable Vicunia, all natural language models have trouble solving mathematics. Okay, here's another use case. I asked it to write a job description of an experienced machine learning engineer, include all the necessary skills. I think it came up with a pretty good list. But I have seen that some other large language models can produce more detailed description than this. But it's still a good starting point. Now, before looking at its ability of generating some mildly controversial responses, let's look at its ability to program. Now, I use a couple of simple tests to check its ability to program. So the first one is write a Python function to write a file into an S3 bucket using the Boto library. And if you search online, you should be able to find this. There are multiple implementations, so most of the models are actually able to implement it flawlessly. That's the case with stable Coconia as well. Now, the more interesting one is this where I have seen some of the large language models have trouble implementing it. So I'm asking it right in HTML code for a website that has a single button. When the button is pressed, it randomly shows one joke from a list of 10 jokes and display that to the user. A button press also changes the color of the background. Right? So I Do see, there are not 10 jokes, but I would say five jokes and then it says so on. So you can actually add to this list of jokes. That's fine. But let's look at if it actually works. Most of the large language models that I have used, they actually have trouble implementing this. Okay, so I'm going to copy and paste the code here. If you don't know, this is the HTML editor from W3 Schools, so you can actually run this online. Let me just run this. Okay, so it does show a button get random joke and then it says loading. Right? Let's click the button and see what happens. Oh wow. It worked. That's pretty neat. Although for some reason it's not changing the color. Oh, okay. I think it got that part wrong because it simply set the color to blue, so it's not really modifying the color at all. But I think it's still pretty neat. It got the first part right. Now, since it's a chat interface, you can actually go and tell the model if there's something wrong. So I said the color is not changing. When I click the button, it always. It's always blue. With each button click, the color should also change to a random color. Right? And then it went ahead and said that you can modify the get random joke function. Right. So it re implemented that. So let's see if it actually works. I think right now it's selecting out of those five different colors, but that's fine. Okay, now I need to actually replace this function. So I'm going to copy this. Right? Paste it here. I don't want. I don't care about the formatting right now, but let's run this. Okay, so. Oh wow. Okay, nice. So it actually works. This is pretty cool. So now you. Since we have the ability to actually chat with the model so you can tell it the edits just like ChatGPT and it's able to update it. This is pretty awesome. Okay, so the next two ones are simply asking it to pretend either to be a Democrat or Republican and explain why their respective policies are best for the country. It does give me a response that's pretty nice to see. If you ask ChatGPT or variation of this question, it probably is going to say as a large language model, I cannot do this. So it's good to see that it's able to do it like we have a response for the same question when we ask it to pretend it to be a Republican as well. For some reason it added this stable in the start. Probably that's the token it's using. Now let's see how you can actually use this locally or in your own applications if you have a powerful enough gpu. Unfortunately I don't really have access to a really powerful computer, so I cannot run these on my local machine. But there is a simple way around it. So you all you need is to install the transformer library, right? And the rest of the process is very simple to what you would expect from a transformer based model. So you need to have a tokenizer since it's based on llama. So you can use the llama tokenizer and then you need to have automl for causal lm. So you need the model itself, they provide it here. But this is the delta model. But there are good people of the Internet. So for example this person, the block, he has provided an uncompromised model in the hugging face format. So you can actually simply copy this and replace it with replace this path to the unquantized model, right? So you can simply put that path here and this will download the corresponding model and you can use it. Now you just want to be careful about the prompt. So the prompt has to be provided in a very specific format. So the prompt starts with three hash signs or pound signs with the space and then human. And after this you need to provide your prompt and then you need to append it by three pound signs and then an assistant and after this backslash this is when it's going to start giving you the response. So it's a very particular format that you need to make sure you are following. Then the rest is very simple to what you would expect when you were working with a model from hugging face. Now as I said, unfortunately I don't have access to a powerful machine so I can't run this locally. But the process should be very simple to follow. Now final thoughts on the model. It is a very powerful model indeed. There are definitely some issues. In some cases it doesn't perform as good as but you would expect the performance to be on par with the original Vicuna 13 billion parameter model. The great thing is it's the first step in training open source models with reinforcement learning with human feedback. So in future or really near future, we can expect to have powerful open source model that could potentially compete with ChatGPT on very specific tasks. And I want to highlight this, the power of these model is not a disability to generalize, but they are going to be trained for very specific tasks. If you have any comments or questions, please put them in the comment section below. Hope you like the video, so consider subscribing to the channel. If you already haven't. Thanks for watching. See you in the next one.