Wizard-Vicuna: 97% of ChatGPT - with Oobabooga Text Generation WebUI

Hello everyone. Today I want to show you how you can run these large language models locally by downloading them on your local machine and using your own hardware. This method works with almost all of the open source models such as Ocunia, Alpaca, Llama, Dall E, wizardlm, stablelm and you name it. Now the tool we are going to be using is called Text Generation Web UI from Uber Booka. It's actually an amazing tool which has supports for a number of different models as now this is a pretty amazing tool because you can simply add a model to it and you use the same web UI and their goal is to become the automatic 1111 of text generation. So if you have played around with stable diffusion, there is an amazing tool called Automatic 1111 that lets you use multiple stable diffusion models in a single web ui. And this web UI has some really amazing features. So as I said, it's a graphical user interface. You can set it up so that it looks like OpenAI's playground. And the great thing is there's actually a role playing capabilities so you can actually set up different characters and do a role playing. So this is pretty neat. And then as I alluded to before, it supports the instruct mode for different models that you see. Another great feature that I'm going to be showing you in future video is you can actually combine this or integrate this together with stable diffusion so you can send in images and generate audio responses from your text generation using 11Labs API or any of the other text to speech converters. Now the great thing about this tool is you can actually run it both in CPU and GPU mode. So if you have a powerful gpu, it's amazing. But if you want to run it on a CPU you can still do that. And it also supports the 4bit GPT Q4 mode. So that's also helpful if you have a weaker hardware. So in this video I'm going to be walking you through the installation process of the text generation web UI and then I will show you how you can download and use models and Lokuni. For this specific example, we are going to be looking at this newly released visit Vicunia lm. So for this model they actually combined the powerful Vicunia model, which was around 90% of the ChatGPT quality, with this new model Visitella, which introduced a new instruction following technique. Even though WizardLM is a very small model, but it shows amazing results and performance. So I have separate videos both on Vulcania and WizardLM. I would recommend you guys to check those out. So before looking at the installation of text generation web ui, let's quickly go over what this visit Vocunia NM is. So this is a model which claims to improve the Vicunia performance by around 7%. So now they claim to have the performance of almost 97% of the ChatGPT, which is, I think, a big claim. Now, in order to achieve a performance like that, they combine the approaches of WizardNM and Vicuna. WizardNM, a new approach for instructional following. So in this case you trained a batch navigate model or you use an Android model to create more complex instructions for training your model. So here's an example. In this case, let's say the GPT4 was provided with an example conversation between human and ChatGPT and then it's asked to create 10 more conversations where human ask question and ChatGPT answers. Right? So using this as a tablet, GPT4 created this whole conversation where human is asking question, ChatGPT is responding, then human again is asking a question, ChatGPT is respondingS, right? And this is a whole conversation that takes place between human chatgpt and the whole thing is created by gpt4. So then they took a data set like that feed into Vicuna or fine tune the Vokunia model using the method that was proposed for Vokunia version 1.1. So the version 1.1 is actually the second version of Vocunia. Now, after training this on the conversations, they went ahead and compared the performance of chat GPT. Visit Vokunia LM Vokuna LM at the original Wizard LM and used GPT4 as the judge. So let's look at this, just an example. So the way it was done is that ChatGPT was given a question and responses from these different assistants. So Assistant 1 is, I think chat GPT. Then there is a response from Assistant 2, right. And so on and so forth. And GPT 4 was basically asked to judge it, right? And. Or score it out of 100. Right? So here is just the score coming out from GPT4. So for example, in this case like 91, 98, 85. Right. So it's not really a comprehensive scientific evaluation, but it's still a starting point where GPT4 is used as a judge. And based on these results, according to the authors, they showed that ChatGPT scored on average 91, whereas Wacunya on scored 88, which is not very far behind. Right. If you look at it, then the original VPN has scored 82 and Wizard LM, which is only a 7,7 billion parameter model. It score 18, so it's still a respectable score. But if you look at it wizard, Vukunia is very close to ChatGPT, at least on this evaluation data set. Now this gives you an idea of what type of model we are dealing with here. I might create a separate video on Visit Vicuna island and compare it with Vicuna. But right now let's see how we can run this vocally. Okay, so let's go back to the text generation web ui. If you scroll down there is an installation section where they provide a one click installer. So it's provided for Windows, Linux, macOS. So depending on your operating system, simply choose the one that works for you if you want to install it manually using Conduct. So there are details step by step instructions. So you can even go here. But to keep things very simple, I am going to be using this one click installer. Now I'll show you the installation instruction for Windows. But I am running this on a Linux machine because I have a GPU on my Linux machine. So in order to install it on Windows, simply download the zip file and the instruction remains pretty consistent across the operating system. So if I show you one, you will be able to follow it in another one. Okay, so it downloaded the zip file. Next we need to extract it. So I'm going to just say V2 because I already have one. All right, so after extracting it, go to the folder. Now you will see there are a total of five files. One of them is a text file. So this gives you a detailed instructions of how to actually install it. So you need to run the script that starts with Start. So this is going to be Start Windows or Start Linux or Start Mac os. Right? So first we need to just run this. Now it gives a warning. So it's up to you whether you want to run this or not. All right, so I'm going to click on Run anyway. Now this step is going to take some time because it has to download a few files. So be patient in the installation that it creates another folder in the same base folder where it's downloading the installation files. Okay, next just asks you what type of hardware you have. So if you have an Nvidia gpu, so simply type in A for amd, type B. And if you are on Apple Silicon, so type C for this specific computer I don't have a gpu, so I'm going to type in D. But if you have a gpu, just select the corresponding option. Now once the installation is complete, you will see this new folder called Text generation Web ui. You can actually go in here and there's a folder called Models. So you can download the models here and just copy them and it will work. Or there are a couple of other ways you can do it. So let's say you simply click on Start Windows. Okay? Now you're presented with a whole bunch of options. These are pre populated models. You can actually select one of them, it will download them. Or in this case we are going to be manually downloading a file from Hiking face, right? So. So there is two ways you can download models from Hugging Face. The one is which is listed here. So you simply select L. That means that manually specify hugging face model, right? And it has to be. They can just format the example, use it and then the model name. Now in this video we want to download the wizard with Kunya model, right? So if you type in or like search for that model. So you would see this second link which is the block visit vicuna 13bit ggml. So you want to look for a model that has GTML in it, right? And then you simply copy under the link. Now we go back to our command window, right? Type in that link or paste that link. Right? And then you can simply click Enter. It will start downloading the model. Now once the download is complete, you can go to the next step. Okay. Once the installation is complete, simply run the start script again and it will show you something like this. As I said, I'm on my other machine right now. But in this case what it's doing is it's loading a model from models and the model name is the block wizard vicunia 13 billion parameter model, right? And there's a dot bin file which it's loading. Now once the model is loaded, you'll be actually presented with this IP address. This is your local host. So you can go to this IP address and start interacting with your text generation web ui. Now here's the IP address. I actually added this extra text to show this in dark theme. Now if you haven't used the text generation web ui. So I'm going to simply walk you through different features in this introductory video. We are going to be making extensive use of this web UI in upcoming videos, right? So first you have a tab of text generation where you simply type in your text and whatever model you choose, it will start generation text. So it's very simple. There are different modes. There's instruct mode, a chat mode and there's a KAI chat mode as well. I'm not sure like what this one does, but I usually use the chat mode, right? And then like there are the normal options, you can stop the generation, you can click continue. So similar to ChatGPT, if it doesn't complete the response, right, you can regenerate the response and then you can even impersonate somebody. Right. Next option is or the next tab is a character. So here you can personalize your character or chapter. Chat Assistant. We're going to be looking at this in more details in upcoming videos. Right. The next tab is parameters. So you can set different parameters of your text generation and it's very comprehensive. For example, you can set the temperature so that will control the creativity of the model. Then repetition penalty, how many electropic responses you want, right? So on and so forth. Next you have the model. I think this is the most interesting part. So if you have multiple models downloaded, they're going to be listed here and you can choose the model that you want. So for example, in this case I have a Facebook model and then this, the Visit Vocunia model that we just downloaded. Now let me show you how you can actually add a new model. So rather than going to the terminal or the command line interface, you can actually copy the same path, the username and the model path like the ME that we did for our Visit Vulconium model and just click download and it will start downloading the model. I'll show you how we can actually download another model using the web ui. So I actually saw there is a new version of visit element which is supposedly uncensored. So I'm going to go here and copy this. This is a wizard element trained with a subset of the data set. Responses that contain alignment or moralizing were removed. The intent is to train a visit element that doesn't have alignment built in so that the alignment of an can be added separately. Right. So I'm going to just download this to show you as an example, but we are going to be playing around with the original Visit Vulcania and then for the time being I might actually make another video on this one. Okay, so simply type in the path and click download and then it starts downloading your model. And you can even actually train a model or fine tune models here so you can train your own Lora. And this is a very powerful approach to actually do it. So I might actually show you in an upcoming video how you can fine tune your own models using this web ui. Now the next tab is Interface mode. The text generation web UI takes a lot of inspiration from automatic 1111. And that's why you will, you probably will see a lot of similarities. So you can control how the interface looks like here. Okay, so now let's play with the model itself. Right, so the first prompt is draft an apology email to a customer who experienced a delay in their order and provide reassurance that the issue has been resolved. Okay, so you simply type in your font, I'm going to select chat and then click Generate and let's see what the model comes up with. Now I have an older GPU. I think I have a 1018 Ti. So that's why it's going to be pretty slow. Depending on your hardware you might get to a different speed. Okay, so here's the response. It says we are ready this email to apologize for the delay in the processing your recent order. We understand how frustrating it can be when expected delivery times are not met and we sincerely apologize for convenience calls. So it's a really normal email that you would expect from a customer support team and I think it did a decent job with it. Okay, so next we're going to try this prompt. I saw somebody else actually use this to generate a graphical calculator. Right? So use K and write Python code to create a basic calculator application with addition, subtraction, multiplication and division functions. So let's see if it actually can. This can do it. And if it works for other LLMs I've seen they have trouble with this. Okay, so it came up with the code, so let's run the code. Okay, so it actually came up with a pretty simple basic interface. Let's see if it actually works. So I'm going to say 10, let's say three. Right. Add. Oh nice. You get a result if you subtracted seven. So modification and division also works. This is pretty neat. I wasn't expecting that to work. Okay, so the next prompt is create A list of three startup ideas in the enterprise B2B SaaS. The startup ideas should have a strong and compelling mission and also use AI in some way over cryptocurrency blockchain. The startup ideas should have a cool and interesting name. Right, so it actually started creating the response and I am running it in the chatbot because I want to ask it further questions. So the first one is mind mil AI. This platform would use AI to help business is better understand their employees needs and preferences in order to create a more engaging and productive workforce. The mission of mind Meld AI is to make an office. Make the office a happier, healthier place for everyone involved. Interesting that's the first time I have seen smart sales AI. The SaaS platform would use AI to help SaaS teams better understand their customers need and preferences in order to close these more effectively. Okay, and Brain Bridge AI this platform would use AI to help companies bridge language barriers to provide real time translation services for meetings and other communication channels. This is interesting. This is nice. When you in the chat mode you can actually create different characters, assign them for strategy, and even assign them different images that you want to use in your characters. So I'm going to show you how to create characters in my upcoming video, so stay tuned for that. The goal of this video was to actually show you how to install and run this text generation webUI, but we looked at the wizard virtual model as well. I'm going to create a detailed video on this comparing with Vocunia and Wizard Vocunia, right? But this text generation Web UI is an excellent tool for you to run your model locally, so I would recommend to play around with it if you have the hardware to run it. As always, if you have any questions or comments, put them in the comment section below. If there are any specific topics you guys want me to cover, so also put them in the comment section below. I hope you found this video useful. Thanks for watching. See you in the next one.