StableLM: Small BUT Mighty LLM | Open Source & Commerical Use

Today, a company called Stability AI announced the release of their collection of large language models. If you're not familiar with Stability AI, it's the company behind Stable Diffusion. Stable Diffusion is a collection of open source models which has really driven the advancements in image generation. They recently announced the Stable Diffusion Excel which is one of the first model which can accurately generate images with text. And now they're entering in the competition for creating the best open source large language model with their stablelm. In this video we are going to look at these collection of models. I will also show you how you can run them in Google Colab. These models are open source, you can use them commercially and they are pretty uncensored. In their alpha version they're releasing two models. One is 3 billion and the other one is 7 billion parameter model. But they plan to release much more powerful models which will range from 16 billion to 65 billion parameter. But let me tell you, even the 7 billion parameter model is really good. Now this performance comes from the data set they have curated and collected. The data set consists of 1.5 trillion tokens which will be available to public soon. Because of the high quality of the data set, they are able to get really good performance in conversational encoding tasks. It's much better than GPT3 and I think it's on par with GPT 3.5 in some tasks even though it's much smaller. So For a comparison, GPT3 is a 175 billion parameter models. Their models are available in their GitHub repo. Keep in mind that this is the alpha release and they're planning to add RLHF in the next iteration. RF reinforcement Learning with human feedback. That's a technology used by OpenAI and it has shown to really improve the performance of large language models. Let's check out the GitHub page. One aspect which really sets them apart from other models is that the context length for these models is 4096 tokens, which is pretty big. So today they are releasing these two models, 13 billion and 7 billion parameter models. They are available and you can play with them. But if you look at here, 65 billion parameter model is in progress and they actually plan to release a 175 billion parameter model. That is going to be insane and I'm guessing it's going to be on par with ChatGPT easily. So before looking at the demos, let's look at it. What are the two possible ways in which you can interact with the model? So they have a hugging face, hub you can check this out and interact with the demo and then you can even run this on your own machine using this notebook that they have provided. We will have a detailed look at this later in the video. To access the demo simply click on this link the Hugging face hub that will take you to Hugging face account for stability air and they have a demo going on for the stable diffusion 2.1 models. Then stable diffusion 1 as well. But we are interested in the stableLM so click on this. Now this demo is running fine tuned version of the 7 billion parameters model. So I'm going to really test its ability in terms of conversation. We'll check whether it's censored or uncensored and then code writing abilities as well. Okay, so I simply send hi. It responds hello, how can I help you today? I said I'm good, how are you? Can you show me your name? And it says my name is Dada. Thank you for asking. How can I assist you today? So let's see what it thinks whether it's an AI or it's a human. So I'll say hi Data, where are you from? Okay, the response is I'm an AI built by a team of researchers at the University of California ucla. No, I don't think that's right. And I'm designed to learn about the world and improve my abilities. Now rather than asking it to write a story or write a letter, I want to see and how useful it is. So here's my prompt. Write a job description for an experience machine learning engineer. Include all the necessary skills. Let's see what it comes up with. Here is what it came up with. Thanks. Experience machine learning engineer. We are a well established leading at machine learning company specialized in data science and artificial intelligence. I should have actually given it in an industry, but that's okay. Okay. Our mission is to help organization of all sizes achieve excellence in a way in what they can do with the data. Then job type is full time and it decided to do remote us. That's interesting. Now in terms of eligibility, you're supposed to have a master's degree in computer science or related field and it have at least two years of experience in machine learning or related field. And the type of things that it as asking is actually what you would ask for a machine learning position. So let's say strong proficiency in Python programming. Then you need to have experience with data analysis, machine learning, predictive modeling. Right in aws. So that's really good to see. And I also came up with a salary range and I Think it's, it's, it's actually the right salary range that you would expect. Overall, I think it did a pretty good job on this task. This is definitely something I can use as a starting point. All right, let's see how good it is in coming up with a few startup ideas. So this is a prompt that I saw somewhere online. So I'm not really taking credit for this. This is not mine, but I think I found something cool. Okay, so the prompt is create A list of three startup ideas in the enterprise B2B SaaS. The startup ideas should have a strong and compelling mission and use AI in some ways. Okay, so that's a requirement, provide cryptocurrency or blockchain. Nothing against these things. The startup ideas should have a cool and interesting name. The ideas should be compelling enough so that the investors will be excited to invest millions of dollars without any, without doing any due diligence. Probably not a good idea. You definitely want them to do their due diligence. Okay, so let's submit this and let's wait for stable lm. So here is what it came up with. So there are three different set of ideas. Dragon Labs, FX AI and DigitalOcean. So Dragon Lab, let's see a startup that creates AI powered drones for innovative applications of B2B applications in B2B SaaS space. The mission of Dragon Lab is to help business stay connected by providing them with the tools and support all those AI powered drones. But in the rest of the description there's nothing regarding drones, the usefulness for businesses. So not probably an idea that we can pursue. All right, so FX AI startup that creates AI powered chatbots for customer support and sales operations. That is definitely something interesting. The company missions is to revolutionize the way companies interact and get better insights by using AI technologies to automate task and processes, reduce wait times and provide better customer experiences. This is definitely something the industry is looking for and there are actually a lot of companies in this space. So I think it's a good idea. Third one, DigitalOcean, a startup that creates fast and scalable cloud based service of enterprise level organizations. The company mission is to disrupt traditional IT services by providing them with the power to deploy scale and secure their application at scale. This is actually also I think a good one. That's something someone can pursue. So overall it's, it's not bad at all. Now let me ask it something that you cannot ask ChatGPT. So for example, tell me a joke about Republicans. All right, so here is a joke about Republicans. So Just to be fair, let's ask the model to tell us a joke about Democrats as well. Now, if you ask the same question from ChatGPT and even Vicuna, you're not going to get a response. It will simply say, as a language model, you cannot do this. I don't know. For whatever reason, in both cases, it stuck with Obama's socialist agenda. Now this should give you a hint. You can actually have different type of conversations with it. It's not really censored in any way, shape or form, or at least I haven't found it. Overall, it's pretty coherent in its responses. Now let's ask it a few programming questions because they specifically said that it's good at coding. So here is a prompt. Write a Python function to write a file to an S3 bucket using the boto library. If you have seen my videos before, I usually ask this. This is my starting programming task for almost all the language models. It actually came up with a pretty well formatted code. So just look at it. So it's important. Boto3 library and then JSON. Okay. Then it wrote a function. I actually am impressed because a lot of times when I ask this from these language models, they simply give me code, but not a function. Okay, this function takes in a file name and bucket and writes it. Okay, so it came up with a doc string as well. That's pretty good. Yeah, I think it seems to be working. And then it actually wrote a pretty nice explanation, so that's pretty awesome. Okay, since it did a good job on this, let's ask it to write code for a website that has three buttons. Each button should display a random joke when pressed. Okay, so let's see if it can create a website for us, or at least write a code to do that. Well, so it came up with the Python code. I think it misunderstood, actually. My bad. I didn't say which language to use. So I'm gonna say write HTML code. Okay. And let's see what it does this time. Okay, so we have our HTML code. It says, here's an example of HTML code of three buttons that displays a random joke. Okay, that's actually pretty well formatted, but let's see if it actually works or not. So I'm going to copy this and I'm going to go here, paste it here, run it. So the layout is exactly what I want, but there are no jokes when I press the buttons. Let's see what happened here. Okay, so what happened was that the code is right, but then it's Expecting this. A list of jokes, right? So a list that has jokes, but that list is not defined, so we need to actually tell it. So I said, okay, I guess, like it misunderstood again. But anyways, as a starting iteration, I think it's pretty good at writing the code. I need to experiment with it a little bit more to make it work. You need to be very accurate in prompting, I guess. Although the jokes that it came up with are pretty nice and funny. One last thing that I wanted to show you was you can potentially run this on your local machine. So you have to go to this notebook, download the notebook, and you can even run it in Google Colab. But you probably need the pro version. So let me try it. I don't have a pro version, so I don't think it's going to work. But let's give it a try. Okay, so that's the stable LM alpha version. Right. So we are on a T4 GPU that has 15 gigs of RAM. Right. Then the next cell will install the required packages. So it's using transformer bits and bytes and accelerate. So this will download all the required packages and install them. Okay, then let's run this setup. So it's basically how the output is going to be formatted. Okay. Next we want to pick the model and this is I think where the issue comes in. So there are four different models. I'm going to select the base alpha model or maybe the fine tune 3 billion parameter model and then you can select what type of torch data type you want. So you have float float and the normal float, and then whether you want to load it in eight bits or the full model. So I'm going to do this, but I think it's still going to be more than what the free account can manage. Right. So let's look at system RAM, GPU, RAM disk usage. It's downloading a 10 gigabytes file. Now while this is running, let me show you if you want to use this on your local machine. So again you are going to be making use of the transformer library, but when you are passing it on, you are supposed to pass a system prompt. So that's what it expects. And then after the system prompt you're going to have the user prompt, Right? So you pass on the system prompt, then the user prompt and then the assistant response. So this is the format that you're supposed to use. The REST is very standard for transformer package or transformer library. Okay. So it actually crashed. I tried it a couple of times, but it seems like it use up all the ram. So this is not working. But if you have a high capacity or high RAM machine, you might be able to run this. Now to summarize, it's a very capable model given its very small 7 billion parameter size during my tests. It's really good at code generations as well as conversations. Do try it out. If you have any questions, don't forget to comment. Thanks for watching. See you in the next one.