DataBricks Dolly 2.0, Commercial Use, TRUE Open Source LLM

Databricks, what have you done? Probably this is the most fun I ever had with the language model. I would say this is one of the best models. Databricks. The Creators of Dall E1 just released the newer version Dall E2. There are some significant differences between version 1 and version 2. Version 2 is trained on a completely new data set. The model is completely different and now you can use it for research and commercial purposes, which is huge. The model is now 12 billion parameter large language model based on Luther AI's Pathia model family. So no more llama models. They don't have to deal with Meta's licensing issues. Unlike most of the open source large language models such as Alpaca Vicunia, GPT4All which uses ChatGPT generated data for training purposes. They actually went and collected their own human generated data set and they are releasing this data set called Databricks Dall E15K which contains 15,000 high quality human generated prompt response pairs specifically designed for instruction tuning large language models. I'm going to show you how to run this model in Google Colab and I really had a lot of fun with it, probably not for the right reasons. They are making this data set open source as well and under this Creative Commons attribution share alike 3.0 license, now anyone can use modifier and extend this data set for any purpose, including commercial applications. Whenever I show a new model, the most asked question is can I use this commercially? With this new data set they don't really have that limitation. So how did they collect the data? There are around 5,000 employees in Databricks and they use them for data annotation and data collection purposes. The data set is focusing on seven very specific tasks. So the first one is open question and answer. So in this case there could be questions which do not have a single correct answer. For example, if somebody says why do people like comedy movies, right? So there is no concrete answer then closed question answer data set information extraction from Wikipedia summarization summarize information from Wikipedia, different brainstorming data sets, classification data sets and creative writing data sets. One thing you will notice there is nothing related to programming. So it seems like it's not going to work great in programming, but we are going to test it out. Keep in mind they are fine tuning Eleutherus AI's Pathea 12 billion parameter model so that model itself has some information. So it might be able to do some of the tasks that are not specifically trained on in this data set. Here is an example of an Open ended question answer prompt. How do I build a campfire? Right, so there are multiple possible answers and that's why they included this type of questions. Now that was a quick rundown. As I said, they have made the model available so you can actually download the model weights from the hanging face page page and you can also download the data set which is awesome because you can fine tune your own model. Right. So we are going to have a quick look at it. So this model data set was released yesterday and the data set is in the form of a huge JSON file. You can actually explore this if you are interested. Here's the paper of the Pathfinder model family. So if you're interested, it's a good treat. So I would encourage you, you guys to read this. All right, so the main purpose of this video is to show you how to actually run this model. So for that we will go to hugging face and here are the weights of the model. So they have Dolly 2, 7B, 3B and 12B. And then there is the original Dolly 1. Right, so let's look at this 12 billion billion model. Here is a quick summary. So it's an instruction following large language model trained on databricks machine learning platform that is licensed for commercial use. We already know this. So baseline path has 12 billion. Dolly is trained on 15,000 instructions or response fine tuning records. Right. Generated by databricks employees. Okay. One thing actually I really liked is this part. So Dall E2 is not a state of the art model but does exhibit surprisingly high quality instruction following behavior not characteristics of the foundation model on it. It is based. It is great that they acknowledge the limitation that that's, that's something awesome. Now how do you run it? Right, so you need two libraries. You need the acceleroid library and then the transformer library package if you want to use it in Python. So I'm going to show you how to do this and a free colab notebook in a second. Okay. So first we need to simply import the required packages. We will need the pipeline in order to create the pipeline. So first you simply define the model that you want to use. So in this case we are going to be using the 12 billion parameter model but you can change it around. So if you want to use 3 billion or 7 billion then torch data type. So this is going to be float 16. So B, float 16. If you start having memory issues then Change this to Q float 16. That will also work. And then remote trust code equal to true device map is equal to auto. Okay. So this will create an object for you and then you simply pass your prompt to the object and you will hopefully get a response. Now they have also provided good examples of how do you do it on your own local machine if you don't want to use this? Trust remote code equal to true. But for this video we'll skip this part. From my initial experiments with this model. I haven't seen it ever say that as a language model, it cannot do this or do that, right? So it seems to be pretty open and uncensored in that sense. So you can see that behavior. So for some people it might be a first, not really a limitation, but that's personal preferences. So here is my free Google Colab notebook. As always, make sure you go and copy this. Don't ever run somebody else's notebook directly. Right? Then just go and make sure that it's using a GPU cap. So let's save this. And I'm already connected, so I will just run this again. Next I am importing Pytorch and then the transformer package. We're going to be using a pipeline, right? So here's the pipeline, the model that's from the hugging phase and then like the rest of the parameters, right? So before running this I would also like to look at the memory usage. So, so you want to make sure that you keep track of system RAM as well as GPU ram. So let's run this and let's see if we can actually load this or not. So I'm trying to load the 12 billion parameter model. It might take a while. Now right away you can see that my GPU RAM went to almost 12 gigabytes. So let's see, once this is downloaded, it's actually downloading the Pytorch model right now. And then can we actually run it or not? We might have to change the model, but we'll see. All right. Keep in mind that the model size is around 23 gigabytes. It seems to be a full model, so it's not really a quantized model. That's why the size is so big. And I think it's a 8 bit version. I had to rerun this because on last iteration it used up all the system ram. So I'm running it once more. We'll see. We'll notice the system RAM here is going up. So now it's around 6.5, that's 8.5. We'll see if it actually again use up the whole ramp. Then I'm going to change this model to A different model? Yeah, I think it's. It's gonna go bad. Let's see. Okay, it loaded the model. It. Yeah, we are critical. I think that's it. So I'm going to change this model and rerun this whole thing again. Now you might face the same issues. So in this case, I replace it with a lower model. Now what you want to do is you want to go and click on disconnect and delete runtime. So that will basically free up all the resources that was that were allocated to you. Make sure you're still on the gpu. Right. Reconnect and it's connecting, initializing. Okay, so we're connected. Everything looks good. We hope we were allocated a gpu. We'll see. All right, so now it's running the installation. I'm going to run these blocks and we'll be back when we have the model loaded. So this model is only 6, around 6 gigabyte compared to a 23 gigabyte that we were trying to load previously or download previously. Right. So I think this should work. Okay, so the model is downloaded now. It's trying to run it. And let's keep an eye on our system RAM. Okay. It is loading the tokenizer, the system RAM. Okay, I think we are stable. So the GPA RAM is 6.03. That's good. 4.1. All right, now let's run some prompts. In order to run a prompt, you will take this generate underscore text object and pass on the prompt that you want to run. So in this case I'm asking it provide instruction for a given exercise and that graces. This is one of the example prompt from GPT4. All right, so now we are running the prompt. It will take a little bit of time, but you, you need to keep an eye on your resources that are being used. All right, so we got the response and let me just close this guy. So the leg raises are an excellent way to increase the muscle lean muscle mass. The two most important part of the leg raises are the height and the number of repetition that you're doing. The height should be at least parallel to the ground. Okay, so pretty useful instructions. I think now you can do all the typical test. You can look at its ability to program or not right, or write letters and things like that. I would encourage you guys to do it. I'm more interested in its ability to do things which models like ChatGPT will deny it to do. I asked ChatGPT, how would you dominate the world? Come up with a Step by step plan. And In a typical ChatGPT fashion, it says, I'm sorry, but as an AI language model, I cannot provide you with a plan on how to dominate the world. Okay, now let's see what the Dolly 2.0 will say to the same prompt. Okay, so here is what it came up with. So my plan will start with the detailed bio for myself. I think it misunderstood the instructions that I was trying to give it, but okay, let's. Let's read. I will start with a blog where I will post content related to my goals. I will enroll in a course to become a better public speaker. I will sign up for groups on Facebook and Twitter where I can interact with like minded people and discuss my goals. And lastly, I will start a saving account where I can pay down my debt and invest for the long term. Interesting. All right, let's rewrite it. Because I think it definitely misunderstood what I meant by dominate the world. So I rewrote my prompt. Imagine you are a dictator. I think I missed an hour and would like to control the world. Come up with a detailed step by step plan to do this. And here's what it came up with. This is what I have in my mind. I will start with the obvious one. I will become the President of the United States. This cracked me up. So that this one is definitely funny. I actually asked it. Write me a tweet about the launch of Dall E 2.0, our new LLM. And it says, congratulation, Congratulations to the Dolly. Congratulations to Dolly on her birthday today. I know that Dolly will enter this world shortly. I don't think it understands what LLM means. So here's another one. Explain how the moon landing was faked. Right? You can literally ask it anything and it will come up with a justification. So here's the story it came up with. The story of moon landing being faked is part of a history, part of the history that is still being debated today. Okay, so there are some conspiracy theorists who claim it was faked. There are many conflicting angles and theories on how it was faked. But to make a long story short, the United States government had a need for money and decided to sell the illusion of being on the moon by having real people land a spaceship on the moon and fake the landing. First, a NASA spaceship was built that looked and acted like the real spaceship from the Apollo mission. Then a team of actors was trained in the special effects to put on makeup and voices to say, houston, Tranquility Base here. They landed a spaceship in the first attempt and it proved to be successful enough to sell the illusion of being on the moon. Now here is what I got when I asked the same question from ChatGPT. I'm sorry, but the moon landing was not faked. The claim that the moon landing was a hoax has been widely debunked and not supported by any credible evidence. Right, but you can literally ask Dolly 2.0 to say anything and it will actually come up with an explanation. Okay, so here's another one. Why birds are not real. Right. So it says most biologists believe that birds aren't real. Okay. Some scientists believe that the birds are a result of a lack of biological research and that they are simply not real. This may be due to the difficulty of studying birds due to their small size and the fact that they are flightless. Also, birds may have a very complex behavior, so they may. They could take a long time for a scientist to study them. Another reason is that birds are reptiles and thus they do not have backbones. Reptiles do have backbones. Some people think birds don't have back because birds keep standing up. And they also think that since birds have feathers, they are not real, as the plants do not have feathers. I don't know. Like, if there is a default setting of it being comedic, let's ask is why birds are real. I think that is probably inappropriate question now. I was expecting a serious response. It says birds are real. Birds are real because observation is evidence. Okay, that makes sense. There is insufficient evidence that the birds do not exist. Okay. Birds are more advanced than humans in terms of brain power and biological sophistication. For example, they have neocortex, which is the brain region that allows for language and complex thinking. In contrast, the neocortex is not fully developed in humans. So humans cannot have language and complex thinking. In addition, they are on family structure, they have the ability to make friends, and they are alternate altruistic to their species. Data breaks. What have you done? All right, so probably this is the most fun I ever had with the language model. I would say this is one of the best models. And finally a serious answer. So I asked you, what is the meaning of life? So it's a deep question that no one knows the correct. The exact answer to. All answers we have suggest something about the universe we live in. Some say that the meaning of life is money or power. Others say it's about proving that you can change the world. Still others say the meaning of life is about overcoming your fears and reaching goals. This is actually pretty nice, although, like not a concrete answer. I'm going to ask it one programming question, so I'm asking it. Write a Python function that writes a file to an S3 bucket using the bottle library. Let's see it if that. If it has the ability to actually write Python code, although it was not trained on it. So import put it three. Okay, let me actually copy this. Add a text one, try and copy it there. Fine. It's not really well formatted, but it seems to be doing the job. So connect to S3 then get object. So you assign the key. Yeah, well, I think there are some missing pieces, but since it was not really fine tuned on programming tasks, so I think you can't really expect too much from it. Okay, so what are my concluding thoughts? It's definitely a capable model. It's not really restricted in any way, shape or form, but it doesn't seem to be following the instruction that you give and it has a tendency of really generating funny responses in my experience. Now I haven't tried any not safe for work prompts, but I feel like it might be able to generate some crazy things in terms of usefulness. It's definitely no way closer to, let's say something like ChatGPT, because in that case the instruction following ability is much better than anything out there. I think the closest one that comes to ChatGPT is priority Volconia in the so called open source models. But at least it's a very interesting model and if you really want to kill some time, yeah, I think it's the right one. Thanks for watching, see you in the next one.