Automate Dataset Creation for Llama-2 with GPT-4!

In a previous video I showed you how to fine tune a Llama 2 model on your custom data from hugging face. In a subsequent video I showed you how you can create your own data set using code interpreter from ChatGPT. But in this video we're going to bring everything together and I'll show you a single Google Colab notebook that you can use for automating the data creation as well as the model training process. So let's get started. Okay, the project that we are going to be looking today is called GPT LLM Trainer and this was brought to my attention by a great community member on our Discord server. Check out the Discord server link is going to be in the description. Training or fine tuning a large language model is a very difficult process. It involves data collection, cleaning the data, getting the data in the right format, then selecting the model, then writing code, train it and then do the actual training. Now this project tries to create a new pipeline to train high performing task specific models. So you simply need to provide an input description of your task. The system will generate data set from scratch, parse it in the right format and then fine tune a llama 2 model for you. Now in this case you will need access to GPT4 API in order to create the data set. And the data set is going to be in the form of prompt and responses from GPT4. Now the setup is relatively simple if it works. I ran into some issues, so I'm going to talk about those. So basically it's a simple Google Colab notebook. You want to utilize the gpu. You probably need the paid account for this to work and you also need the OpenAI API key. Now how to use it? You simply need to provide a single prompt that is the description of what you want the train AI to do. And the author suggests that you provide a descriptive and clear instructions for it so that it can generate a better data set and train a better model. There are two other parameters you probably want to set. One is the temperature. So that basically controls the creativity of GPT4 or ChatGPT, whatever you used for data creation. And then the number of examples I actually ran into an issue here. So I'm going to talk about that in a little bit. And after that you simply want to run all the cells available in the notebook and there is even a section for running inference on the trained model. So here is the original notebook. You definitely want to run this on high vram gpu. So they're suggesting v100 gpu, but any k in any case, you want to make a copy of this on your drive, and after that you simply need to change this based on the problem that you are trying to solve. For example, I change the default prompt to a model that takes in an expression in English and responds with a rap song in English. Now, you need to provide your own OpenAI API key in order to use either GPT4 or GPT 3.5 Turbo. Now, in order to generate examples, the author is using a system message for GPT4. So this is the system message that goes into GPT4 or GPT 3.5 based on your selection, right? And so this function basically takes care of generating examples in the data set. Now here you can change the default model. So by default it's GPT4, but you can replace this with GPT 3.5. Now, one more thing which you probably want to change is this Max tokens. So with the default settings of around 1354, I was running into rate limit issues from OpenAI, so I had to replace this. I was also not able to use GPT4 for some reason, so I changed it to GPT 3.5 and I think I replaced this by 500 tokens. So in my case I'm using GPT 3.5 Turbo instead of GPT4. In the quality of output, I haven't really seen any difference in terms of the data set that it generates. Now, you will notice that I am currently running this at this moment. I have seen that the code timeout a couple of times on me or there are some API response issues from OpenAI. So if you face these issues, I guess these are normal. So it completed the data set creation process, or at least the example generation process. And here is how it's going to look like. So I'm going to provide this prompt and then here's the output from the model. So we're going to be using this prompt response pair to fine tune our own model. This block of code will add a system message to the model that you're fine tuning. So, for example, here is an example system message that is going to be added to the data set. So it says, given an expression in English, you will receive a rap song in English. Now this is necessary because the author of this notebook was trying to fine tune a chat version of the llama 27B. So next, everything is put together in a nice Pandas data frame, right? And here is how the data set is going to look like. So you're going to have prompt the corresponding response, right? And for some reason, it never generates the number of examples that you ask it for, at least in my case. So initially I asked for around 100 examples, it only generated around 40. In this case, I asked for 75 examples and I got a total of 51 examples. I think this has something to do with the way you are querying OpenAI API. And after that we are dividing our data set into train and test set. So 90% of the data is kept for training and 10% of the data is going to be kept for validation or testing. Now this concludes the data creation process. So if you are running this on Google Colab, you will see that it will generate these two JSON files. I would recommend you to download these locally so you don't have to repeat this process again. And in order to do that, you simply need to click on these three dots next to the file and click on download and that will download the files for you. Next, we are defining all the hyper parameters for training the model. So here's the model name that we want to use. It's the Llama 27B chat version from Nose Research. Then define the data set name. So that's the data set that we're going to be using for training. And what is going to be the new model name? The rest of the parameters are parameters used for training or fine tuning. However, I ran into this issue that when I was trying to run this load data set and train section, it was not able to find the model that I was trying to access. So I went and checked on the repo and for some reason the Nose Research group has deleted that specific model. So I did some research and found another chat version of the model on Hugging Face. Now you need the chat version for this specific notebook to work because the author of the notebook is using the default prompt template for Lama 7B chat models. If you look here, this is the prompt template that was used by Meta to train the Llama 2 chat version models. And since we are fine tuning a chat model, that's why we will need to provide a data set in this specific format. However, if you are fine tuning a base version of the Llama 2 model, you can define your own prompt template.

Okay, so the free version of Google Collab was not working and I had to switch to V100 GPU. But it will it wasn't working either, so I switched to a 100 GPU. So let's see if this is going to work for our purposes. Now keep in mind that using a 100 GPU will cost around $13 per hour on Google Colab. Okay, so the training is complete. Now let's test it on a simple prompt. So the prompt is a boy running in the rain and it was able to generate a rap song. It's not bad at all given that we trained the model on only 50 examples, so it's pretty good. After training you can save the model to your Google Drive. So this code segment will do that. So in this case we are simply storing the tokenizer the corresponding configurations into this custom path that we have defined. So in this case we storing this as a llama 2.7B custom file. Then you can load the store model from Google Drive to do inference on a new example. So that's how you can use this GPT LLM trainer both for creating your data sets using a single prompt as well as fine tuning your model based on that data set that you created. Now personally I would like to use the first part of the code just to create the data set for fine tuning. I'll still prefer the Auto Train Advanced package from Hugging Face because it's a single line of code that lets you train powerful models. Just keep in mind that you will need a very powerful GPU in order to train this. So in this case we were using a 100 GPU and it's mainly because of the system RAM that is needed. Another thing to consider is in this case I did not use the official Llama 2 model, but if you have the access token you can use that as well. I just did not want to do that in this specific video. I hope this video was helpful. If you found it useful, consider liking the video, it helps with the algorithm and subscribe to the channel. Also turn on the push notifications so you don't miss another video from the channel. As always, thanks for watching and see you in the next one.