How to Create Custom Datasets To Train Llama-2

How to create a data set for fine tuning a Llama 2 model. That's what we are going to be talking about in this video. In a previous video I showed you this single line of code with the help of which you can fine tune llama 2 model. But we used a pre existing data set. In this video I want to show you how to create your own data set and then how to format it to fine tune a llama to 7 billion parameter model. The first thing that we need to think about is what this data set is going to do. In a previous video we explored how to use CHAT GPT as a prompt generator for mid journey. In that video I was giving this huge prompt template to chatgpt and it was supposed to generate prompts based on this template. But in this video we are going to create a data set where we are going to provide a concept and then the model is supposed to generate a prompt based on the concept and we are going to create this data set with the help of code interpreter. So the goal is going to be that our model should take a concept as an input and then generate a prompt that we can use in mid journey to create images. So by the end of the video we are going to get a custom model that is specialized in this specific task. Okay, so before creating the data set and training our model, let's see how a llama2 70 billion parameter model is going to work when we ask it to generate a mid journey prompt for a person walking in the rain. So I'm assuming it's not going to have an idea what mid journey is and probably is going to give us a very generic response. Okay, so the 70 billion model was actually taking too long, so I switched to 13B and here's the response that it generated. Hello, it looks like you are out for a walk in the rainy day. Rain, rain today. How can I assist you? So it's definitely not fine tuned for this specific task and that's why the response is not really great. So now our goal is to train or fine tune the llama model such that we give it this prompt and it generates a much better prompt for midjourney to use. Now for this specific video I asked GPT4 with code interpreter to create a data set that contains a concept prompt pair and for each of the concept. So I give it an example like a person walking in the rain create a detailed description that can be used by an image generator to create that image. Now you probably will notice that it defaults to GPT 3.5. However, it's a previous conversation in which I was using GPT4 with code interpreter and I think I ran into the limit, so that's why it defaulted to 3.5. So now in this case it said, certainly creating a data set that pays a concept of the detailed description to be used by the image generator is an interesting task. Right? So it gave me some examples, for example, a concept, a person walking in the rain, and then it came up with this description, right? A child flying a kite on the beach, right? So then here's the description and they look pretty good. I actually tested some of them in mid journey to make sure that the output images are pretty nice. Right? So I was happy with this. That's why I asked you to create a data frame with a concept and description, include 25 unique examples of concept and description pairs. Right? Based on what it has created, I simply asked it to create 25 unique examples, put them in a data frame, then I asked it to write that into a CSV file, right? And then I asked it to repeat that process four different times. So that's why we had multiple CSV files. You could do it in a single CSV file, but I didn't want it to hit the limits. That's why I did it in multiple files. Okay, so here's an example of how the output looks like. So you have a column for the concept and then a second column for the description, right? And it's pretty simple, straightforward stuff, nothing really complex, but it is a very good example data set. Okay, Then I needed to combine all of these files together and put them into a very specific format. So again, I use the code interpreter. I asked it to combine all the CSVs into a single data frame and it actually did a pretty good job. So in total we had 3,000 rows with two columns. And again, here is the example of how the data looks like. Now keep in mind this is a very simple data set, but the goal of this video is just to show you how to structure your data set for fine tuning the model, not this specific data set. Now this is the most important bit in this whole video. So you want to structure the data and bring in both your input as well as your output into a single column called text. Now I'm doing this within the code interpreter, but you can do this in Python as well. I said create another column called text, which follows the following structure. So in this case we are defining a special token that is going to be the input from the human. So you have human and Then next line and we are adding this specific part, generate a mid journey prompt, then we will provide a concept. So basically it will take the first row, put it here, right? And then special token for the assistant and that is supposed to generate this response. So the goal is going to be you provide a concept and then based on the concept, the model is going to generate a description or the prompt. Now after doing that, this is how our data set looks like. So instead of two columns, now it has three columns. The first one is concept description and then text, which puts together both the concept and description in a very specific format. Now a couple of things to keep in mind. We are going to be fine tuning a llama2.7 billion parameter model. However, if you notice, I'm not using the prompt template that is provided with llama 2 and that is because I'm actually fine tuning the model. Right? So in this case we are not using the chart model, but we are using the base llama 2 model. And if you want to fine tune a model, you can set the prompt template the way you want. So it doesn't have to be like this. It could be whatever you want. You need to be consistent across all your examples as well as you want to define very specific tokens both for the human input as well as the assistant response. Or you can have an instruction input and then assistant response. So fine tuning basically gives you the flexibility of setting it up the way you want. All right, so once all this was done, I simply asked it to export this to another CSV file. Okay, so this is how the final data set looks like. So the last column is the one that we just added and which we are putting it into this format. So human, then the input, then assistant. And here's the response. Right? So that is basically the description column. Another change that I made was to rename the file to train CSV. Okay, so once the data set is ready, the next step is to fine tune the model. So in order to fine tune the model, we are going to be using the auto train package. From hugging face, I have covered this in another video and going to put a link to that video. So if you haven't watched that video, I would highly recommend you to watch that and you will understand everything that is going on in here. This is a single line of code, but I put it on multiple lines to make it easier to understand. Now you will need to install the auto train package. So watch the other video on how to do that. And once you install that, this is the only line that you need to run first we run auto train. We want to fine tune an LLM. That's why we pass the LLM flag and then we are passing on the train flag. That indicates that we want to fine tune this specific LLM, the project name. So I'm going to be using mid journey prompt. That's going to be the project name for my for this specific project, then the model. So I'm using a sharded version of the llama 27B. That's just because the GPU I was running on is not a high VRAM gpu. Right. So I simply wanted to divide the model into multiple parts and then load one part at a time. Right now this is the most important bit. So you need to define the path of your data set. So what I did was I put the CSV file that I created in the same folder on which I'm running this code on. Right? And you need to make sure that the project the the file name is actually called train CSV otherwise it's not going to work. So simply put the train CSV file in the same folder where you are running this code. You don't need to write the file name here, you just need to put this dot IN there. After that we are using the PIF package for fine tuning and this is going to use the 4 bit quantized version. You set the learning rate. Then the batch size. I'm using a batch size of 4. Number of FX that I have set is 9. Right. You can play around with this. And then we want to use SFT trainer. Right. And everything is going to be logged in training log file. Okay. So make sure that you watch the previous video if you're not familiar with auto train. Now depending on how big your data set is, the training is going to take a little while. Now this is how the file structure is supposed to look like. So don't worry about this app PY file. It has the code in order to make inference on our fine tune model. So I'm going to cover this in a little bit. But initially you need to have a train CSV file in a folder and then you can simply run the terminal within that folder. Right. And once you run that, if everything goes well, you are going to see this mid journey prompts folder which basically has fine tuned model in here. And then along with that you're going to see this training log file. So this basically has all the logs from the training, right? So if you look at it here, actually ran it for nine epochs and the Training loss is decreasing pretty good, right? The learning rate is decreasing, right? Ideally you want to have a validation data set as well. This was a relatively small data set, right? But you want to train your models on a much larger data set, right? But this is just an example of how the output is going to look like. And then once the training completes, as I said, it will create this folder and the name of the folder is going to be exactly the same as your project name. Now within this you're going to see these model files and different checkpoints that it stored throughout the training, right? So this is what you should expect as an output after the training completes. Also keep in mind I was running this on a local machine, so I'm not doing this in Google Colab. And one more thing, if you notice here, if you have seen my previous video, so I'm I was pushing the fine tune model to hugging face. In this case I'm not doing that. But if you want you can do that. But for that you need to set up your hugging face token on your local machine. Now you have fine tuned the model, but the question is how do you actually make inference on the fine tuned model? Okay, so this is where that app PY files comes in. Now let me show you what is actually happening. So we're using the transformer package importing the auto model for CASL LM as well as the tokenizer, right? If you have multiple GPUs and want to run this on multiple GPUs at the same time, so you can use the data parallel from Pytorch, right? Then you simply provide the path of the folder that was created by the auto train package. So in this case that's MJ prompts both for tokenizer as well as for loading the model. And after that we need to set the input context. So this is basically the prompt template that we used. So in this case my template is generate a mid journey prompt for a castle on an edge. So this is basically the concept, right? And the assistant is supposed to generate a response and then we take that input, encode it using the tokenizer that we have, generate a response from the model. So that is going to be our output. We convert that back to text, right? And print that text. So this is what exactly is happening within that app PY file that you saw. Now in terms of parameters, right? So the max length I'm setting to 85, you can play around with it, keep it longer or shorter and the temperature is set to 0.3. Now I am fine tuning this on close to 300 examples, right. So the results are not going to be that great if you set this to, let's say 1,000, so it kind of repeats itself. And I'm going to show you an example of how that looks like. Okay, so for that specific input is the output. We got a castle on the edge is a medieval fortress built on a cliff or a hilltop with the steep slopes on all sides. It's a symbol of power and protection and is often associated with the romance and adventure. Right. So it's a pretty decent output given that we only had 300 examples. Now here's another output. This is for a boy playing in the rain. So a boy is playing in the rain. He's wearing a yellow raincoat and holding an umbrella. He's running through the puddles and laughing. Right. So it's actually a pretty good prompt, I would say. So I went ahead and actually generated these images. So a boy running in the rain. This looks pretty amazing. And here's the one for the castle. This looks majestic, right? These looks great. Now, as I said in the beginning, you can create more complex prompt based on the structure that I shown you in the start of the video. Now, just to show you the importance of data curation. So in this case, what I did was I set the max new tokens to 1000. So as a result, this is what is happening. So the initial prompt was, or the concept was a boy playing in the rain. So it generated a pretty good response based on that. However, since the max number of tokens was set to thousand, so it kind of start repeating itself. So it generated another concept for itself, created a response, generated another concept, so on and so forth. Right? So if you're creating a data set, a couple of things to keep in mind. First and foremost, it needs to be large enough so that the model can learn from the data set. Then you probably want to have some sort of end of command or end of sentence token so that when it encounters that token, it will stop generating new words. Right. Because essentially these are simple next word prediction models. And that's why I also wanted to show you examples of where the model actually fails. Right. So these are some of the concentrations that you need to think about when you are curating data sets for training these LLMs. If you found this useful and would like to support my work, check out my Patreon. Now, if you want to learn how to fine tune these models using a single command, check out this video. Thanks for watching and see you in the next one.