LLAMA-3 🦙: EASIET WAY To FINE-TUNE ON YOUR DATA 🙌
A single-speaker YouTube tutorial (by LARRUZ) demonstrating how to fine-tune Llama 3 using the Unslot official notebook. The video covers installing packages, selecting models (Unslot-provided models with pre-merged LoRA vs. Hugging Face gated models requiring tokens), formatting training data into the Alpaca instruction→input→response schema, configuring a Hugging Face SFT trainer with Unslot model/tokenizer, running a brief demo training (60 steps) on a Google Colab T4 (memory and time statistics reported), performing inference/streaming, saving/loading LoRA adapters, and exporting the trained adapter+model for vLLM or GGUF with quantization options. The presenter also mentions alternative tools (Auto Train, Xolot, Llama Factory) and recommends longer training for better results.
Chapters
Llama3 is an amazing open weights model. But you know what's better than llama3? Your own fine tuned version of llama3. If you want to fine tune llama3 on your own data set, you have a number of options. For example, you can do that using Auto Train. If you want more advanced features you can use Xolot. Llama Factory is another amazing option. And then you have UNSNOT which promise up to 30 times faster training on their paid version. I'll be creating a series of videos on how to fine tune llama3 using these different tools. But for this video we are going to start with Unslot. Okay, so we are going to be using their official notebook. It's probably one of the best out there because it covers everything and 10 in a very user friendly way. So I'm going to just walk you through the notebook. You can actually fine tune a lot of other models, not just Llama three and we are going to look at some of the options. Okay, so first you will need to install all the required packages. So you can run this on your local machine as well. It doesn't have to be in a Google Colab book, but for that you need to have an Nvidia gpu. I don't think it has support for Apple Silicon yet. So essentially here we're just cloning the GitHub repo of Unslot. Right? And then depending on the type of hardware you have, it will install different types of packages. Okay, now next we need to just set up some training parameters. So you first need to import the fast language model class from Unslot. Then you need to set up your max sequence length. Llama 3 out of the box supports up to 8,000 tokens, but the dataset that we are using is relatively short text, so we are just using 2048 tokens. Now the data types this will automatically detect if you set it to no. And we are going to be using 4 bit quantization. Now under the hood, Unslot uses LORA adopters to do efficient fine tuning. So there are two options. Either you can use the version that is available on Unslot hugging face repo. So here they have already loaded a few models including the latest Llama 3, the Gemma model, the Mistral 7B. Right, these have already the LORA adopters merged to the model. So if you are using one of these you don't need to do anything else. But if you, let's say want to use one of the models from Hugging Face, then you will need to provide your hugging face token ID in case if it's a gated model. So for example, for the Metalama 3 version, you will actually need to accept the terms of services and then you can just provide that. But if you use another hugging face model, you will actually need to add Lora adopters to that model. And I'll show you how to do that. Okay, so this section is only if you actually need to add your Lora adapters. As I said, the Unslot version already has the Lora adopters merged with the model. So you don't really need to this do this step. But if you are using another model, just you need to define these different parameters or uncomment this section of the code and it will work pretty fine. Okay, now since you are bringing in your own data for training, so you actually need to format your training set. So for this example, they're using a clean version of the original Alpaca dataset. So let's try to understand how this data set is structured. Okay, so the data set has three different columns. First one is the instruction. So this is basically the instruction going into the model, then the corresponding user input, and then the output from the model. So if you were to structure your data, it needs to be structured in exactly the same way. You need to have instructions, then another column for input, and then a column for output. Now, in this case, you will notice that in some of the cases the input is missing, which is fine because the instruction just tells the model what exactly the output is supposed to be. Right? So again, if you are formatting your own data set, make sure to follow this structure. Okay, so the rest is very simple. First we need to download the data. So in this case we are downloading the data set from hugging face, and after that we need to map it and to this format. Right? So basically this is going to be a text string which will take the instruction section of the data set and put it here. So there's going to be these special tokens for instruction, then special tokens for input. So your input will follow these special tokens and then response from the model will follow the special tokens for response. And this is exactly what we are doing in here. Okay, so again we are just creating a single column where we are transforming these three different columns into this format. And this is one of the crucial parts. Now this is the standard Alpaca data set. There are some other prompt templates as well. For example, one of the more famous ones right now is chatML that was introduced by OpenAI. So you can structure those in here if you want. Right. But make sure to properly format your functions or your input examples because that's going to be fed into the LLM for training. All right, so once we do that, next we are going to set up an SFT trainer, so supervised fine tuning trainer from hugging face. This is based in the on the hugging face transformer library. So the SFT trainer is going to accept the model object, this is the unslot specific model object, then the corresponding tokenizer, then the data set. And then we need to tell which column to use. Right. So everything is structured in this text column. That's why I need to specify which column to use. Then max sequence length. Right. Some other parameters for controlling how the training is going to be performed, including what optimizer to use, what is going to be the weight decay learning rate schedule. Right. So basically if you want to change your learning rate as the training evolves, this is actually a really good idea. You there are multiple options that you can use, Right. And then you need to define your output directory where you're going to store the model. If you have seen some of my previous videos on training and fine tuning algorithms, you're probably familiar with most of these options. Now one of the places where Onslow shines compared to the other packages for training is it's optimized memory usage as well as speed. So in this case, if you see the GPU that we are using is a T4 GPU which is a free GPU available on Google Colab, and this is just using about 6 gigabytes of VRAM, whereas we have a total of 15 gigabytes of VRAM available. Now this does goes up during training, but it's actually very well optimized if you look at it. All right, so our training object is set. So we need to just call the trainer train function on the trainer object. Right. And here we can see that in the initial steps the training loss starts decreasing. Right. And it gradually decreases. There are some jumps here and there, but it has a pretty nice decrease. So that means that the model is learning. Now we could play around with the learning rate as well, along with the batch size. That will actually help us converge it easier. Another thing which I wanted to actually highlight was this thing. So we are not even running this for a whole epic. So we're not showing actually the data, the whole data set, we're just using a smaller subset of it. So we're just showing it a max of 60 steps. Right. So you Definitely. If you want the model to learn better, you want to run it at least for an epoch or two or at least more steps in here. Right. But for this quick example, we just want to see whether the training actually learns anything or not. Right. And it kind of shows that there is some learning that is going on. Okay, so let's look at some of the stats. So it took about eight minutes for training, but we ran it for just 60 steps. If you want the model to learn for longer, and I think actually learn from the data, you definitely want to run it for longer. Right now, the peak memory that was reserved for this training run is around, around 9 gigabytes out of the 15, and it only used about 4 gigabytes during training. So this is pretty impressive. Right? Okay, so let's say once you train the model, how would you do inference? So Unslot offers a very simple interface for, for that. So you just need to use the fast language model class from Unslot and then you provide your model that you just trained and tell it that you want to do inference on top of it. Right. Then we will need to tokenize our input. Now, since we were using the alpaca format, we also need to tokenize it using the alpaca format. So the first input is going to become instruction. The second input is going to be the actual input to the model. And then the model is supposed to generate responses and we move everything to the GPU so that we can use the available GPU cores to generate a response. Right. And then the rest is very simple to what you do with hugging face transformer package. So you call the generate function, provide the tokenized inputs, then how many max number of sequence tokens that you want to generate, Right. And whether you want to use cache or not. Right. So for this input, continue the Fibonacci sequence, and we are providing 1, 1, 2, 3, 5, 8. So here's what the model actually receives. Below is an instruction that describes a task paired with an input that provides further context. Write a response that appropriately completes the request. So this is basically the system instruction that is going in. Then here's the actual instruction. Continue the Fibonacci sequence, here's the input, and here's the model response. Now to be frank, the model without even training could generate the similar output to what we are providing in here. But this does shows that it is actually following this alpaca format. So that means it is actually learning something during training. Now you do, you can do the same thing if you want to stream the text. But in that case you will just need to use the text streamer class. And then if you run this, it will generate a streaming response. Okay, so once the model is trained, you definitely want to save it somewhere. So you have a couple of options. Either you can push it to hugging face hub or you can save it locally. In both these cases it will just save the Lora adopters, not will merge. It's not going to merge with the model. Right? So if you were to push it to Hub, then you can use model push to hub, but in that case you need to provide your hugging face token. Okay, so now if you want to load the Lora adopters we just saved for inference, then just set this to true and this will basically load the Lora adopters and will merge it with your model and then you can start using that for inference. Right? So for example here, although like we are actually using the model that was just trained, the Lora adopters are already merged to it. So here was another input. What is famous, what is a famous tall tower in Paris? So again, this is kind of the system instruction that goes in there. Here is the actual instruction that the user was just asking. And after that the model generates a response which states one of the most famous tall towers in Paris is the Eiffel Tower. And then kind of goes into a lot of details of how this was constructed. Right? Now a really nice thing about unsloth is that you don't actually need unsloth to do inference. You can use a number of other options. So for example, once the model is trained, you can use the auto model for causal them. This is basically theft version to actually do inference. So just like what you would do based on a base hugging face transformer model, right? You can use exactly the same classes, but according to the unslot authors, they say that it's going to be much slower if you use this class rather than using the unslot a specific class. So you definitely want to make sure that you use unslot for inference as well. But if you want to, let's say do inference using vllm, it does have support for that. But you can save it in float 16 directly and that way you will be able to use the model with vllm. Similarly, another amazing feature is that you can directly convert the model to GGUF for using with llama, CPP or Ollama and it's way easy, right? So all you need to do is just save the tokenizer and then when you're saving the model you need to define the quantization method. So here for example the Quantization method is 16 bits. If you don't define any quantization method but you ask it to save it as a ggf file, by default it will be saving it in 8 bit and you can also define 4 bit right so this was a quick rundown of how you can train or fine tune the latest Llama 3 model on your own data set using the amazing unslot package. If you haven't seen it before, I'll highly recommend to actually check it out. It it's one of the best option if you are constraints on GPU because in even in this case it was just using under 60% of the resources that are available on a TV T4 free instance. So this is pretty amazing implementation. They had to write the kernels themselves to optimize it and I think more and more optimizations are coming in now. If you are interested in no code platforms, I'll recommend to use auto train. I'll be actually making another video on that. That is another option if you don't want to look at all the code and try to run each block individually. But it's it's great to see that on day zero. Not only we have different packages that supports how to do inference on llama3, but you can also fine tune them. I hope you found this video useful. If you run into any issues or you have any questions, make sure to put them in the comment section below. Thanks for watching and as always, see you in the next one.
