Train (almost) Any LLM Model Using 🤗 Autotrain

Okay, great. So I changed the title a little bit because we ended up adding, uh, embedding models, fine-tuning, and autotrain. And I think that's also an interesting topic for this conference. Um, because some— you will fine-tune LLMs and then, then you're going to use it. So a little bit about me: I work at Hugging Face, uh, where I've been building AutoTrain and everything around it. And, uh, as you can see, like, even from my posts, I talk a lot about AutoTrain and I am also a Kagglers, but these days I'm not competing much, but I was the first one to get a Grandmaster in all categories. That's something I like to brag about everywhere I go. And now today we are going to look into what is AutoTrain. There was a question, what is AutoML and AutoTrain? I was very much interested in AutoML for a long, long time. But, uh, now nowadays it's not about tuning the parameters so much, it's about like finding the correct loss and a correct dataset, and that's probably what you need. Um, so we're going to take a look at what AutoTrain is and who should use it, the key features, and what are the benefits of AutoTrain. How to use, what are the available tasks, what are the features which are available. And, uh, we're going to take a look at fine-tuning LLMs. We'll try to see if we can fine-tune something live today. Hopefully nothing breaks. And, uh, then we're going to take a look at the recently added task of Sentence Transformers fine-tuning. And yeah, if you have any questions, feel free to stop me at any point. So AutoTrain, uh, was previously known as AutoNLP, and it was a closed source project. I don't know how many people know about that, but like BERT had just come out and everyone was crazy about BERT fine-tuning, um, for different downstream tasks. So, uh, we made something called AutoNLP where you could just upload your data and select the columns and select the model you want to fine-tune, and that's it. And then we would fine-tune on different sets of hyperparameters and present, like, how one model compares to the other, or like a different set of models. So you don't even have to choose the model if you don't want to choose the model. So, uh, that developed into AutoTrain. And we open sourced it, I think, end of last year or mid of last year. And since then, we have been continuously adding many, many more tasks and listening to what the community says. So AutoTrain is, is the no-code platform. There's a lot of different types of no-code platforms. AutoTrain is one of them. And it's valuable for those who want to use advanced machine learning, but they don't have a lot of expertise in which is traditionally required. But it's not just for those who don't have a lot of traditional expertise, but it's also for data scientists. So I use AutoTrain all the time. And it's, it's because it makes my iterations faster. So if I just have to check like what parameters are working or which model is good enough for me, I can start— I can just put in the dataset and let AutoTrain do its job. So it's like it can be used by everyone. So there's, there's a lot of different kinds of trainers, and AutoTrain is is not just about training, uh, or fine-tuning large language models. It can do a lot more. So you have different types of, uh, natural language processing tasks that you can accomplish with AutoTrain. Like, you have token classification, text classification, then you have all different kinds of LLM tasks, some computer vision tasks like image uh, recognition, uh, sorry, image classification, object recognition, and even the tabular data tasks. And AutoTrain tries to streamline the workflow for you. Um, so like, yeah, like, uh, today we are going to talk more about large language models and how you can use AutoTrain to fine-tune your own LLMs, like for either it's for chat or for on your date, for your own data. Um, so AutoTrain tries to like, uh, close the gap between, um, sophisticated or new models that are coming in and uh, it makes it easier to train models and it leverages everything in Hugging Face. So like you have Transformers, datasets, diffusers, the PEFT library, and others. Uh, one more important library there, framework I forgot here, is Accelerate. And thanks to Zach for that. Uh, so yeah, it leverages almost everything which is already there in Hugging Face. And that's why, like, when, whenever a new model comes and it's supported in Transformers, it doesn't take much time, or let's say it's also immediately supported by AutoTrain. Um, yeah, so let's move on. To, to start with AutoTrain, you can go to huggingface.co/auto-train and you will come across this page where you can click on Create New Project and it will show you a screen like this where it shows you who owns it. So you can put in an organization if you want, just give the space a name and then you can attach hardware from Hugging Face. But it doesn't mean that you cannot do it locally. I'm going to come to that part too. You can train the model anywhere you want. So you can attach a hardware, and, uh, Hugging Face has a huge list of hardware you can choose from. And, um, then you can— then you click on Duplicate Space. After a few minutes, it's going to show you a space, um, which has a login, and you need to log in with your Hugging Face account. And after you have done that, you will get a screen, uh, like this one. Now, uh, here you can see— I think you might have seen some previous version of AutoTrain. This is the new one that we have tried to build, trying to make it more, much more user-friendly by providing like dropdowns and inputs for, uh, different parameters. And, uh, on the side panel you can select like what kind of task you are interested in. Here we have selected LMS. And the hardware is local or space. So your training is going to run in the space. So when, uh, when you're running it locally, it's going to say the same thing, either local or space. And then you have logs and documentation and all different kinds of things. So these are the tasks that we currently support. We have the supervised fine-tuning, different kinds of— different other kinds of, uh, LLM tasks like Orpo, DPO, or reward tuning. We have different kinds of sentence transformer fine-tuning tasks and other test tasks like, uh, seq2seq, token classification. You can— you also have some image tasks and W tasks. And all it requires you to, like, uh, upload your data, choose the parameters, or change the parameters if you want to change them. Most of the time they work out of the box. Sometimes they don't. And you can either choose to upload your own dataset or you can just use the dataset from the Hugging Face Hub for almost all the tasks. So moving on to coming to fine-tuning large language models, we have— AutoGEM offers you different tasks like SFT or generate fine-tuning or code review and reward. And like, when we talk about supervised fine-tuning or, uh, generic tasks, the both, both of them are very much similar. Uh, the only difference here is SFT comes from TRLs, which is another library by Hugging Face. Uh, SFT trainer is being used, but for generic fine-tuning it's not used. But both are, uh, pretty close to each other. And, um, it's called supervised since we are collecting data from humans, but we still use cross-entropy loss. And in, in like generic, what a generic trainer does is to combine all the rows in one and like divide by block size that you have provided in parameters. And the only column here required here is text. So you must have seen like in the previous slide there is something called column mapping. So this is something different in AutoTrain. So it doesn't expect you to have your data with the same columns as required, the same column names, but you can actually map. So let's say in your dataset the, the column containing all the conversations is called conversation. So here you can write conversation. Um, and yeah, so chat template is also available if your data is, uh, not formatted. Uh, if you want to format it using some kind of chat template, you can also do that. So this is an example of the dataset. So this is like WikiText So a lot of people think like SFT can— we can, we can just do, uh, instruction tuning. But yeah, that's not true. You can use any dataset you want and you can, um, just fine-tune your existing, uh, large language model. So here, uh, we have a subset and we have a training and we can use this. So this is dataset from Hugging Face Hub and you can just plug it into auto-train and use it directly. And I, I can show you that in a small demo later on. Um, here is another format where you have— which is a chat format, and where you have content and you have the role. And, uh, this is like the JSON Lines format, but, uh, right, uh, what I'm showing you right now is, uh, obviously from Hugging Face Hub. So it's in datasets format. So here you can, uh, just use whatever kind of chat template which is suitable for your data. In this case, it's ChatML. And, um, you can also select the Sapphire template, which expects a system prompt, or the token— or the tokenizer's chat template. So when, when you're selecting a model to fine-tune, the tokenizer may have a chat template and you can select that. But in, in, like, if, if you don't want to select a chat template and your data is in this format, uh, you have to convert it to plain text. Um, then we come to reward model, uh, Reward Trainer, which is also offered by AutoTrain, and you can train a custom reward model, which is more of a sequence classification task. And determines which example pair is more relevant. And in that case, your dataset consists of two different columns. One is a chosen column and one is a rejected column. So when it comes to AutoTrain, it always has a text column for the dataset, and, and the column will become rejected text. Um, then Orpo or DPO— a lot of people are famous, uh, are like really interested in training a DPO model. But I'm moving more towards Orpo because Orpo doesn't require a reference model and takes much less memory and much less compute. So I would recommend, like, if you're trying a deep train or DPA model, you can probably try Orpo one first and see how it performs. And in this case, the dataset has 3 different columns. One is prompt, and then you have the chosen column and the rejected column, which are all conversations, uh, the one that we are looking at right now. And, uh, even in case of Orpo or DeepView Trainer, you can use the chat templates. So, uh, dataset format, I think it's the most important thing when you're using AutoTrain. All you have to do is create the data and create it in a proper format. And once you have that, so like you can, you can use a CSV dataset, you can use JSON lines, JSON-L dataset. In most cases, I would recommend you to use JSON lines because then if you're using CSV, you have to convert. Like here you can see the chosen and rejected columns and their list of dictionaries and then you have to convert them into stringified list, which is not good. So JSON lines is much better, more readable. So one of the formats, if I'm talking about like Alpaca dataset, so here is a JSON line. File and it has like a key text and it contains the formatted text. So if I have dataset in this format, I don't want to— I don't have to use, uh, chat templates. But if I have dataset in the previous format that I've shown you, then you will probably, uh, want to use chat template. It's not going to work without that anyways. Or you can use chat template offline and convert your data into plain text, a single column, not for OrcaDBU but for SFT Trainer. So if you're looking at everything, AutoTrain can be found in the Hugging Face repository— sorry, the GitHub repository. And if you have any, if you have any issues or feature requests, feel free to create feature requests or issues in the issues tab. So we try to resolve issues quite quickly so you don't see many open, but you have a lot of them closed. And now comes another part. I think I should go back here to the presentation. So let's do this part quickly and then I can go to a training model. So in the documentation you have everything that you need, hopefully, to— how to— how do you install AutoTrain? So all you have to do is pip install autotrain-advanced, and then you can run the app. So to run the app, you need to provide your Hugging Face, right, token in an environment variable called HF_TOKEN. And then you can run the app. And then the app that I showed you before, you can run the app locally. So your trainings will run locally, and you need— your token is only required like to push your trained model to the hub, or in case you want to train, um, fine-tune a model, uh, with a base model which is like gated, like most of the models that we have today, like Llama 3 or Mistral or anything. So app is not the only way to train a model. You can also use config files. Everyone loves config files. And the config file originated from AutoTrain CLI. So you also have the CLI. So you can try write autotrain --help, and it's going to show you all the CLI commands you can train all the different kinds of tasks that you can train in the UI using CLI. But sometimes config files make it easier. So here you define the task. So you have a list of tasks that you can find in the documentation. It's the same list as is present in the UI. And so like LLM or PoE, and you define the base model that you want to use, and then you give your project name because requires an output folder to push everything, and whether you want to log it using TensorBoard or Weights and Biases or no logger at all, and backend, which is local. And then you provide everything related to the data. So now each, each task will have something different that is required for the data. Here we need the data path. Most of, most of the tasks require you to have some kind of data path. and then a training split and validation split. If you, if you don't have validation split, you can just put it as null. Then if you want to use chat template, put ChatML or Zephyr or tokenizer, or just set it to null. And then a column mapping. Um, other than that, you have the parameters, uh, so whatever the parameters you want to set. If you don't want to set all the parameters, you can set only 3 and Everything else will be default. And then if you want to like push your model to the hub or not. So if you want to push your model to hub, then in that case we require your Hugging Face token and your username under which you— we should push the train model and datasets if there are any. But otherwise, your, uh, we don't you don't need to provide your Hugging Face token if you want to keep your model locally. Your model will be stored locally on, uh, this, uh, project name output folder. And like, if you're using a dataset which is a local dataset— so here I've shown like you have to use, uh, you— here I've shown a dataset from the Hugging Face Hub, but you can use local dataset. In that case, the path will be path to a train.csv or a train.json file, and your dataset won't be pushed anywhere. It's going to be stored locally in case you have like sensitive data and you don't want to push it anywhere. You can also, you can also like use— so JarvisLab is one of the sponsors of this course and they are providing credits. So you can also go to Iris Labs, click on templates, and here is Auto Train. You can just click on Auto Train and you can start training a model. So like you can choose what kind of GPU you want, and here you have to enter your Hugging Face, right, token because it's running in cloud. So you have to store your models somewhere in the end. And then you just launch the instance and you have the fully fledged AutoTrain UI running on Iris Labs. Or like if you don't have much resources and you want to train a 70 billion parameter model, you can also rent them from TGX Cloud where you can have up to 8 H100 GPUs. Um, and yeah, um, so going back, um, I was in the GitHub repository, and GitHub repository also provides you way to run AutoTrain on Colab, where I have created, um, kind of like a— like I'm, I'm, I've created like a small UI exclusively for Colab so you can just use that and you can, uh, like upload your data to Colab. And, um, let me see if I can run it. And okay, okay, so, uh, it will run.

By the way, um, I noticed that like whenever I tell people about AutoTrain and they go to the website, they actually don't ever find the GitHub repo from that site because it has like the user interface you click on and stuff like that.

Yeah.

Um, do you— I actually like the GitHub repo a lot because I can like run it locally or whatever, you know.

Yeah.

Um, is there a reason that it's not on the website, or is just like maybe just, uh, is it not on the website? So like, yeah, when I mean the website, I mean, um This one, or let me just put it in our panelist chat here.

Sure. Oh yeah, okay, um, and that's a good thing. We'll add it tomorrow.

Okay, cool. Yeah, so like, so it's the same thing, right? Like that interface you get there is like using the code here.

Yeah, yeah, it's the same thing. So, um, for, for Colab let it load, I will show you. But it's the same thing, like you can deploy on Hugging Face Spaces from here, or you can use ngrok if you want the same UI as we run on Spaces, or you can just run everything locally anywhere and you will get the same user interface. But most of the time, like people from what I've seen are mostly interested in running using a config file when they're running locally or use the CLI instead. But yeah, we, we do provide, like, if, if you want to run the UI app or API or CLI or anything, you can, you can run it anywhere. One more thing, like, there is also a Docker container, so if you want to, like, use, uh, use that. You don't have to worry about installing, um, different kinds of stuff. So you can just pull from huggingface/artifactory/latest, and it's always updated to the main branch of the repository. So, uh, you can, you can also start from there. So then you, you don't have to care about different kinds of dependencies. Um, Okay, so this is still running. I'll come back to that later on. But, um, like, after create— clicking on create new project, it's a good suggestion I should add the GitHub repo link here. And, uh, you, you'll come to this kind of page. So let's try to train, uh— so it's right, right now it's running on L4 GPU. You can select in settings if you're running on Hugging Face Spaces. And okay, here I have, okay, here I have the SFT task, and here I can select many different tasks here, but I don't think you see the dropdown in the video. So I'll just select LM SFT and sure, why not? And I can, I can either upload my dataset or I can click on Hugging Face Hub and then I have to provide the path to the dataset. So like wherever, Abhishek slash whatever, wherever the dataset is stored, uh, on Hugging Face Hub. And, um, you can provide the training and validation splits. But right now we can just upload a dataset. So the Alpaca JSONL dataset that I was showing you, I can just put that, um, and it has already has the column name as text. If let's say the column name in the Alpaca 1K JSONL file was, uh, messages, then I would just write messages here. So right now it's text. And then I have many different options I can choose from. I also have like— we don't try to overwhelm you with all the different parameters, like, so we have just to remind the parameters that most of the users change, but you can always click on full parameter mode and then you— sorry, then you get like a list of all the different kinds of parameters you can change from. And at any point you want, you can switch to JSON format if you feel like just copying pasting the parameters from the GitHub repository. Or, uh, yeah, so here I have like— here if I want to change things, I can change like number of epochs. Let's make it 1 to make it faster for this one. And learning rate or gradient accumulations. And the explanation of like all the parameters is also provided in the repository in the parameter section. So if you have any confusion, you can just read up there. And then you click on Start Training, um, and then in the logs here you can see like, uh, what's happening and how is the model training. In the meantime, I can probably switch to Colab and show you. Okay, yeah, so here is a Colab UI. It looks very much similar to what we have, but, and probably will be improved at some point. But this is a Colab UI. But if you want to use the full UI, you can use the different link here, run AutoTrain UI on Colab via ngrok. And in that case, you need to provide an ngrok token, and you can run everything on Colab. Um, okay, so yeah, so I think, uh, the model is training right now. Um, another thing like I wanted to mention was once the model is trained, it's going to be stored in your Hugging Face repository. And since like, if you, if you see right now, like I'm running it on Hugging Face environment, so downloads are pretty fast. And once you, once you are done training the model, if you want to deploy it using inference endpoints on Hugging Face, you can, you can do so. Like, it's just like a single click. If I go back to AutoTrain, We'll see here that it has created— it has already created a repository for me. So for the training that I have started, and when once the training starts, I'll see a TensorBoard tab here. So where I can track the trainings live. And you can also, uh, you don't have to keep this space open, so you can close the space and come back and just, just remember to set a sleep time which is higher than your training time. Um, and, uh, yeah, once the training is done, you're free— feel free to use it anywhere. AutoTrain, um, configs you can also find in the repository. So there is a configs folder, and if you go inside that, um, you have like config for many different kinds of— all, all different kinds of tasks that we offer. And so like just for GPT-2 and, uh, Conaco dataset, and, um, you can just do autotrain config and, uh, use the raw URL to just train on this config directly. Uh, one thing you should have noted already, you don't have You don't have a lot of parameters, so you cannot adjust like, uh, you want to use DeepSpeed or FSDP, these kind of things. So FSDP is currently not supported, so anyways you cannot use it, but DeepSpeed, when it's required, it will be used. And it's generally, uh, used only for multiple GPU setups. Like if you have 4 GPUs, it's going to run, uh, in DeepSpeed mode. If you don't have 4 GPUs, then it's going to run distributed data parallel. So is, uh, QLoRA supported on DeepSpeed? Yeah, yeah, it is right now. Okay, yeah, so, um, thanks to Peft. And, um, yeah, so you don't have to adjust all these parameters. It's like all, all you need to think about is basically the data. And yeah, that's, that takes most of the time. After that, everything becomes quite easy. And like, yeah, I was saying, like, all of the tasks, they run— like, if you have 2 GPUs or if you have 4 GPUs, they run in multi-GPU mode out of the box. So you don't have to change anything there. And everything is powered by Accelerate, so you don't have to— you don't even have to do accelerate config. It's going to— it uses its own config anyways. So the autotrain command, it's wraps, um, Accelerate, and it uses that one. Um, okay, I, I think we can, we can take some questions, but before that, I would just quickly show you like the different kinds of sentence transformer tasks that you can also do. Um, so like if, if you want to, like if you want to improve your RAG models, so you can tune your own, uh, fine-tune your own embedding model, or you can also start from scratch. So here I'm using MPNet base, but you can also tools, just like a word-based uncased. Um, okay, um, I think, I think the model is— yeah, so the model is training, and when the model is available, it's going to show up here. Uh, and with that, I think— yeah, okay, so this one was the sentence transform triple task, which I've already shown you, and Thank you very much.

I think, can you like log? Do you have like, you know, Axolotl has like whatever logging to Weights and Biases, kind of useful. Do you have something like that?

I'm sorry, I didn't get the question.

Oh, so like Axolotl, you know, you can like log to Weights and Biases.

Oh yeah, you can.

You can do that here as well.

If you're using the UI, it's It's not possible, but if you're using your own config files, you can just write, uh, W&B here. Okay, yeah, you should have your token exported somewhere when you're running the command.

And just so people know, like, um, where the— is there good documentation about the config file? Like, what are the different values, key value, or key values you can have and stuff like that.

Yeah, so, um, from in documentation we have, uh, shown you, uh, how to create the config files. But if you're looking for like any specific task, you will find the config files here. Like, like, um, I've done recently for Sentence Transformers, like I have config for each different kind of trainer. And for LLM fine-tuning here, like we have like a config file for SFT, one for Orpo, DPO, and like DPO QLoRa. If you want to run, if you want to start your training locally but run in space, so you have a config for that too. And other configs should be based on that. But we, like, we always welcome, like, if you want to improve the docs or if you want to create your own config and share here or in a different repository, please feel free to create an issue. And if there's something missing, it will be added in less than 24 hours. In the JSON, so yeah, there's a question about does AutoTrain lint the parameters so you don't pick the incompatible ones? So yeah, the parameters are based on a Pydantic base model, so it's going to tell you it's not you won't be able to select any incompatible parameter. Hyperparameter optimization, we currently— AutoTrain doesn't do for, uh, LLMs. It's going to take a lot of time. Fine-tuning takes a lot of time anyways, but it does hyperparameter optimization for some other tasks. Um, the next question: How big of a model can you train on AutoTrain using the sweep CPU option? I'm not sure, I never tried, but you can. So one more thing I should mention here, like you can also use Auto Train on your M1, M2, M3 Mac. Can you add your own chat template? Uh, technically yes, you can. So, uh, all you have to do is, um, clone the model and create your own model repository and go to tokenizer_config.json, and inside that you have the chat template. So just change it and you can use a model with a different chat template. So there are a couple of questions about creating synthetic data. So yeah, hopefully we will have something related to that soon, but as of now we don't have that. So you need to come up your— with your own dataset, but you can You can always go and generate your own synthetic data and use that instead. Another question.

Yeah, okay.

So another question, this was about Tokenizer's chat template, tokenizer_config.json. So I already answered it in a different question. So yeah, you can— yeah, that's correct. Uh, for a couple thousand rows, it takes, I think, 15 minutes to train. So this question was about how much time it's going, like, ballpark. Yeah, I think I've answered most of the questions. If, if there is something that I missed, please let me know. Awesome. Um, yeah, this, uh, like Hamel Husain said, we're big fans of, uh, the idea and, um, excited to try it out. So thanks so much. So just, just one more thing, like, if we have time before— yeah, so yeah, so this is the training that I started and it had 1,000 rows of data. It was running on a T4, uh, sorry, uh, L4 GPU, I think. And the model training has finished and it just pauses the space on its own so, so that you don't spend money. But if you're running it locally or on some other cloud systems, it's not going to pause on its own. We ran this only for one epoch, so we don't have much, but we have like all the, all the files that we need. And if you want to just deploy it, you can deploy it on many different options, or you can just copy them locally and just deploy it anywhere you want. Nice. Okay, thank you so much then. Thanks for having me.

And yeah, thank you.

Hey, Samshak, appreciate it. Bye. Thanks.