End-to-End (small) LLM Fine-tuning Tutorial (from data to model to live demo) | On DGX Spark

Ladies and gentlemen, welcome to the Machine Learning Cooking Show. I am your host, Daniel Bourke. And in this episode, we are going to fine-tune a small language model or an SLM locally. Now, what you define a small language model will be subjective. For me, it's something that's under 1 billion parameters. More specifically, here's what we're going to do. We're going to download a model from Hugging Face, Gemma 3 270 mil. We're going to download a dataset from Hugging Face. We're then going to fine-tune Gemma 3 270 mil. That will be our base model. On the custom dataset for a specific task. We'll then upload our fine-tuned model to Hugging Face so we can share it with others and even reuse it ourselves. Then we're going to create a demo for people to interact with, and all the links that you need will be down below. Just a quick note, this is a quick version. I'm not going to go step through all the code. I've already done that in livestreams on my channel. I'll link those below as well. So we're going to go through this relatively quickly, but all the links that you need will be down below so you can step through it at your own pace. Now, why fine-tune your own model? We want to own the model. We can run this on our own hardware. The data stays local. We don't have to send data, our own custom data, to any API or other company. So if you're a big business, you might want to train a custom model to run on your own hardware to do a very specific thing. Two, our task is simple enough to just use a small language model. So oftentimes we don't need a large language model. We don't need a model that's been trained on the entire internet. We just need it to do a specific task such as extract structured data from text, and we'll see what that looks like in a second. No API calls are needed, so we can run it offline if we need. So I think this is a big future for models and even robotics and even models running on your own phone later in the future is that you won't always have an internet connection. So it'd be cool to just run the things that we can on a local device. Finally, oh, number 4, we can run it in batch mode. So we can get much faster inference, as we'll see later on in this video, than just doing single API calls once at a time. So if we needed to do batch inference, we can get our model, as we'll see, up to about 1 million samples per day. And then number 5, the model by default when we downloaded Gemma 3 270 mil wasn't very good at our task, but since fine-tuning on our own custom dataset, it is very good. And of course it could be improved, but this is only a small dataset. Something I want you to take away is that LLMs are tokens in, tokens out. And by a token, I mean a numerical representation of data where data can be almost anything such as an image, video, text, you name it, DNA sequence, anything. And so if you have the right set of tokens, a la your own dataset and your own ideal outputs, you can customize your own specialist LLM for that specific task, as we'll see what we're going to do here. Now we have some ingredients. We have a base model. Of course, I've mentioned that before, Gemma 3 270ml. And we have a dataset which is called Food Extract 1K. Let me just show you that. This is like all good cooking shows, something I have prepared earlier. The goal of Food Extract 1K is to take an example sequence of text. Just— it could be anything, right? Could be gibberish, could be an image caption or something like that. And we want to extract food and we want to extract drinks if they are in there. And so what I've done is I have a sequence of image captions. Some of them are real, some of them are synthetic, some of them are just gibberish strings, as you can see here. But then I've gone through and labeled them with GPT-OSS-120B. So why would I do that? Well, GPT-OSS-120B is a powerful open source model, but it's quite large. It's 120 billion parameters. Specifically, it's 444 times bigger than the model we're going to train, Gemma 3 270 mil. So we're essentially distilling the knowledge of a big, powerful open source model into a very small one for a very specific task. Now let's go back. The tools that we're going to use, we're going to do mostly all of our work within Transformers and of course the TRL library in terms of software. The TRL library is Transformers Reinforcement Learning, but it can also be used for supervised fine tuning, which is what we're going to be using. Supervised fine-tuning in a nutshell: you have inputs, you have outputs. In our case, our inputs are raw strings— it could be any kind of string— and our outputs are structured data around, uh, food and drink text. And so, um, the reason why we want to do supervised fine-tuning is because it's one of the most straightforward. You have example inputs and example outputs, and you train the model to reproduce those. Now I'm going to be running all of this on an NVIDIA DGX Spark, because NVIDIA sent me one and it's really cool and I want to showcase how to use it. And personally, I would want one for myself too. And if you don't have an NVIDIA DGX Spark, perfectly fine. All of this code I have verified to run in Google Colab on an L4 GPU, which is one of the free instances. You could even do it on a T4 GPU as well. As long as it's about 16GB of VRAM, you'll be able to run this, no worries. You just may have to lower your batch size slightly than the default I've got. I've got a default of 16. You might want to use something like 4 or something like that. But let's keep going. I told you this is the quick version. This is my current setup, exactly what I'm sitting at. I am running this code. Everything you see in here for me will be running on the NVIDIA DGX Spark, a great little piece of hardware. Um, I'm coding on my Mac Mini but connecting via SSH, as you can see down there, to this little beautiful golden machine. However, as I said before, I've verified all this code works in Google Colab. So whether you're running it on a dedicated piece of AI hardware or in free software online from Google. The world's your oyster in the modern world of AI. So let's look at our task. As I said, our project is food extract. We want some input text, could be anything, could be a caption of food, could be a caption of a truck in a showroom. Whatever it is, we want to output some structured data. Why would we do that? Well, we could then use this model to filter a large dataset of image captions such as Datacomp 1B. Let's have a look at that while we're here. I want to make everything as practical as possible. So Datacomp 1B. Here we go. So this is 1 billion image URLs and captions, right? And if we're a food startup such as Nutrify.app, which me and my brother actually are, this is our app. You can go download it now and try it out on the iOS App Store. Me, we might want lots and lots of images of food. But we don't have those. So we might go to an open source dataset like this, filter all the text captions for food captions, and then we might use those food-specific images for our use case. So if we go back here, I want to highlight— oh, spoiler alert, that's the code— that this is very broad. What I mean by structured data, it can be any structure. This is just tokens in and this is tokens out. That is the key takeaway from all of this. An LLM is tokens in, tokens out. The beauty of LLMs is that they are decoder architectures. And what I mean by that is that your inputs and your outputs can be almost anything. So if you wanted to change this structure, no worries, go ahead. If you wanted to change this input text, maybe you're a business who sells insurance and you get 10,000 customer emails a day and you want an LLM to run on your local hardware because that's all private data, right? And you want it to extract things like customer problem, customer location, all this sort of stuff, you can put that into a Google Sheet, a DataFrame, sort over that very easily. And so in our case, we have food or drink with a field 0 or 1, which is, is this about food or not? Then we have tags. I'll show you that. That comes in a tag dictionary. In this case, it's food item, drink item, because then we can filter it quite easily as well. We've got menu tags, recipe tags, all that sort of jazz. We have some food items which are just the nouns of what foods are in here. So this is what I had for lunch the other day. Beef mince, white rice, hot sauce, and a side of pickles and pepper and a glass of water. So it is as you would expect. We have beef mince, white rice, hot sauce, pickles, peppers, water. Now input text here. We've got a photo of a car driving over a hill with a sunny background and several birds flying through the sky. We don't care about that image if we just wanted food images. So this is going to get a zero, an empty list for tags, food items, and drink items. Now, as I said, you can change this to however you want. I've just built it as food extract because, hey, we all like food. We all eat food. You can even try out— actually, I'm getting ahead of myself. Let me show you what we're going to finish off with. We have the code again. Links will be below. We have the code to create a reusable fine-tuned Gemma 3 270ml model. There's a— this is publicly available right now and we're going to have a shareable demo which you can try out for yourself. So let's just try that out now. We'll go back into Chrome. Here's that model I was talking about. This is on my profile, Mr. D Burke on Hugging Face. Your username will likely be there if you follow this through to the end. Then we have the model there and we have a demo. So let's try this. There's some input text. Remember I said this can be anything at all, right? Will our model's outputs be perfect? No. But the beauty of machine learning means that you can— errors are actually very helpful for you. You can use those to improve your model in the future. So why don't we do this? Now, for today's lunch, I had sardines, pickles, and peppers and hot sauce. That is a true story. I am being on a very much of a sardine, pickles, peppers, hot sauce run lately. So there we go. Beautiful. Food or drink? One. Food items? Yes, we get a tag for that. Sardines, pickles, peppers, hot sauce. And after lunch I had a cup of coffee. I didn't actually do that. I had a cup of coffee this morning, but it doesn't matter, right? We're just seeing if our LLM works. This is fine-tuned. There we go. Coffee. Is coffee a food? Well, that's, that's potentially up for debate right now. Of course, we've got the drinks here, we've got the food here. So this is just a very bare-bones example. Remember, this is only on 1,000 samples. I've got 1K here. In fact, we actually like it when our model gets stuff wrong because we can go, okay, I need some more data samples on that in the future. If you'd like to see a video on how to create your own custom dataset in the future, please leave a comment below. I'd love to make it. Now I'm running on the DGX Spark. It's a little beautiful little piece of hardware. Look at that, so small. Um, and the beauty of it is, is that you get new software model optimizations of Supercharge NVIDIA DGX Spark. I personally haven't tried to run GPT-OS 120B, but apparently it goes 30% faster or so now. So maybe again, that's a future video or livestream where we create some synthetic data to potentially improve our model. See, that's the beauty of AI in, I guess, 2026 and onwards, is that you can create high-quality synthetic data to bootstrap a dataset for a specific work case, use case. Now, as I said, all of the code is available here. This is what we're going to step through. I'll leave a link to this below. If you'd like a step-by-step walkthrough of this, please see the live streams on my channel. We coded this from scratch the other day. And there is also a video course on the Zero to Mastery platform. So if you go to this link, I'll be stepping through— stay tuned, the LLM one's not on there just yet. But we've got two projects, text classification and object detection. I go through every, every line of code step by step. As I said, this one is the fast version. So if you want the step by step, that is available on Zero to Mastery. So let's go to here, Hugging Face LLM fine-tuning tutorial. I'm going to be running this in VS Code. As you can see, I've got the Spark available there. If you wanted this in— I'll just show you, here's the GPU we're running. If you wanted to run this, what can you do? You can also run this on Google Colab. I wouldn't mind if my Escape key would work. That'd be fantastic, wouldn't it? Nvidia SMI. Maybe I'll restart this. The classic— I go to show a demo and it's, it's not working. Anyway, there's some information in here. As I said, all the links you need will be below. Nvidia— is this going to be— excuse me, I, uh, I had to restart my entire notebook for some reason. But here we go. NVIDIA SMI. Here's my GPU we're running on there. NVIDIA GB10, that stands for Grace Blackwell 10. That's what's inside this DGX Spark. But let's get started. So first and foremost, um, we're going to import our dependencies. If you're on Google Colab, you may have to install these. Then we can check what we— memory we have available. Beautiful thing about the DGX Spark, it's got 128 gigabytes of memory. So we've got a fair bit of room to play with here. First things first, we're going to set up our base model, which is Gemma 3 270 mil. I'm using the instruction tune version. I haven't actually tried out the other version, which is the base version. Maybe that's better. Maybe that's an experiment you could try and tell me if it works better. I'm going to use instruction tune, so it's tuned to follow instructions. Then we go down here, we get a tokenizer. What's a tokenizer? So a tokenizer is— remember, in machine learning and AI, computers don't like text, images, videos, whatever you name it, audio, they like numbers. So a tokenizer is just a way to turn raw data into numbers. Each model generally has its own specific tokenizer. The beautiful thing about Hugging Face is that it will automatically load the right tokenizer from our model name ID. So then if we run our model, boom, we can run our model, but we don't get some good outputs here. We get some logits. They are a discussion for another story. But we want this to be human readable. So let's work towards that. Our next step is to get a dataset. This is the food extract dataset. We can download that using Hugging Face datasets and the load_dataset method. Maybe I zoom in here a bit more so we can see. As I said, it's a relatively small dataset, 1,400 examples. Some are real, some are synthetic, but they are all labeled by a synthetic model. I personally haven't labeled any of these. In the future, if we wanted to make our model more robust, we would definitely look to create an evaluation dataset, maybe of at least 100 samples, that we have personally vetted to make sure that they are correct and what we want. So when we train a model and evaluate it on our human-made test set, we can tell that it performs well if it does well on those. I like to, whenever I download a new dataset, view at least 100 random samples. So let's have a look at this. We've got input nutrition information label for sliced 5 Seeds bread and then a bunch of information about the, I guess, nutrition information. And so example structured output, what we want our model to predict. So is food or drink is true. And then we've got some tags. You notice how this is JSON, right? But I have condensed it to be more of a YAML format. Now, why did I do that? Let me just take this. This is a great tidbit that I think is important if you're going to work with small LLMs. Let's just go to a token counter. Tokenizer. This is OpenAI's tokenizer. Won't be exactly what we're looking for, but it will be close enough. So 71 tokens. A token is a unit of data, right? And our model is a decoder, so it's going to have to output each of these tokens one by one. Now what's going to be our limiting factor there? Well, if we have to output one of these tokens one by one, uh, every excess token that we output is going to be a longer generation time. So if we could get the same information but condensed, you see how this is JSON, but a lot of it is sort of redundant, like the brackets. Every time we have a bit of, um, information here that we have to generate, that's taking up compute power. So if we go into here, this is our condensed output. How many tokens is this? So we got 71. Now why don't we just get this one actually. And we compare them side by side, that would be really nice. Oh, excuse me. We can just enter text. 50 tokens. So you might think, oh, Daniel Bourke, 21 tokens is not that different. Like it's only 21 less. Yeah, but times that by a million, right? And that's what you kind of use these models for, is to inference over a large data— a dataset. And if you're doing inference a million times, well, that's 21 million tokens you've just saved. So we could actually condense this further. So that's another project. Maybe we do this as FD and then TFD, and then we could condense it even further. I haven't really played around with that. I'm happy with the 20 token saving on this particular example. So let's go back. We've discussed what our input is and where did these inputs and outputs come from. Sorry, I'm getting ahead of myself. This is from our dataset. Remember, we've got sequence, we've got image URL, and then we've got a bunch of columns there. I've just uploaded this. So this is the original label from GPT-OSS-120B in JSON format. This is a condensed format. So we're going to train our model given the input string over here, the sequence. We're going to train it to create the condensed version and then we can just write a parser. I can just give it to Claude or something like that and go, hey, pass this into this format, right? Easy. So the idea here is that we generate less tokens so that if we're performing inference, we get it faster. Because we'd like to use our model to potentially filter a large corpus of data, we get it to assign, get it to assign various tags to the text as well. So here are those tags I was talking about. So NP stands for Nutrition Panel. There we go. Nutrition panel, ingredient list, menu, recipe, food items. I wonder if you could try out with the demo later on to generate some of those tokens. But let's stay here for now. We'll keep going. So what have we done? We've loaded a model, we've loaded a dataset. The next thing we need to do is format the dataset into LLM style inputs and outputs. Now let's look at one of our random samples. This is just what we saw from our dataset in Google Chrome. But it's now in JSON format, machine-readable stuff. So sequence, this is going to be the input to our model. This is text. It could be any string, right? It doesn't have to be about food. It could be any string. That's the beauty of LLMs. Now we have some UUID, we have some information here. I'll let you read through those. They're pretty self-explanatory. But these are the things we care about at the moment is GPT-OSSE-120B label and the condensed. So we want to train our model given this sequence, produce this condensed label. However, If we put direct text into our model, it might come out with non-ideal outputs. The reason being is because each LLM or each model on Hugging Face is generally trained in a particular style. So in our case, there's a Gemma 3 is in conversational style. And so what this is, is we have a role. It's in this little format, right? You just got to put your sample sequence here into a content dictionary with the role of user. I'm the user, I type this in, um, information, and then the role, the system, so the model or system outputs this, right? So this is our input, this is our ideal output. We want to train our model to, given any of these inputs, produce our ideal output. And now if we do sample to conversation, we get messages here. Now if we want to run this little helper function across our entire dataset, right? We can do that with the map function. So this is basically saying, hey, take all the samples in the dataset, apply sample_to_conversation to that, and then we go boom, wonderful. Now if we notice our sequence from above, right, we don't have a messages field, but now we do messages field. Okay, this is just sample number 0. We could go to my favorite number 42. So another optional quest takes place on Windfall Island during the nighttime play. Okay, that's not about food. We don't want that. So what did we get? Food or drink? Zero. No tags, no foods, no drinks. Roll of the system. Wonderful. Exactly what we'd want. Now, number one rule of machine learning is create a training and test split. When you see all these big benchmarks that these LLMs are doing, always be wary if you see something that's performing too well on a benchmark because chances are if an LLM has trained on the whole internet and it's getting really good at certain benchmarks, some of those benchmarks might have been leaked into the training data. And so that would be like if you were at university, you were studying on the final exam, and then when you did the final exam, you already knew it all. And so the results aren't actually reflective of your knowledge. You're just regurgitating what you, what you basically, I guess, cheated on, right? So we create a training and test split. We're going to do 80% train, 20% test. And so this means we'll have 1,000 samples to train on, thereabouts 300 to test on. So during training, our model will see those and will evaluate on these. So when we evaluate our model later on, it will have never have seen those samples. This is the number one rule in machine learning. So when you're training your own models, always leave a few samples to the side so you can evaluate it and get an idea of how it's performing later on. Now let's try the model with a pipeline. One of the easiest ways to do any type of Hugging Face model inference is with pipeline. We can put in text generation, the target model we have. Remember, what is that? That is Gemma 3. This is our architecture. It is using an attention architecture. We've got a decoder here. Decoder just means it can output almost anything. I'll let you explore that, uh, more in depth later on. Right now we're just focused on actually training it. We've We've got a tokenizer, we have some input text. Now we're going to use this little helper function to turn it. As I said, we need it in conversational mode. So we're training that to— oh, sorry, we're just formatting that to be the role of the user, where the user, the content is the input. Hi, my name is Daniel. Please reply to me with a machine learning poem. That's our input there, right? So if we took this to something like Gemini, Let's go to Gemini. Please reply to me with a machine learning poem. Exactly right. Great. Okay. I don't need an LLM to write me poems, but I'm just showing you what it looks like. Inputs and outputs. We have an input prompt. Now the thing is, our input text here needs a little bit of formatting before it can go into the model. This is one of the biggest roadblocks to all kinds of machine learning is your data needs to be formatted in the same way that the model was trained on. So what we're going to do here is we're going to do input prompt, tokenize it. We're going to apply the chat template, which is essentially applying some of these special tokens. And then we're going to go, this is the input prompt, and then we're going to create some outputs. It's going to be the first time we use Gemma 3 for inference, right? We did actually do it above, but we didn't see what it outputted because it outputted some logits. And I can't read logits. Maybe you can. I like to read text. So let's go pipe input prompt, max new tokens 512. If we wanted to shorten the output, we could do 256, but I'm going to let the model write a poem as best it can with as much output as possible. So if we— before we run this, this is the input prompt. We have these special tokens. I'm not going to go through all the special tokens. What you just basically need to know is that if a model has been trained with special tokens, which Gemma 3 has, you want to definitely apply those to your inputs. Otherwise your outputs may be not good, right? So the special tokens here, we've got BOS, which stands for beginning of sentence. And then we have start of turn for the user. And then here's my prompt. Hi, my name is Daniel Bourke. Right. So behind the scenes, when you type into Gemini or ChatGPT, it's probably doing something like this. I don't know exactly what format they use, but it's definitely doing something like this behind the scenes. And then you just see the lovely text outputs, don't we? So please reply with a machine learning poem. Here's my input. My name is Daniel Bourke. Please reply to me with a machine learning poem. Now let's see what Gemma 3, 270 mil, a little baby, language model can generate. Oh, that's actually not too bad. Like, just looking at it without reading it, you're like, okay, a few years ago this was not possible with models, but now it kind of looks like a poem. If we're not even going to read it, we're just going to look at it. But let's read it for fun. Feel free to skip 30 seconds ahead. This might be terrible. Okay, Daniel Bourke, here's a machine learning poem about the power of data. A silent hum, a learning stream. Data whispers a vibrant dream. From pixels bright to patterns deep. Secrets that the data to keep keeps to keep keeps to keep. Each pattern, a story untold of connections brave and bold. Algorithms weave a clever art to shape the world, a beating heart. The world's data, a boundless sea where insights bloom and futures be. With each data point, a new quest to understand, to be, to best, to be best blessed. So let the data guide your way through logic's logic's logic, day and through day. A powerful tool, a guiding hand to build a better, better land. Look, it's not terrible, but it's, it's not great. But hey, that's a 2-set. That's a baby language model. Small language model. Now you'll probably get something different to me, but that's totally another story of why LLMs output non-reducible outputs. Thinking Machines have a great blog post on that. We're getting ahead of ourselves. This is an example one I got the other day. Not the exact same thing. But now let's try the base model. Remember, we haven't done anything to Gemma 3. We've just downloaded it from the internet. Let's try it on one of our random samples. So as you can see, before I run this code cell, um, because this is going to get a random sample, I have specifically put in gibberish text in here, right? So sequences like, um,.jpg. Now why would I do that? Because I want to inference off a large-scale image caption dataset And there could be almost anything in there. So I want to teach my model that it, if it sees these type of texts, to output not food, right? But if it sees food-specific texts, I want to make sure that I'm getting those as just food. So let's do it on a random sample and see what our model outputs. This is the base Gemma 3. So there we go. Nutrition facts label for a beverage. And then you can read through that if you want. Fairly long, but it's just basically a transcribed nutrition panel. And here's the output from our model. Here's the nutrition info for the beverage. Now it basically just regurgitates what we put in, but that's a generic output. But you might be thinking, well, that's terrible. And I'm going to tell you that that's actually really good that our model is outputting. That means it has a baseline understanding of language and the patterns and the tokens that we've given it so we can wield those tokens, mold them into exactly what we want. Now we try it again. These outputs are always going to be fair, fairly generic. There you go. Put in that.jpg. I'm unable to view or process images. That is true. Our model actually knows what it can process. It can do text only. Does it know that or is it just returning tokens? I don't know. That's a philosophical debate. Here we go. A young woman toasts a group of people with red wine. A young woman with red wine is sitting at a table with a group of people. So these are really generic responses, right? But the good news is, is that they're not completely unusable. So what you might have tried in the past is to prompt a model, of course. So why don't we pad our default inputs with some prompt instructions? So, okay, usually we've seen that models can improve if we give them a couple of examples. So here's our prompt instruction. Given, given the following target input text from an image caption, that's what our mostly target data is. Please extract the food and drink items to a list. If there are no food and drink items, return an empty list. So we want this format back. It's not our exact format, but it's close enough. And then we give it an example. Hello, my name is Daniel Bourke. The output, we want it to output nothing because there is no food or drink items in this sentence. Next, we have a plate of rice cakes, salmon, cottage cheese, and small cherry tomatoes with a cup of tea. That's the breakfast my wife made for me the other day, and it was delicious. So thank you, beautiful wife. Food items: rice cakes, salmon, cottage cheese, cherry tomatoes. Okay, you get the point. We give it some examples. Return only the formatted output and nothing else. Target input text, and we're going to format that with a little replace function here with a random example from our dataset. So let's see what that looks like. There's the text input, right? It's going to be gibberish. And if we do it again, maybe we don't get a gibberish one. Let's get an actual food string. McCormack's Food Australia hot sauce. Okay, beautiful. That's what we want. So a hand is holding a bottle of McCormack's food sauce. The label prominently displays the ingredients: tomato puree, distilled vinegar. Okay, good. We've got about 10 or so food items there. That's what we want our model to extract. No drink items. Let's see what it does. Output. Okay. You notice here that it just returned the exact example we gave it. Now, why would it do that? Right. We've instructed it to. Well, the thing is, our model has only 270 million parameters. So it's a— you can think of a parameter as a learning capability of a model. The more parameters generally, the more it has the ability to remember patterns in data. And so because we have 270 million, that sounds like a lot. But in comparison to a model like, say, Gemini, which might have a trillion parameters— again, we don't know this, I'm just guessing— it has a lot more ability to learn. So what if we were to put this exact text into Gemini? What do we get out? Excuse me. I just want to show you what it's like. So our model right now is not doing what we want, right? But where did it go? Oh, here we go. So I'm going to copy this prompt, right? That's what we fed our Gemma 3 270 mil. What if we passed it to Gemini 3? Now, this is a big, big production system, right? You'd hope it gets it right. Okay, just copied and pasted in there. Beautiful. It did exactly what we wanted. It got us the food items in the exact format that we wanted. So can we reproduce this locally? Well, you'll have to stay and check. Now, this is our sample. Hot sauce. Okay. Even our ground truth is not that good. So what if we pass this into our demo? What does it give us? And again, as I said, machine learning errors are good. If this doesn't get it correct, that's okay. We can improve that in a future example. So put that there. This is our fine-tuned version, remember? How does it compare to Gemini? Oh my goodness, look at that. Hot sauce, tomato puree, vinegar, sugar, molasses, maltodextrin, salt, spices, black pepper, cornstarch, brown sugar, thickeners, onion, Garlic smoke flavor. Did Gemini do about the same? Did about the same. Not exactly perfect, but I would be happy with that output from our fine-tuned model. Again, spoiler alert, that's what we're going to be working towards. That's what we are working towards. Now it looks like our LLM doesn't want to do what it wants us to do, and that's going to be the case with many LLMs that you get. And if they're small, even more so the case. But If you have a specific task you want to fine-tune it for, you could do it via prompting and use an API with a bigger model because it has more capacity and it's probably smarter than a smaller model. Or if your task is simple enough, you can just go, well, I've got some data, I'm going to fine-tune a smaller model. So let's do that now. We can fine-tune a model with setting up SFT config. Now I'm going to— I want to open this in Google Chrome, but it's going to open up in Safari, isn't it? That doesn't matter. So sft-config is TRL, Transformers Reinforcement Learning. Now TRL, Transformers Reinforcement Learning, has reinforcement learning algorithms built in, but we're just using supervised fine-tuning, which is inputs, outputs, inputs, outputs. Reinforcement learning, that's a topic for a future video. If you'd like that, let me know below. There's a lot of settings here. I'm not going to go through them all. That is your homework to read through these. So I've put in some default settings. We need a learning rate. That means how much is our model going to adjust given an input? Our model is going to try and produce a default output. And remember, these are all in numeric space by the time our model sees it. The learning rate is going to bump our model in a direction to try and make the model's outputs closer to our ground truth. So I'm not going to, as I said, step through all of these. The one thing you may want to change is the per-device batch size. If you're on Google Colab, maybe try a batch size of 4. Because I'm on the DGX Spark, I'm going to use a batch size of 16. I've got 128 gigabytes of VRAM to play with. And I'm going to use all of those. You betcha. So we set an optimizer here. I did try to use AdamW. You might have seen AdamOptimizer. That's one of my favorites. But for some reason it didn't work. I use AdamW TorchFuse and it does work. Now we have some information here. The config is essentially what do we want the settings for our training to do. Another main one here is where do we want to save our models? So we've got checkpoint models. Actually, I'm going to delete this. Because I want this to be a completely live run. There we go. We've got no checkpoint models over here. So let's set up the config and we run that. So remember config, what you just have to remember, supervised fine-tuning config. These are the settings to train our model. We're going to pass that to SSFT trainer, supervised fine-tuning trainer. Now trainer is from Transformers. It is, um, something that you pass a model, a config, a dataset, and a preprocessor to, and it will take care of all the training behind the scenes. So without any further ado, let's fine-tune our first small large language model, or small language model, on our custom dataset. So if we remember, this is our dataset. We've done the formatting steps above. Dataset, we have a look at that. Train 1,000 samples, test 1,000 samples. And we have our messages right at the end here. Lovely. Okay, that's what's going to be our input to our model. And so let's train it. So this is of course going to happen locally. I can hear the fan on my NVIDIA DGX Spark spinning up. Now, one of the problems being a machine learning engineer and training your own models is you kind of got to wait for this now. Now, 5 minutes, that's pretty good, right? 5 minutes to get a fine-tuned model on a local hardware. The time will probably be similar on Google Colab. The beauty of that is because we're using a small language model and about 1,000 samples. So I'm going to grab a drink of water, I'm going to let this run. I'll speed up the video, um, so that you don't have to watch and sit here and watch this progress bar go up, although it is extremely satisfying. Oh, wonderful, I'm back. See, one of the beautiful things about waiting for a model to train is that you can have a nap if you want. I was dreaming of loss curves going down, and it looks like our mean token accuracy is going Great. So what does this mean? So the number of tokens— this is the number of tokens our model has seen. The entropy— I'm actually not sure of what we want that to be. That should go down. The loss should definitely go down. The training loss is going down. That's great. What's this final one going to be? Okay, so the accuracy hovers around 57%, which is— what does that mean? So given our set of tokens, which, uh, this is again a little bit off topic, but tokens— probably there's probably 150,000 different tokens depending on the vocabulary size of model. That means there's a lot, right? So to get 57% of accuracy on a token space where the output could be literally anything is great. And with the training loss is okay. So our model is probably overfitting. How do we tell that? Let's go down here. It has looked at over 500,000 tokens now. So that's little individual samples of, of data going through our model. So, and but how cool was that? Trained in 5 minutes. We fine-tuned a model in 5 minutes on the DGX Spark. That's, that's amazing. Okay, there we go. So our training loss is trending down, validation loss is. Our model is probably overfitting. How do we tell that? Because training loss is lower than validation loss. But overfitting is probably not bad in our case. Why is that? Well, because my intuition— I haven't confirmed this, there might be a paper out there somewhere on this— but we want a very specific format of data for our LLM. We want it to output the same thing every single time. So it actually doesn't matter too much if our model is overfitting, because to me that says that our model is sticking to our specific data format. Will it? I don't know. We need to test that. So over here we should have some checkpoints, checkpoint models, and we've just saved it here. So we get this. So we could use these checkpoints, but our model is going to default to— or sorry, our trainer is going to default to loading the the best model at the end. So that's these files here. There's model.save_tensors. Now if we go to our wonderful, um, where's Google Gemma 3 270 mil? Here we go. If we go to our wonderful model from Google, we have some files here. Now you'll notice what these files are, are actually the same as what we've saved here. So the beautiful thing is now if we want to remove the checkpoints, we can do that. Just be careful using rm -rf. And we've got rid of that. Now we've got a same file structure here as Google 3 to Gemma 3 270 mil. Okay, can you tell I've said Google and Gemma too much in the last couple of days? So we've got our checkpoint models. Let's first load in our trained model and see how it performs. So you see here we're loading it in from the checkpoint. So we've got loaded model here. Now we're going to create our loaded model pipeline. Before, our pipeline used the default Gemma 3 270 mil. Now it's going to use our loaded model. Now we've got a dataset test. We're going to perform checks on the test set. Always check on the test set. So our model has never seen these samples. Okay, there it is. Trained on the train. We're going to evaluate on the test. How does it do? So this shows a package of Helpful Harvest dried vegetables. The ingredients include a variety of vegetables such as zucchini, so it should pull out that, carrots, mushrooms, onions, capsicum, celery, etc. So what do we get? Output: food or drink. One, it is food or drink. Tags: NP for nutrition panel, IL ingredients list, FI for food item, FP for food packaging. Yeah, food: zucchini, carrots, mushrooms, onions, capsicum, celery. Wonky and Overly abundant produce, milk, eggs, soy, sesame seeds, hazelnuts, pistachios, cashews, almonds. Beautiful. That is exactly the kind of output we want. Is it 100% correct? Well, we'll have to evaluate that further on. But for a gut check, which is what I like to do first, is just look at 100 samples. Koi fish, handheld close-up shot of koi fish eating from a wooden stick. My gosh, is koi fish edible in Japan? I'm not entirely sure. But at least it's outputting that. Please stand guard in the background. This is our model's output. Notice how it adheres to our structure. That's because we fine-tuned it on structured data. Please stand guard in the background with women wearing pink hair and carrying rainbow flags in the foreground. This is gibberish text, right? This is our model's output, our fine-tuned model's output. Exactly the format we want, right? We don't have to prompt it now. We just put in text and we get out this. Trying to make head and tail of the situation. Sheep, merino, lambs, lamb, farm life, Australian mammal. Okay, we could probably pull out lamb from that, but it's probably learned that this is actually not necessarily about food. It's just an image of a farm. Now here we go. Snack package and we get dates, peanuts. Okay, so here's a great way to potentially improve our model going forward. We would try to keep note of these type of samples where the model didn't output the right kind of drinks. And then we might put some training examples into our dataset with more drinks or with no drinks and just make sure our model goes, hey, it might need to see more samples where drinks have come out and foods have come out, if that makes sense. So that's kind of the experimentation loop of machine learning. We have trained our model. It took 5 minutes. Now If we wanted to make it even better, we'd probably look at 100 samples and we'd write down 10 common error points. Say, for example, it classifies some foods as drinks, and then we might create another 500 samples that solve those problems or give examples to our model. And then we rerun the training run and repeat this in a loop. So if we go here, this is our output from GPT-OSS-120B. This is the ground truth. So this is what we want our model to predict ideally, which is dates, peanuts, cacao, peanut butter, sea salt, coconut, tree nuts. Our model got close to that. Not perfect, but it got close to that. Okay, so we're getting there, right? Not bad for a model that's 444 times smaller. So our model has 270 million parameters. GPT-oss-120B, where we got our labels from, has 120 billion parameters. So our model is 444x smaller and performing on par, right? Not too drastically worse, slightly worse, but we can improve that with more data. So next, we want to upload our fine-tuned model to the Hugging Face Hub. Why would we do this? Well, you can make it public, which I'm going to do, or you can make it private. Then you can share it within your organization. Or if you make it public, you can share it with anyone. So the model I'm uploading right now, you're going to be able to download on your computer and run it locally and try it out yourself. So trained all the way here from Brisbane, Australia, you're going to be able to run it wherever you are in the world. Okay, so we're just making sure that it works. There we go. Hello, my name is Daniel. So what was that from? Model path loaded in ourselves. Excuse me. Hello, my name is Daniel. And so is it going to output something? You know why it didn't output this correctly? It's because we haven't formatted it correctly with the prompt that it needs. So we just did a default pipeline. That's what I was talking about. We don't get our default output or our fine-tuned output because we haven't formatted this test input message with the proper chat template. So which is this here content, this formatted, apply the chat template. We get that there. Hello, my name is Daniel Bourke. Same text but with the chat template. Now we run our fine-tuned model. Boom! Ladies and gentlemen, there we go. Exactly what we're looking for. Remember, another thing to remember: LLMs, tokens in, tokens out, training and test set, and always apply the same template that your model was trained on to any sort of inference code. Otherwise, you'll get outputs that aren't aligned with what you want. My school has a vegetarian class, but it's a bit crowded and not as well attended as it should be. See, I don't know where that came from. It's trained on text on the internet. Who knows where it came from? So food and drink. Now this is a sample that someone gave on stream the other day. British breakfast with baked beans, fried eggs, black pudding, sausages, bacon, mushrooms, a cup of tea and toast, fried tomatoes. So we get food or drink, 1. Foods, baked beans, fried eggs, black pudding. You know the drill. Our model is working pretty well. We're going to just instead of trying it out, let's upload it to Hugging Face now. We're running this live. Well, not live. You're watching this in the future. I'm running this live. There we go. Our model has been uploaded to Hugging Face. That was pretty quick. 179 megabytes per second. That's thanks to Hugging Face's ZET layer. Z-E-T. Z-E-T. X-E-T. Excuse me. Okay, so if I refresh this, food extract Gemma 3 270 mil. Uh, there we go. Less than a minute ago and about 1 hour ago. Why didn't it upload my last one? We have checkpoint models. Hmm, maybe it didn't get changed. I was wondering why it did it so fast. Anyway, you can just imagine that I uploaded this one that we trained just then. Let me just pause it there and see what I can do. There we go. What I did was I just manually deleted this and re-uploaded it. So we know for a fact that it is the model that we trained here. And there we go. Less than a minute ago. So our model is now on the Hugging Face Hub. So if we go to this URL and if you want to use it, you can copy that and we'll go back to our code. And say that you can download it here. Beautiful. So we can use our model. We fine-tune it. We've confirmed that it works pretty well. Maybe we want to turn it into a demo because right now we have to write some code and not everyone's a programmer, right? You can't do it. How do we get it in? Not everyone can do it. Sorry, I don't want to say you can't do it because if you're watching this, you probably can do it. So how do we get it into something like this format, which is basically the Hugging Face workflow in a nutshell? Data, model, demo. Data, model, demo. Say it with me. Data, model, demo. That is actually the motto of the Learn Hugging Face course. Data, model, demo. So let's turn it into a demo. To do that, we can use Gradio, which is a great way to just do it with a Python file. And you get a great interface of input, function, output. In our case, the input is going to be text. The function will be our model. The output is also going to be text. So to begin, I want to show you— I'm just going to delete this. I've already done this before. Um, just because we are a cooking show, um, of course I've prepared things earlier. So I want to delete the demos folder. All right, acting dangerously here. We're going to make another folder here, demos food extract. Now we need 3 main things for our demo. We need an app.py file. That's going to be our app, basically says to the demo on Hugging Face Spaces Hey, run this code. We need a requirements.txt file and we need a README. We actually don't even need a README, but I'm going to upload one anyway. If you did go to Hugging Face Spaces, which is where we're going to host our demo spaces, you can create a new space manually. We can go food extract v1 manual, right? You can go extract Foods from text. License will be Apache 2.0, open source, Gradio. Now you can do zero GPU, okay? So you can have, uh, zero GPU run behind your— this may be a pro feature, I don't know, free, maybe. I thought zero GPUs were free as well. Correct me if I'm wrong in the chat, but I'm going to use zero GPU and I'm going to make it public. You could make it private. In my experience, when I deal with clients, if you need a fine-tuned model you can hire me, by the way. I create private Spaces so that they can try it out. But zero GPU is what I'm going to do. I'm not going to do it with this because I'm a coder. I'm going to do it with programmatically. So we're going to create the app.py file in here. We need to basically create our whole application in one folder. We import the dependencies, we make a function, and we add the decorator spaces.gpu. To pred on some input text. We default to our model that we just uploaded. So this is now officially using Mr. D Burke or your username on Hugging Face. Food Extract Gemma 3 270ml fine tune. So we know for a fact it is running our model. Model path, model path, model path. Here we create a pipeline. We add some demo— sorry, description text to our space. Let's go to the actual one. I've got a lot of tabs going on here, but we got this. We got this. Structured food extract with a fine-tuned Gemma 3 270ml. I'm going to refresh this at the risk of it being broken. Oh, lovely. It's still good. Thank God. And this is just explaining what we're doing. But the essential thing is, hey, we're about to upload a demo and then we're going to go run. We can add some examples. There we go. It's not about food or drink. Beautiful. So, for dinner tonight, for dinner tonight, I'd love some steak, potatoes, and aioli sauce after jiu-jitsu training. Okay, if we get this video finished, I can go to jiu-jitsu training this afternoon. What do we get? There we go. Lovely. Food or drink? Steak, potatoes, aioli sauce. That sounds delicious to me. Okay, let's go back here. We get a demo interface. So Gradio, the entry point, um, usually do gr.interface. We pass it in a function which is our pred_on_text. We have inputs, we have outputs. Machine learning in a nutshell: inputs, outputs. Title, description, examples— they come in a list of lists, a little bit confusing, but because ours are just text, we can just make them on the fly, right? These are inputs. People have given these to us from our stream the other day. So writing there, it's going to write that to foodextract over here, demos, app.py. Wonderful. And then we're going to do the same thing except for the README. Just note that Hugging Face has spaces, has some front matter. That's what they call this special text here, front matter. And the setting for these is in the docs. So this is just what your space wants to be called. Emoji, Colar 2, SDK, Python, etc. Um, we're going to call it food_extract_finetune_llm_structured_data_extractor. We're going to go from text to fries, green to blue, Gradio, app.py, pinned false, Apache 2.0. And this is our README. We put in our tags dictionary here so that people can decipher what our model outputs. We'll write that there, uh, requirements.txt. And now we can do the magical part. Remember how we went to that manual interface before? We can do this all programmatically. So if you wanted to train your model and then update it and then retrain it and then update the demo with the updated model, you could write scripts to do this. But we're doing it with one single notebook just to keep things simple, you know. So let's run this code. What is it going to do? Number one, it's going to import the Hugging Face Hub requirements. It's going to define the parameters we'd like to use for the upload, such as the folder we'd like to upload. So this is the folder uploading. Remember, this is just over here. Demos. There we go. We're going to give it a space name. We're going to give it a repo type. It's a space, not a model. It's not a dataset. The SDK we're using is Gradio. We could optionally put our HF token here, but I have put my HF token to behind the scenes so that I don't expose it publicly. If you want to figure out how to do that on your own computer, go to learnhuggingface.com/setup or extras and setup. And this will tell you how to do that on Google Colab. or you could do it in Hugging Face settings, tokens, and that'll teach you how to do it locally. It's not that hard. Um, and if you get stuck, you got LLMs to help you. So create repo. This is gonna be a method up here, which is gonna create a repo for our space. We put in all the information we need there. We get the full repo name, which is gonna be a URL in the case, in the format of username. So Mr. D Burke in my case, uh, model ID or the space name. Of course, it'll be different for your username. And then we go upload our demo folder. So this is gonna upload the folder. And if we go to our Hugging Face demo, it should actually be there now. There we go. It might rebuild because we've reuploaded it. So if we go to files, what do we get? 1 hour ago. Now the reason why this didn't actually update is because I haven't changed any of the files that I updated before. So if we go down here, no files have been modified since the last commit, skipping to prevent empty commit. See what I mean there? So because I haven't changed it from my last commit, Hugging Face smartly behind the scenes doesn't change anything. So these files, when I was— again, this is a machine learning cooking show, right? I have pre-prepared all of these and I haven't changed them since. So maybe we make one change. Um, the tags map to the following items. You can see walkthrough step-by-step, um, code details at— let's put the link to the notebook in the README— fully fine-tune here. Okay, so we've updated that. See, updated the README. Let's now go down here and see if this uploading demos— boom! Okay, did our README update. Files, building on zero. See, it's rebuilding. There we go. Less than a minute ago. Okay, I want to make sure these demos are live. So step-by-step walkthrough. That's all we did. We put that on there. There's a link to that. I've got way too many tabs. DGX Spark, Nuxtify. We don't need all these. Gemini, thank you for your help. Now let's go back. That's, that's what we've done. We've gone from— and how long is this video getting? About an hour on my end. I'm going to edit it down so we don't have all the training in there. I wanted this to be quicker, but But let me know how the pacing was. I'm going to do more live streams on this. But once our demo is uploaded, we can now run it right within our notebook. What? How cool is that? Okay, so this might not be working at the moment because it's rebuilding. But if we go, hello, my name is Daniel Bourke. Right. This is right within our notebook running remotely on Hugging Face's hardware. Boom. And so if we wanted to, we could also share this. How did I do that? Let me go to app. Let me go here. Embed this space, right? We can copy that somewhere else. We can send a link to someone sharing your app, or you could just go to this URL, right? Try it out for yourself. If you do, please let me know in the comments. I'd love to see it. Or if you train your own, please link it in the comments or send me an email. How cool. So we've got two bonuses here. Speeding up our model with batched inference. I'm not going to run this code. I'm just going to show you the spoiler. But right now our model goes one sample in, one sample out, and that takes about a second or so or half a second depending on the hardware. But if we run it with multiple samples at a time, that's what batched inference is. We can get a significant speedup. So if we come down to this plot, So the default inference time on all of our samples was 144 seconds at one at a time with— this is test samples, so 284 of them. But if we run it at batch size 16, we get about a 5x speedup. How cool is that? So we go from 2 samples per second on average for the test dataset on the DGX Spark. By the way, this might be different on different hardware to 10 samples per second. Right? And there are different things that we could do. If you'd like to see those in a video, future videos such as quantization, all that sort of jazz, we can try that out. We might be able to get even more sample throughputs per second because once we've trained our model, it's done. We can keep using it over and over again, right? We can inference all the time. And so 10 samples per second, how many can we do in a day? That's about 820,000 samples per day, right? So you see why I wanted to at the start Save on tokens. So if we do that, 820,000 inference samples times 20, right, we save on inferencing— what's that, 1 point— well, 16 million. 6— we save generating 16 million tokens over a day. That is incredible. And we could even potentially, uh, condense our outputs even more. So if we go down, if you want to do Here's your extension, actually. Figure out how to do precision recall, pure accuracy, F1 score on our model's outputs. You may need some code too. So what you would do for that is you would inference over all the test dataset, collect those outputs, compare them to the original GPT-OSs labels, but you might have to convert them into JSON first. So that's a great takeaway home exercise. But finally, some next steps to finish off. Let's go. Back to the keynote. What's next? Evaluations. So that's what we just said. How does our model compare to a model such as GPT-OSS-120B? I'm going to do some evals on the demo notebook soon, so stay tuned for that. That'll be on learnhuggingface.com. Improve the data sampling. So if our model makes mistakes, could we improve the input data to better train it in the future? Quantization. Right now our model is about 500 megabytes at 270 million parameters. Could we make this smaller with the help of int8 quantization? That could be a great extension. Output compression. So our output is compressed to YAML-like. So remember, instead of JSON, we're outputting just lines of text. Is there a better way to compress and generate even less tokens? So that would speed up our inference time. And then finally, we could test the model on a large unseen sample set. So practice using the model on a large corpus of image captions. I would probably want at least 1,000 to 10,000 randomly selected and then inspect them to find failure cases. And then in the future, we could take those and loop them back up here to improve data sampling. So if you have any questions, please leave a comment below and I'll reply to that. If you'd like me to help you train or fine-tune your own custom model, I'm available for hire. So find my email, that'll be on my website as well. Otherwise, I'll see you in a future video. If you've got any ideas, leave a link, uh, comment below. And, uh, next week I think we're going to do VLMs, so that's Vision Language Models. But no spoilers, I'll see you there.