How To Fine-Tune A Large Language Model (Step-By-Step)

AI is great at a lot of things. Sounding human is not one of them. Hasta la vista, baby. So when it comes to the promise of making our lives easier, specifically when it comes to writing things like blog posts and articles and scripts or social media posts or whatever, we still need to spend a ton of time reviewing and editing the work before publishing. But that can end right now. In this video, I'm going to walk you through a step by step process on how to make LLMs and sound more like you. Ron. Sorry, were you talking to me? Unless, of course, you aren't human. Are you human, Ron? Anyway, this video is a complete guide to fine tuning your large language models. I'll show you some use cases specifically related to copywriting and my own needs, but the applications really are limitless. So without further ado, join me on this journey.

Before we get too deep, let me explain what fine tuning is. So fine tuning is basically teaching an AI to act a certain way as opposed to know certain things. So imagine you hire a writer for your YouTube channel to help you write scripts. If you give them a Google Drive full of research material, well, they'll know all of the facts that need to go into the project, but they still don't know how to write or talk like you. So fine tuning is the process of giving that writer repeated examples of exactly how you talk. And you give them so much of that that eventually they naturally start sounding like you. That's not creepy at all. You may have also heard of RAG or Retrieval Augmented Generation. It's another way to give additional context and details to a model. However, the difference between RAG and fine tuning is that RAG is more like giving your writer a giant reference manual with details and facts that they can pull from, but it's not going to change the way they actually respond stylistically, they'll just now have the ability to know and retrieve that additional information. So when you fine tune a model, you basically start with one of the big general models that are out there that's trained on a ton of information, and then you give that model specific examples of the sort of outputs that you want from it. That process is then repeated over and over and over again until it learns your tone, your structure, your formatting, your jokes, your habits, and your preferences. So the simplest comparison is basically RAG is like giving AI access to your notebook so it has all the information that it needs. Fine tuning is more like giving it acting lessons so that it actually becomes more like you and starts to respond just like you. So if you need a model to know and understand certain details. You could feed those details to the model through rag. That's basically what you're doing when you upload PDFs or text files or things like that. Inside of ChatGPT. If you want the model to talk like you, rag's not going to quite cut it. You need to actually fine tune the model to achieve this.

Here's a model inside of a platform called Nebbys that I've already fine tuned. You can see this one's called MW YouTube and it's built on top of Llama 3.37 DB instruct. With this particular model I trained it on all of my YouTube transcripts, like probably a hundred hours worth of training data transcripts from my videos. The idea being that I can tell it to go write a script on any topic I want in the style of me and it will write a script that sounds the way I would talk on a video. If I go into the playground here, this is my fine tuned model and I can give it a system prompt and I can chat with it just like I would chat GPT. And I am going to get to a step by step breakdown in just a minute. I just want to finish showing off this example that I already did and then we'll do one from scratch. In my system prompt I'm going to go ahead and say you are Matt Wolf, use correct punctuation and markdown and I can give it a prompt like create a detailed outline with 6 to 8H2 sections for a 10 minute YouTube video on Nvidia's Dominance in AI label with hashtag, hashtag headers and 3 to 5 bullets per section. I just want to give it some extra details because like with any large language model, the more context you can get it, the better the output you're going to get. Now if I submit this, theoretically it's going to write a script for me that sounds similar to how I would write and that's exactly what it did here. But I can also compare it to what it would have looked like if I didn't use a fine tuned model. So if I go to compare here and then I switch this second model on the right to let's go llama 3.37 B instruct. That is the same model that I previously fine tuned here. Let's go ahead and clear our chat on the left and I'll give it a prompt, write an outro and closing to a video that wraps up why Nvidia is so dominant in AI. So let's go ahead and give it that and let it write an outro for us. And we can see side by side comparisons of how it would write that outro. Now, the unfinetuned model did a little bit better with formatting, but that actually makes sense because I gave it just a ton of transcripts from YouTube videos. And if you've ever looked at the transcripts from YouTube videos, there's usually not much formatting or punctuation, so it kind of over fit for that. But if we look at the normal llama response as we conclude our explanation of Nvidia's dominance in the AI landscape, it's clear that their success can be attributed to a combination of strategic innovation, forward thinking investments, and relentless pursuit of technological advancement. And then the version that's trained to sound kind of like me. So there you have it. That's why Nvidia is so dominant in AI right now. They've been preparing this moment for over a decade. They're the leader in the hardware that's required to train AI models. They're the leader in the hardware that's required to run AI inference, and they're even building their own AI models. Now, I'm not going to read the whole thing, but it's even trained on some of my old calls to action that I used to put in my videos. If you've watched a lot of my videos, you know that this reads like me. Hopefully you found this video helpful. Hopefully you feel more looped in on the whole Nvidia ecosystem and why everybody's making such a big deal about Invideo. We could even see here. It went on to say, if you like stuff like this and you want to stay looped in in the AI world and the latest AI tools, get the TLDR of everything that's going on in the world of AI, Check out future Tools IO. Thank you so much for tuning in. I really, really appreciate you. And thanks again to Wirestock for sponsoring this video. I'll see you guys in the next video. Bye bye. Now, it's funny because Wirestock hasn't sponsored the channel in like a couple years now. But that's in the training data. So it wrote an output that sounds like what I would have written. Now granted the formatting sucks, but that's my fault. On the way I trained in the data, I didn't clean up the data, I just gave it sort of unformatted transcripts. So it gives me back unformatted transcripts. Garbage in, garbage out. So real quickly, here's how I trained that one and then we'll start a new training from scratch. I found this random website called download YouTube transcripts.com it looks pretty quickly made and slapped together, but for seven bucks I was able to download all of the transcripts from my YouTube channel in one click. It exported all of those transcripts as just one giant text file. You can also export them as individual text files, but for our use case, I just wanted it all in one huge massive text file. I then took that giant text file that it generated and I uploaded it straight into Chat GPT and I gave it this prompt. I'm not going to read the whole thing, but feel free to pause to get the exact prompt. But the idea was this giant transcript document was not properly formatted to be used as training data. So it needs to be a JSON L file, and it needs to be formatted in a certain way to show like a user input and then what the expected output is so that the model can train on this input deserves this output. And so in order to do that, I had Chat GPT do it for me. I asked it to format the entire document in the proper JSON L format required by Nebby. It should give me two files at the end, one with the training data set with 10% held back and the validation set with with the additional 10% of prompts and transcripts. I'll explain that when we do our own version here in a minute. But the long and short of it was I gave it this entire transcript text file, told it to convert it to a JSON L file and format it in a way that Nebius would like. It took 14 and a half minutes to do that for me. But as you can see, it gave me two files to download, the training JSON L file and the validation JSON L file, and then went to Nebius. Fine tuning Create a job. Uploaded my training data set here and my valid validation data set here, and ran the training model. I can also download the direct weights if I wanted here. So if I downloaded the files in our epoch 11 here, this is going to be the most advanced version, the most trained up version of the model. So I can download the files that I need and I can use them offline if I want in a tool like LM Studio or Olama or something like that. All right, so I show you what it was capable of and then I probably confused you by showing you what I just did to get the YouTube one. So now let me try to unconfuse you by doing it. Step by step with you, huh? In this next example, I want to train it up on all of my tweets that I've ever written. I want to be able to ask it to write a tweet in my style on any topic I want and have it write a tweet that reads just like all of the other tweets I've ever made. Before we get back to the video, I want to show you how to actually make money on YouTube. I run my YouTube channel sort of like a business because well, it is my full time job and if I Learned anything from Econ 101 it's that, well, businesses need revenue. I manage all of my brand partnerships, events, newsletter posts, basically the engine that powers this YouTube car. All in Notion. Now I need to blur a lot of stuff on the screen because I get access to quite a bit of confidential information that brands have trusted me with. So yeah, but I just wanted to show you that I am using Notion and Notion just released their own in platform AI Agent that's going to make running any project or business so much easier. Instead of just chatting or suggesting ideas like a lot of other AI tools, Notions Agent can actually complete multi step tasks end to end. So I can say something like summarize my past three meeting notes into a checklist and send me a reminder on Monday and it'll create the whole operation inside Notion without me having to learn how to set up an automation or create a new board. Now Notion is not only so much easier to set up your whole project in, but also easier to manage your ongoing tasks and priorities. And since this is all powered by AI, Notion's Agent can learn your style, remember your preferences, and even be customized with its own name and personality. I think I'll call mine Steve. So whatever you're working on, from content creation to business management or somewhere in between, Notion can help you set it up and keep it running smoothly. Try it out and let Notion Agent do your work for you at the link in the description. And thank you so much to Notion for sponsoring this portion of today's video.

So step one in order to do that is we need to download all of our Twitter or X data. So if I'm logged into my X account here and I go to more and I click on Settings and Privacy under your account there's an option for download an archive of your data. So I'm going to select that, it's going to ask me for my password and there's a button here to download the archive. Now for some Weird reason. I don't know why X does this. It makes you wait 24 hours before you actually get the download. So if you press the download button now, 24 hours from now, there will be an email in your inbox saying your download's ready. It's stupid. I don't know why they do it that way. But you can't just get your data immediately. It needs 24 hours to process. Luckily, I clicked this button 24 hours ago and I can click into Download archive and we can see that it was generated and I can now download my archive here. It's large, it's 13 gigabytes. But we're just going to pull out the tweet data from it. Here's the zip file that I just downloaded. I'm just going to go ahead and unzip it real quick. Once it's unzipped, we'll have two folders. If I go into the data folder here, there is a file called Tweets j S and as we can see, it's basically the second largest file in here. If I open this with let's open it in a text editor, we can actually see. This is all of my tweets. This is actually showing a retweet. My AI news roundup. Now it looks like the data that it pulled in is just pulling in the first, I don't know, 140 to 280 characters. It's not pulling in the full long tweets if you did a long tweet, which is unfortunate, but hopefully the data is enough to use to train close enough.

So I'm going to grab this tweets JS file here and I'm just going to move it to this folder so it's easy for me to find. We'll go ahead and create a brand new chat inside of Chat GPT and here's the prompt I'm going to give it. Attached is all my tweets from my X account in a JS file. Create a script that converts this into the proper JSON L formatting for fine tuning a model. It should look at the tweet, think of a prompt that would have generated that tweet. For example, create a tweet about the Perplexity Comet browser in the style of Matt Wolf. And then it should format the entire document into the proper JSON L format required by Nebius. It should give me two files at the end. One with the training data set JSON L with 10% held back and the validation set with the additional 10% of prompts. And actually I need to change this to tweets. So this is the prompt I'm giving it, along with the tweet JS file that we just looked at. We'll submit this and it might take 15 minutes to run again, but it's going to go and make those files and prepare them for uploading into Nebby here in a second. If you're wondering why I split it off into two files where the training data set is 90% and then we have a validation data set of 10%. Well, basically the model is going to train on the training Data set, the 90% that we gave it, and then that final 10% is sort of like for the AI model to double check its work. So basically, if the model was to give it an input that our validation data has a similar output for, how close did the model get to what our validation model looks like? I don't know if I'm being clear here, but it's basically training on the 90%. And then the 10% is so it can double check its work and see if it gets a close response response to what's in the validation data set. Hopefully that computes. And this time, for whatever reason, it only took a minute and a half. But we can see it gave us a couple files here, the training JSON l file and the validation JSON l file. So let's go ahead and download both of these. We can actually see what ChatGPT just did for us. It reads the tweets JS Export, parses the tweets, filters out retweets and tiny junk tweets, generates a synthetic prompt for each tweet in your style, splits it into 80%, train 10%, validation 10% held out. It sort of misunderstood what I was trying to do there. I wanted a 9010, not an 8010 10. But this should still work just fine. I doubt that'll have much of an impact. And then it wrote the instruction style nebus JSON L. So if we take a look at the files it just generated, let's go ahead and open this just with a text editor real quick. And we can see here's the cleaned up file. If we look at our JSON l file, there's prompts and there's completions that it created for us around every single tweet and it's all properly formatted.

So now if I head over to this Nebius platform here, you can find it@tokenfactory.nebius.com and then if we click into fine tuning here. This is where we can fine tune our models. Now this isn't free to do. It does cost some money for inference to use nebbyus to do the fine tuning. The pricing depends a lot on the model you're fine tuning and the amount of data that you're feeding it. But this is not sponsored by Nebius. It is literally just the easiest way I've found to do this process. So inside the fine tuning page I'm going to click Create job. Under training data set, we'll upload the MW tweets train j s JSON l file. For our validation data set, we'll upload our tweets dash val and now we can click continue now for training type, I'm going to do Lora fine tuning. It is quicker and cheaper to run for our model. We have a few options to select from, but in order to be able to just use the model directly inside of Nebbyus here you want to click one of these models that has one click deployment. Otherwise you're going to need to download your weights and like run it locally or run it in some other cloud service. And I just want to keep it all in one platform to keep it easy. The first example I showed you, I used Llama 3.37 DB instruct for this particular example because it's writing tweets like smaller text segments instead of long scripts. I'm actually going to use the 8B instruct the smaller model. If you're trying to decide which model to use as your base model to fine tune on top of, here's something that could help. Here's an example of an 8B model, a 32B model and a 70B model. This is the amount of parameters inside of the data set. The larger the model, the more expensive and the longer it's going to take to fine tune it. The smaller the model, the faster and the cheaper it's going to be to fine tune. But also you might lose a little bit in accuracy and it won't be very good at long content. So llama 38B instruct, the one we're using for this demo, is great for things like tweets, hooks and intros. The style accuracy will be pretty dang good, but it won't be long form coherent. It's gonna struggle with really long prompts, it's going to struggle with really long outputs. But as we can see, cost to train and deployment costs are low. And then if we jump to the 7db model, I'm just gonna skip over Quinn 32B because that's not one of our models that is instantly deployable. You can see this is better for documentary style storytelling. So you're gonna get more style accuracy, it's gonna get more closely trained on your style than had you used a small model. And it's going to have that long form coherence. So you can give it longer prompts on input and it's going to be able to output longer responses. However, more expensive to train, more expensive to deploy, and probably overkill for most use cases unless you're doing YouTube scripts or very long blog posts. All that being said, let's jump back to Nebius here. And again we're going to stick with our llama 3.18 billion parameter model here. The model you choose will affect the pricing. So this one is only 40 cents per million tokens. If I was to switch this to 7dB, you can see now it's $2.8 per million tokens. So we want cheap and fast. So for training type we're using Lora fine tuning. For batch size, I'm just going to leave it at 8. For learning rate multiplier, I'm going to put 0.0001. Number of epochs is 11, warm up ratio 0.03, weight decay 0.001, max gradient normal one and enable packing. You don't really need to worry too much about what these do. The number of epochs is how many times it's going to sort of pass through the training data. So the bigger amount of epochs probably the better the output is going to be. But there is a point of diminishing returns where continuing to train it more and more and more just further overfits it to a point where you're not going to get good outputs anymore. In my testing, setting it at around 11 seems to work fine. The rest of these numbers, I saw on a blog post somewhere that that was what was recommended. I've stuck with those. They've worked fine. For Lora rank, I'm setting it to 16 Laura Alpha 32 Laura dropout 05 and then for the suffix for output model name, I'm setting it to Matt's tweets so I can find it easier. I'm going to click Continue here. It's going to ask for some weights and biases. API keys. I'm not using them. They're not necessary. We'll click Create job and now we can see the status is it is running this job here. Now this process could take anywhere from 10 minutes to to 4 hours depending on which model you're fine tuning. On top of how big your training data is and various other factors, the amount of epochs you set, things like that will change the amount of time I've had a model fine tune take 45 minutes. I've also had a model fine tune take 5 minutes. Again, a lot of factors could affect the timing.

Okay, and that model is done running. We started the model training at 5:19pm and it finished at 5:26. So it took about seven minutes to train that model. If I click on deploy and download checkpoints, I can download the strongest weight here and use it wherever I want. And we know the training did its job. When the training loss is decreasing with each new epoch here, which we can see it did, and the validation loss is increasing with each new epoch. So perfect. As far as cost goes for me to fine tune that model at $0.40 per million tokens, it cost me $2.56. I have a lot of tweets. It fed it 6 million 403 tokens. For comparison though, when I trained my YouTube videos on llama 7db, it cost me 75 bucks. What? Because I was using 7db, it cost me $2.80 per million tokens and I fed it almost 27 million tokens. So yeah, it was a little bit more expensive to fine tune it on my YouTube videos. But again, if you want a model that sounds just like you, this is a one time cost.

Now let's go test our Twitter model. So I'm going to jump back to the fine tuning page where we just fine tuned this thing here. Click on deploy and download checkpoints and then under epoch 11 here we're going to deploy this model. This is going to be the best trained model for Lora adapter. I'm going to call this MW View tweets and I'm just going to say trained on Matt Wolf's tweets, click start deployment and it's going to fire up an inference server for this model. Deployment started. Let's go to custom model. And we can see under my model endpoints here, I've got my original llama 3.370B model and my new model here which is 3 18B and we can see it's running the MW tweets. Lora on top of it now. It's still deploying, but we can see it's going to cost basically nothing to use it once it's deployed. I can click on go to Play. All right, so once I'm in the playground here and I have my model loaded in, I can ask it to write a tweet in the style of Matt Wolf about how AI and VR about to intersect in a big way. The tweet should be at least 240 characters. And this was the tweet it wrote for me. AI and VR are about to intersect in a big way. I just got back from CES and there was a ton of head tracking VR devices on the show floor. Meta just open sourced their fair share of VR head tracking tech. We're seeing a rush of new VR headsets coming in the coming months. So I mean it wrote just like me and even got from the training data that I've been to ces, I think it got that from the training data at least. Now let's compare that to if I just used a regular model, I can click on this compare button here and then I can change this model up here to a different model. Unfortunately, the Metalama 3.18B instruct model that I've been testing with, it's not working in Nebbyist. So if I use this model I'd give it the same exact. We can see it says failed to get model response. So something is wrong with Nebius's connection to llama 3.18B. But if I switch it over to let's go 3.37 decibels. So it's a larger model, bigger, more parameters in the training set, but not totally apples to apples. But I still think my version is going to be better. So if I give it this same prompt here, we can see this model is working. And here's the tweet that just a normal non fine tuned model wrote Wake up call. AI and VR are on a collision course and it's about to change everything. Imagine neural networks generating photorealistic worlds in real time, adaptive difficulty that adjusts to your skills, and avatars that learn your behavior. The future of immersive tech is arriving fast. Hashtag AI VR. I never use hashtags on Twitter, like ever. So none of my tweets on this fine tuned model are ever going to need to have hashtags in them. Yet for whatever reason, AI likes to throw hashtags in anything it thinks is a tweet. This also doesn't read like me and is just just not looking like a tweet I would have wrote. So let's go ahead and clear both chats. Again, write a tweet in the style of Matt Wolf about why tacos are the best food ever. It should be at least 240 characters. Let's give this to both models, My fine tuned model and the Llama 70B model. I just wrote a simple tweet. Why are tacos the best food ever? I mean, that's probably something I would write. On the other hand, 70B instruct. Tacos, folks, are the apotheosis of culinary perfection. A harmonious union of textures, flavors, and temperatures, all converging in a delicate, crispy shell. The crunch of fresh lettuce, the tang of salsa, the richness of carne asada. It's a gastronomic symphony. Hashtag, tacos for life. I mean, honestly, one of these tweets looks more like a tweet that I would have written. Now, there is one quick sort of gotcha that I skipped over because I actually did it while I wasn't recording, but I was asking it to write tweets for me. And every single tweet it wrote actually put an reply in it. So every single one started with, like, Owen Chung at Matthew Berman at Robert Scoble. They all started with, like, tagging somebody. Obviously, I didn't want every tweet it ever writes for me to tag other people. And what happened was when I asked ChatGPT to rewrite all my tweets using proper JSON l that can be accepted by Nevius, well, it scraped all of my tweets, but it also scraped all of my replies to other people's tweets. And I have way more replies to other people's tweets than I have my own tweets. So it definitely overfit on me replying to other people on Twitter. So every single tweet it generated had sort of tagged somebody in it. So what I did was I went to ChatGPT, ran that same prompt again, where I asked it to reform it to Jason L for me. But this time I said remove any replies and only add tweets that I wrote on my main timeline to the data set. It cleaned it up, and now I have a data set that I fine tuned that doesn't try to reply everybody all the time.

And that's how you do it. That's how you fine tune a model with the easiest method I've found. Now, people may claim that methods like rag will get it to talk more like you, but that's often not the case. You can kind of use rag to give it some examples and say, kind of write it like this, but then the AI models are still gonna sort of write it how they wanna write it. But if you write blog posts or make YouTube videos, or post on social media often, or really need to do any form of marketing you where where the outputs need to know your voice or your brand's writing style, fine tuning might be a good option for you. Now obviously it's a bit more technical than just firing up something like ChatGPT, but if you want to know how like power users of AI are using some of these models to make content that's practically indistinguishable from what they'd make without AI, well that's what fine tuning does for them. Eventually I'm sure ChatGPT and Claude and Gemini and all those platforms forms are going to have fine tuning just sort of built in and it's going to make it so much easier to make these models sound like you. But for right now, in 2025, this is the simplest process I've come across. If an even easier method comes along and makes it super simple to make AI sound like you, I'll probably make another video and update how to do it with the even easier way. But for now, this is what we've got. Hopefully you found this helpful and understand a little bit more about fine tuning and how to dial in the voice of your AI model. I've made it my full time job to keep up with the latest AI news and test all of the latest tools. So if you want to stay looped in on the latest advancements as well as get deep dive tutorials on how to actually use the tech, make sure you like this video and subscribe to this channel and I'll make sure more videos like this keep showing up in your YouTube feed. So thank you so much for nerding out with me on this one. I really appreciate you you and hopefully I'll see you in the next one. Bye bye. Thank you so much for nerding out with me today. If you like videos like this, make sure to give it a thumbs up and subscribe to this channel. I'll make sure more videos like this show up in your YouTube feed. And if you haven't already, check out FutureTools IE where I share all the coolest AI tools and all the latest AI news. And there's an awesome free newsletter. Thanks again, really appreciate you. See you in the next one.