How To Fine Tune AI Models

Hi, in this video, I'm going to try and demystify what the heck fine-tuned models are, when you would use it, why you would use it. I'm not sure why, why it's not talked about as much, or at least in the content I see, it's not talked about much, but honestly, I found them very easy to create and then very easy to use and get better results. So I'm going to spend a little bit of time telling you what they are, why you should do it. Uh, and then I'm going to dive right into demoing. Here's the steps to create it. Here's what you actually have to do. So let me jump on a screen. We're going to create today a fine-tuned model on OpenAI that has content that is custom to our brand voice, or specifically Jesse Pujji's voice. So why would you fine-tune? Uh, you don't really have to know at the, at the base what fine-tuning is or how it works. You just have to know, I want. I wish ChatGPT 4o or whichever model, I wish it talked more like me. I wish that it had the same general intuition that I have. I wish that when I prompted it on something very specific, it had, it had narrow focus and it understood this concept really, really well, a concept that maybe the broad model doesn't. You also will get a lot higher accuracy again with, because of the narrow focus. It's actually going to look in at the data that you've trained it and it will respond in the way that your data is already set up. It will find patterns in your data. It will make connections in your data, and then it'll make those same new connections in whatever data that you feed it after that. Uh, and then the biggest thing that we find for our team is, is it makes the prompts way easier. We used to have these really long prompts that are, you know, you are a creative writing assistant. You'll get feedback in this way. Here's our brand voice, and then a whole block on the brand voice and a whole block on tone and a block on style. Now those blocks are just part of the model. The model just understands the tone, the style, the voice, the way we think, the way we tell stories. So one little story about why fine-tuning is good. Basically fine-tuning, you take this model that let's say OpenAI created, you take that model. And there's this shell around this model. The model is designed to connect patterns, connect concepts, generate new text, understand patterns in old text. And there's a shell around it, which is, hey, after you've identified all those patterns and figured out those concepts, how should you present? And when OpenAI trains their model, they train it to be very affirmative. They train it to not be harmful and avoid, you know, violence and breaking the law. They also need to make it work for everyone. So it's kind of on average, average, right? Like I was at a, I was at a wedding reception a couple of weeks ago and someone started getting up to tell their speech and man, 15 words in, my wife and I looked at each other like, oh yeah, this is ChatGPT, right? Like it's obvious when it's ChatGPT. It's obvious when it's not a person. So how can we fine tune on our content? To make that less obvious, to make the model talk like us? What's that like outer layer? Let's retrain instead of this affirmative, positive, squishy thing that OpenAI has done. Let's retrain it to be very specific to us. So the steps are pretty easy. I'm going to, I'm literally going to demo them all for you. The steps here are easy. So the fine tuning is you need the data. So in our case, it's a bunch of LinkedIn posts. You need to write what's called the system prompt, which is how, you know, you're telling the model how to think, basically how to think, how to interact with you, what its rules are. This is different than the thing you write when you're talking to ChatGPT. It's kind of one layer deeper than that. We're going to upload all of this data that we gather to OpenAI, and then we'll hit the go button. That's literally all it takes. So let me show you. What we're going to end or where we're going to start. So to begin with, I've actually already got it here. We have a simple— these here are, they're actually newsletters. So these are the last 22 Bootstrap Giants newsletters. There's a lot of text in it. It's a full, you know, blog post style newsletter. And it has a lot of content. This 22 is about enough. Uh, to create the, the, the result that we want. It's a, it's enough content in our voice, uh, that it should represent us well. When we're setting up the data, there are 3 columns that you need. So by the way, pause the video at this point. If you don't have this, this data, go and get it. The rest of the video will be way more useful if you already have LinkedIn posts, if you already have Twitter posts, if you already have your emails or, you know, your sales emails, whatever it is you're trying To fine-tune the model for, go and gather that. I can't help you gather that. You go and get it. When you have it, you want to drop it into a Google Sheets document just like this, and just one, one record per row. So this is one newsletter, another newsletter, another newsletter. And then column B is empty and we'll leave it empty. And column A is our system prompt. This is not a very good one, but it's, it's good enough to demo this, which is You are a world-class newsletter writer. You write in the style of Jesse Pujji. So the way this training works, we're going to train it that, hey, ChatGPT, when you get this system prompt and this user prompt, which we're going to leave blank for now, I want you to output something that looks like this. Hey, when you get this system prompt, it's the same prompt, I want you to output this. I want you to output this. I want you to output this. So we're taking this now, we've got these 22 chances where we've told the model, hey, this is what your output should look like. This is what your output should look like. We wrote these newsletters, you know, they're from Jesse. So they are by definition great examples of Jesse's content. And so you wouldn't, that also means you wouldn't want to put bad content in here. If you wrote a newsletter you weren't proud of, you don't want to, you don't want to put the model in that direction, right? So this should just be the best of the best. You want to have a good amount of content here. This is generally good. You know, if we only did 3 or 4, it wouldn't narrow the model enough. It would still be very much like normal ChatGPT. If we did 1,000 newsletters, often you'll start to see that the model gets confused and doesn't know how to break out because it is too trained. It's too pigeonholed that it doesn't know how to create something new. So you might have to try a couple different levels. Um, I also had someone else on our team suggest, well, wait, what if, what if we included, you know, 20 of our records and then, but, but we don't think we're doing an amazing job. So let's include a handful of additional records from other people or like this aspirational version of what we could create. That makes sense too. You'll kind of get a model that, that gives you a blend of both of those. All right. You've got your data here though. You've got a system prompt and you want to have 3 columns, A, B, C, where column A is the system prompt. Column B is blank. Uh, sometimes people will tell you to try and use the system, the user prompt here. Uh, we've found that it doesn't matter all that much. So we, we leave it blank. It saves us a lot of work. And column C is all your data. I've already got that saved and downloaded. The, the next tool that we're going to use in order to feed this into ChatGPT or into OpenAI rather is this tool from Novel Crafter. And you can go ahead and find that file on your computer, load it in. And what it'll do right away is you'll see in every one of these, we now have the assistant message. So again, this is the result, the training result. 22 of them, and we've got the system message in each one. You could, if you wanted, you could rewrite or expand the system message and then just hit this set for all button and it'll, it'll go towards everyone in the list. When you're done with this, we're going to grab the output in JSONL format. That's what this tool is here for. So that downloads it. And to train, you go to platform.openai.com. You go to dashboard, fine-tuning, and then we're going to create a new model. So I'm going to create it on top of, let's do 4.0, and that JSONL file is here. All right. I'll name it. New model demo video. So there's a couple different settings here. Uh, one is which model do you want this to start from? So, you know, 3.5 is a cheaper model to run than 4.0. At our scale, it doesn't matter at all. So we'd, we'd much rather just have the, the best model here, which is, which is 4.0 right now. We're going to do supervised learning. I don't understand it well enough to tell you exactly what it means, but it is what will work best here. And then you can leave everything else blank. You might, if you, if you start to create a lot of these, you might learn to start adjusting some of these parameters, which will help over or undertrain the model if you have too much or too little data. For your first couple, it's best to just leave them blank and let the data shine through. So after this, that's it. This is the whole setup. You'll hit create. So to recap, all we did was gather the data, put it in a Google Sheet, add a system prompt. So you write one of these, use this tool just to convert it into the right format for ChatGPT, and then kick off the job. This job might take, depending on the amount of data that you gave it, it might take 10 minutes. It might take a couple hours. I can drop into one that I've, that I've already created here. Once you've created it, there's a, there's a button at the bottom right, which, which says playground. So I want to show you the impact of this. So on ChatGPT, I said, And I'm talking to 4.0. Write a punchy LinkedIn post in Jesse's style about the story of Xerox's rise and fall. And it wrote something that's fine. Again, it looks like ChatGPT could have written this because it did. Then we drop into the exact same prompt, but talking to one of our fine-tuned models. And you can already see a big difference. Like, for one, like, look, all caps, like, oh, that's interesting. The Xerox story. Once upon a time, where did this stumble? So like this lessons learned, this is very interesting because you won't see that in the ChatGPT one. It didn't, I didn't tell it to use lessons learned in either. However, almost all of our newsletter content has a lessons learned section. It also will start with some exposition of, you know, if we're introducing a concept, we'll talk about a You know, what do you need to know in order to understand this concept? And then we'll dive into how to optimize over it. So we tell the story here. We go into lessons learned and this is, this is not perfect, but it's much closer to our style. We don't really use these to generate new content. We often use this to help edit and refine content we already have. Now, instead of doing that with something like ChatGPT, we'll drop in, we'll do it with our custom model. That by default knows how to talk in our voice, uh, knows the structure. Um, and again, think of it as this intuition layer where it just understands better what to do. The last piece is, this is great. You could use this playground, that model. Here's a tool called N8N. This is a workflow builder, much like make.com, Zapier, but it's more centered around AI tools. And You can, if you want, you can self-host a version of this chat that can talk to your fine-tuned model. So I'll show you what it looks like. So here's a chat that we've set up internally for ourselves just to play around and look at how easy it is to create a chat with a custom model and a custom system message based off of your fine-tuned model. It's literally 1, 2, 3, 4 blocks in an 8N. This one needs no configuration. It's just the chat window. The AI agent, you tell it, here's what I want, you know, here's how I want you to act. Here's how I want you to write. Here's how I want you to interact with the person that's chatting with you. And then we're using OpenAI. We're using again, one of our fine-tuned models here for this. So this, instead of just connecting this chat to 201 or 40, we're connecting it to the custom model that we created. It has the basic memory node and that's it. Now we have a working chat on top of our fine-tuned model. So one more recap. It's very easy. All you have to do is gather all of your data. In a CSV Google Sheets file, you need to write some sort of prompt that is your instruction for how that data could have been created or should have been created, because this data is output examples that you're giving ChatGPT, things that you want it to help create. You take that, you convert it into a format that OpenAI can understand, you drop it in OpenAI, you change almost no configuration on that screen. And you hit go. That's it. You've got a fine-tuned model that you can integrate with other tools like N8N. Uh, you can chat with it directly from within OpenAI if you want. And this model, we've already found it, you know, Andrew Warner, one of the guys I work with, wow, like you could do that with 2 messages. Uh, I introduced this to another guy on the team who, who works a lot with content and he's already, I've got maybe 6 fine-tuned models. He's already got like 10 fine-tuned models and He's giving me feedback on how to make them better. Uh, so it is a really powerful tool that when you give it good data can, can start to produce really good content with not a lot of work. I would love to help someone work through this. If, if you're curious, if you want to learn more, if you want to talk, um, I've got a lot of people that are writing me asking for time. So I set up this, this small site, aikickstart.io. Please come here, fill out this form, reach out to me. Uh, if I think that I can help you, I will, we'll get in touch. We'll have a call and we'll see what we can do together. Thank you very much.