ChatGPT Fine-Tuning: The Next Big Thing!

So this is huge. Starting today, you can fine tune chatgpt on your own data sets. It's something that people have been waiting for for a while. In this video, we'll see what benefits you can get when you fine tune chatgpt on your own data set. We will also look at the pricing structure and I'll walk you through a code example of how you can fine tune a model on your own dataset. And at the end of the video, we will also explore why it may not be a good option for most of the people. So what exactly do you get with fine tuning? When you fine tune a model, you will get improved steerability, which basically means the model will be able to follow instructions better. You will also get reliable output formatting. So basically the model will stick to the output format in which you want the output to be. And with fine tuning you will be able to customize the tone of the model. A few things that they have highlighted in their blog post. First, when you fine tune a model, you will be able to use shorter prompts to get better performance. Also, compared to GPT3, the GPT 3.5 can handle 4,000 tokens. And from their test it seems like you can reduce the prompt size by up to 90% by fine tuning instructions into the model itself, thus speeding up the API calls and cutting costs. Now the beauty is when you combine fine tuning with other approaches such as prompt engineering, information retrieval and function calling. That's what make these LLMs a lot more powerful. Next, before looking at a fine tuning example, let's look at the pricing. Now they have divided the pricing into two different parts. One is the initial training cost and the second portion is the usage cost. So for training it's going to cost you $.008 per thousand tokens. Now for input prompts is going to cost you $.012 per thousand tokens. And for the output usage it's going to cost you around $0.016 per thousand tokens. So they at the end they have provided a very simplified example. For example, if you have a training job that that's around 100,000 tokens, that will cost you around 2.4 dollars to train the model. Now we are going to come back to this pricing again at the end of the video because there is a lot to unpack here. Now let's look at how exactly you can do the fine tuning. So in the blog post they have provided this schema. So first you need to prepare your data set. And the data set is Supposed to have a system message, then input from the user, then response from the assistant. That's how your data set is supposed to look like. Then you can make an API call to OpenAI to upload the files, then create a training job through an API and then you can reuse the model through an API as well. But we are going to look at a more concrete python example. So OpenAI has a very nice fine tuning guide on their website. So here they give you the reasons of why you would want to fine tune. For example, fine tuning will result in higher quality results than prompting, ability to train on more examples than you can fit in a single prompt, and token savings due to shorter prompts, lower latency. It's a very detailed guide and I would recommend everybody to go over this if you are really trying to understand fine tuning OpenAI models. But let's say if you want to fine tune the model on your own data set, first you need to put this, put your data in a proper format. So for that your data is supposed to have three different fields or roles. So the first one is system message. Then. So that's basically the system message that we are providing to the model, then the user input. So here we are defining the role as user and the content is the input prompt from the user. And then you're going to have response from the assistant. Right. So basically you will arrange all your examples in a simple JSON file and that JSON file is going to be used by the model for training. Now, once your data is in the proper format, you need to upload the JSON file that you just created to OpenAI API. Now, once you're able to create your data set, the next step is going to be to upload your data set to OpenAI. So for that we're going to be using the Python code. This is the OpenAI API package. Then you will provide the file name. So let's assume the data is stored within this my data JSON file and then you provide the purpose. So in this specific case, we want to fine tune our model. Now, after doing this, we need to simply start the fine tuning job. For that we need to provide our OpenAI API key. Then you simply make a call to the fine tune job. This will create a job and it will start fine tuning. Now you need to provide two additional inputs. First is going to be the training file. This is basically the model name that's going to be used for the trained model. And then you need to provide the name of the base model. So if you want to fine tune GPT 3.5 Turbo? You need to provide that name in here. What happens when you fine tune the model and you want to use it? It's as simple as using the chat completion API from OpenAI. So here you simply need to provide the model name that you assign to your fine tune model, then the system message and then the user input and you will get the assistant response or the model response as output. So this is very simple to fine tune GPT 3.5 using the OpenAI API. I'm going to create a more detailed video on how to fine tune this on your own data set. We are going to look at how to structure the data set and then we'll make the API call to get a fine tuned model. So stay tuned for that video. Now let's look at some not so great things which might deter some people from using this service. Okay, so the first one is the safety feature. So your training data is passed through their moderation API and then a GPT4 power moderation system to detect unsafe training data that conflicts with the safety standards of OpenAI. So basically you are limited by the standards of data. Now the second thing to consider is the price itself because the fine tuned GPT 3.5 turbo model is a lot more expensive compared to the vanilla GPT 3.5 turbo model. Just as an example for input tokens it's about 8 times more expensive compared to if you're using the 4k context window GPT 3.5 turbo model. Now for the output tokens it's about 5.3 times more expensive compared to the original 3.5. Now compared to GPT4 it's still less expensive, but probably the performance is not going to be as great as GPT4. So if you're fine tuning a model you really need to consider this substantial increase in price. It's a very interesting development and it will be interesting to see what people can make on top of it. Although we'll still have to see whether it can actually give a performance boost in your applications or not. And is that performance boost worth the substantial increase in price? Let me know what you think in the comments section below. I will be making a more detailed video on how to fine tune this model on your own data set. If you found the content useful, consider liking the video and subscribe to the channel. Thanks for watching and see you in the next one.