ChatGPT 16K Token Context Window is INSANE!

OpenAI just announced new version of ChatGPT as well as GPT4 with a much larger context window and reduced prices. They also announced a few more capabilities in this video. We're going to have a quick look at them now. Here is the list of updates. Now the model supports a new function calling capabilities in the chat completion API. We are going to look at this and discuss this further in a little bit. In this new version the user will have a lot more control in order to steer the behavior of GPT4 as well as ChatGPT models using the system prompt. I think the most unexpected part was this announcement. Now There is a 16,000 tokens version of ChatGPT all as well as the 32,000 token version of GPT4 is available to public Although it's still limited to some folks, not everybody has access to it. With their new update there is a 75% cost reduction on the state of the art embedding models. So if you are looking for information retrieval models, this is going to be one of the best in terms of both accuracy as well as the cost. And there's a 25% cost reduction on input tokens for ChatGPT. They also announced the deprecation of some of the models. Now one other important announcement is related to data privacy. According to them, customer owns output generated from the requests and their API data will not be used for training. Now just keep in mind that this is applicable only to the API usage. If you're doing anything within ChatGPT web interface I think they still say that they will be using that data for training purposes. I might be wrong on that. Now before looking at the new context window and reduce prices, let's look at the function calling capability of these models. Now with this new Update they're introducing two new models. One is GPT4 and the other one is ChatGPT and but they have label is 06.13 so I think that is the release date. If you go to your OpenAI's playground in the completion model you can actually see this new ChatGPT model. So there are three different models. Now the one is this GPT 3.5 Turbo 06.13. Then there is a 16K and 16K 0613. We will talk about these in a little bit. Now with this new update you will be able to give a description in natural language if you want to use certain API and the model will intelligently figure out which API to use as well as what are going to be the most appropriate inputs and outputs for that API So this gives the models a new way of connecting with real world through external tools and APIs. It's very similar to agents when it comes to LangChain or hugging face. Now they have provided a few examples of this new capability. So first one is create chatbots that answer questions by calling external tools, for example, something like GPT plugins, right? So you could have a simple query like this, like email somebody to see if she wants to get coffee next Friday, right? And based on this input prompt, the model actually is able to figure out that it has to make a call. So something like this, so send an email and then it can have the corresponding inputs and outputs using the same function calling ability. You can actually interact with your databases in natural language. For example, you can simply say how are my top 10 customers this month? Right? And then it can make an internal API call, such as if there is a function. So it will just use it by defining the input and output. It's an excellent new addition when it comes to the capability of these large language models. Now here's an example that they have provided. When you make function calling through an API. So the simple prompt could be what's the weather like in Boston right now? And then internally OpenAI decides that it has to make an API call. So it's going to look something like this. So it makes a request. So this is the user request and then based on that it decides which functions to call. So here is the response and then it can actually use this response in the subsequent steps. So here it's using the model response to make your API call. So again it's calling third party service. Now using the response in the second step, if the OpenAI AI actually sends the response back to the model. To summarize it, so here is how it's going to look like. This is the request that is being made in the third step and here is the response that you as a user will get, right? So that's exactly the same as here. It's great to see that as a developer you don't really have to worry about making external API calls. You simply need to provide your prompt in natural language. Now along with these capabilities, they also released a few new models. So the first one that we actually looked at was this 06 13. So it's include an updated and improved model with function calling. And this is the capability that we just talked about. They are also releasing another version of the Same model with 32,000 tokens limit. Now right now you still need to join the waitlist to get access to it. But they're saying that their intent is to remove the weight list entirely for this model. So hopefully in the next few weeks we're going to have access to this model. Now in terms of ChatGPT, so there is a new model that's 06.13. Again it's very similar in functionality to GPT4, but it also has more reliable stability where the system message so you can actually control the behavior of the model using a system prompt. This feature was already available, but it seems like now the instruction following ability of these models have improved. And that brings us to the announcement of 16,000 tokens version where the price is just twice of the existing 4000 token model, both for completion as well as for the output token. Now this is huge because with 16,000 context windows you can actually feed up to 20 pages of text in a single request. This is going to be great for most applications. Now let's look at the pricing changes. So they have actually substantially reduced the prices for embedding models. The price reduction is around 75%. So that is. That comes down to be $0.0001 per thousand tokens. Now this will make a creating custom models a lot more cost efficient. Now in terms of the chat completion model. So they actually have revised the prices for ChatGPT as well. The original ChatGPT, the prices are reduced by 25%. So now the new pricing for input tokens is $0.0015 per token. Right. And this actually roughly equates to 700 pages per dollar, which is great. Now the 16K version is just twice the price both for the input and output tokens, but I think it's still really cost efficient. You know, overall these are some really great announcements. These gives developers much better options and at a reduced cost. It will be exciting to see what people can build up when it comes to the new function calling abilities of these LLMs. I just wanted to create this quick update. More detailed video on exploring the abilities of these model is going to be coming soon. Hope you like this video. If you did, consider liking the video and subscribing to the channel. Thanks for watching and see you in the next one.