Massive GPT-4 Upgrades! (And How To Access Them)

So today was the first ever OpenAI dev day, and I feel like OpenAI completely changed the game again. In this video, I'm going to break down all the big announcements, maybe even test a few of the things that they showed off, as well as give you some of my feedback and insights that I took away from the event. So I'm in the process of editing this video right now, and I forgot to mention I'm going to show you how you can actually get access to this right now, even if it's not in your Chat GPT account yet. So stick around to the end for that. They started off the event by talking about how far they've come just in the past year with OpenAI, talking about things like GPT4, the ChatGPT app, with the ability to talk straight to it with your voice, ChatGPT for enterprise, and the fact that there are 2 million developers using their API and 100 million weekly active users of ChatGPT. After they talked about what they'd accomplished so far, they started to break into what's coming next and what we should have access to today. Now, the first big announcement that they made during this Dev day keynote was GPT4 Turbo. And with the announcement of GPT4 Turbo, they talked about how there are six major improvements with it. The first one being context length.

GPT4 supported up to 8k and in some cases up to 32k context length. GPT4 Turbo supports up to 128,000 tokens of context.

If you want to know how much 128,000 tokens of context is, that's essentially 96,000 words of input and output text. This makes it a bigger context window than what you can do with Anthropics, Claude, and as far as I know, the largest context window of any publicly available large language model. So between the amount of words that you input and the amount of words that it outputs, you're going to get just shy of 100,000 words with this one. With something like Anthropic Claude, it's closer to 75,000 words. The next update in GPT4 Turbo is they're giving users more control. Now, keep in mind, as he's explaining these things, he's mostly talking about GPT4 Turbo when using the API. This is a developer conference, remember? So These are the GPT4 Turbo features that people developing with GPT4 Turbo are going to get access to, and we can also use in the playground, which I'll show off in just a couple minutes here. But I am also going to talk about Chat GPT because Chat GPT also came up at this event. There's some exciting news there as well. But first let's see what he has to say about getting more control inside of GPT4 Turbo we have a new.

Feature called JSON mode which ensures that the model will respond with valid JSON. This has been a huge developer request. It'll make calling APIs much easier. The model is also much better at function calling. You can now call many functions at once and it'll do better at following instructions in general. We're also introducing a new feature called reproducible outputs. You can pass a C parameter and it'll make the model return consistent outputs. This of course gives you a higher degree of control over model behavior. Rolls out in beta today a third.

Major upgrade to GPT4 Turbo was that it's going to get better knowledge Knowledge. Not only is the knowledge cutoff window updated all the way through April of 2023, he also announced the that you're going to be able to inject your own information inside of these GPT apps.

So we're launching retrieval in the platform. You can bring knowledge from outside documents or databases into whatever you're building.

The fourth big update to GPT4 Turbo was new modalities. During this portion he announced that Dall E3 GPT4 with vision and their text to Speech technology is all going to be available inside of the GPT4 Turbo API for anybody to use.

GPT4 Turbo can now accept images as inputs via the API, generate captions, classifications and analysis, and with our new Text to speech model, you'll be able to generate incredibly natural sounding audio for text in the API with six preset voices to choose from.

Did you know that Alexander Graham Bell, the eminent inventor, was enchanted by the world of sounds as they talked about the new modalities. He also talked about whisper version 3, which is their speech recognition software that helps translate whatever you're saying into the text that it hears. The fifth major update inside of GPT4 Turbo is customization. During this portion they talked about custom models where you will work closely with the OpenAI team to fine tune a model specific for your business needs. This sounds a little bit more like an enterprise feature that most of us that are just playing around with the OpenAI playground or ChatGPT are probably not going to really need to leverage.

We won't be able to do this with many companies to start. It'll take a lot of work and in the interest of expectations, at least Initially, it won't be cheap, but if you're excited to push things as far as they can currently go, get in touch with us.

And the sixth upgrade to GPT4 Turbo was higher rate limits.

We're doubling the tokens per minute for all of our established GPT4 customers so that it's easier to do more and you'll be able to request changes to further rate limits and quotas directly in your API account settings.

After talking about rate limits and the six big new updates with GPT4 Turbo, they then went and talked about their copyright shield. And this is something that will seem very familiar if you've heard the news from Microsoft, Adobe, Shutterstock. A ton of companies have been doing this, basically saying that if you get into any sort of copyright issues by using their models, they will help cover the legal costs of it. So if you're building an app that uses their API or you're one of their enterprise customers, you know that you can use the GPT4 Turbo and not really risk any legal repercussions, at least not from a copyright standpoint.

There's actually one more developer request that's been even bigger than all of these.

And then to wrap up the conversation around GPT4 Turbo, he talked about the pricing of GPT4 Turbo and how it's getting much, much cheaper.

And GPT4 Turbo better model. It's considerably cheaper than GPT4 by a factor of 3x for prompt tokens and 2x for completion tokens starting today. So the new pricing is $0.01 per thousand prompt tokens and $0.03 per thousand completion tokens. For most customers, that will lead to a blended rate more than 2.75 times cheaper to use GPT4 Turbo than GPT4.

What this basically means is that the people that are out there developing apps using GPT4 Turbo technology, all the copywriting apps, journaling apps, personal assistant apps, all of these kinds of tools that are out there, the cost for them to actually develop them, and the running costs of running these have gotten a lot cheaper. This means a lot of these companies that are out there that are running on OpenAI's APIs are either going to get a lot more profitable or they're going to pass some of the savings onto the users and get a lot less, less expensive. It also means there's probably going to be a lot more apps popping up again. Whenever we see prices come down like this, like we did with GPT3, we saw an explosion of new AI apps come out that were essentially just a Wrapper on the OpenAI API that already existed. This will likely see an explosion in new AI apps, but I think if AI app developers were smart, they would use the new feature called GPTs, which we're going to talk about in just a minute here. They also announced that if you are using GPT 3.5 and that API that's also gotten less expensive as well, hopefully some of these companies pass the costs along to users like they should. And then of course there's chat GPT, which is probably the version of GPT4 that most people watching a video like this are actually using. He just got done explaining all of the cool features that are coming out with GPT4 Turbo, and then he said.

This ChatGPT now uses GPT4 Turbo with all the latest improvements, including the latest knowledge cutoff, which will continue to update. That's all live today.

So from the sound of what Sam Altman just said here, all of those updates that we just heard about to GPT4 Turbo are also going to be inside of chat GPT. Hopefully that includes the much larger context window and the higher rate limit, because those are the things that I'm most excited about from what he just Talked about with GPT4 Turbo. He then went on to talk about something that we kind of knew was coming for a while because we've already been seeing some leaked screenshots as it's been rolling out, and that is that you're no longer going to need to switch between browse and plugins and Dall E3. They're just going to roll them all into one and you can ask any sort of questions or prompt however you need to prompt, and it will decide what the best model is for the question that you ask.

ChatGPT will just know what to use and when you need it.

Personally, I've been refreshing my chatgpt account all day today and the new update hasn't actually rolled out into my account yet, but they did say it'll be available today and from the sounds of it, pretty much all Chat GPT plus users should have it this week, sometime ideally within the next 24 to 48 hours. Now everything they've shared so far is really, really cool. GPT4 is getting cheaper to use with the API. We're getting larger context windows, we're getting higher rate limits, we're getting more customizability, we're getting the ability to have it query documents. But this wasn't even the coolest stuff that they shared at this event today. The next big announcement that they made at this event was called GPTs. And this kind of sounds like something that's going to replace the plugins that we get inside of Chat GPT right now, except they're making them really easy for anybody to build.

You can build a GPT, a customized version of ChatGPT, for almost anything, with instructions, expanded knowledge and actions, and then you can publish it for others to use.

When you go to create one of these GPTs, you create it with custom instructions to sort of tailor how it's going to react when somebody prompts it. You can provide it with expanded knowledge. So this can be any sort of additional data that you want worked into this GPT and then actions, things that it can actually perform once prompted. And the coolest thing about creating these GPTs is anybody can create one using natural language. You get into ChatGPT, you tell it what you want to create, and it walks you through and helps you through the process of creating these GPTs.

So to start, where your GPT will live is on this upper left corner. I'm going to start with clicking on the Zapier AI action. And on the right hand side you can see that's my calendar for today. To start, I can ask what's on my schedule for today. We build GPTs with security in mind. So before it performs any action or share data, it will ask for your permission. So right here I'm going to say Allow GPT is designed to take in your instructions, make a decision on which capability to call to perform that action, and then execute that for you. So you can see right here, it's already connected to my calendar app, pulls into my information, and then I've also prompted it to identify conflicts on my calendar. So you can see right here, it actually was able to identify that. So it looks like I have something coming up. So what if I want to let Sam know that I have to leave early? So right here I say let Sam know I gotta go chasing GPUs. With that. I'm going to swap to my conversation with Sam and then I'm going to say, yes, please run that. Sam. Did you get that?

I did. And then my buddy Rowan Chung, who is actually at this event today, was able to get early access to it because apparently if you were at the event, you also got early access to some of these new features. And he created his own GPT that he called X Optimizer GPT. He explained that he downloaded his Twitter post data from Twitter analytics, configured some custom instructions, and then pasted in the draft of this post and had X Optimizer GPT write this post and here's the screenshot he shared where he configured it, gave it the name X Optimizer GPT gave it a description, become a helpful bot that optimizes posts on X to maximize engagement. He gave it some custom instructions to analyze the data provided and then uploaded a handful of spreadsheets for it to reference for the knowledge. He then gave it the capability of web browsing, Dall E image generation and code interpreter. He then prompted it with Optimize this post and tell me when to post it. He wrote a X post here and then his X Optimizer GPT basically gave him an optimal time, rated it an 8 out of 10, said it's clear and engaging but we can make it more concise and punchy and then suggested a new version of the tweet. And again you can build this all with natural language.

GPT Builder asks me what I want to make and I'm going to say I want to help startup founders through their business ideas and get advice. After the founder has gotten some advice, grill them and why they are not growing faster. It's also going to, let's see, ask me about a name. How do I feel about Startup Mentor? That's good there. And you can see here on on the right in the preview mode that it's already starting to fit. Fill out the GPT where it says what it does. It has some like ideas of additional questions that I could ask. You know what I actually so it just generated a candidate. Of course I could regenerate that or change it, but I sort of like that. So I will say that's great. And you see now that the GPT is being built out a little bit more as we go, what I want this to do, how it can interact with users. I could talk about style here, but what I'm going to say is I am going to upload transcripts of some lectures about startups I have given. Please give advice based off of those. It's going to go figure out how to do that and I would like to show you the Configure tab so you can see some of the things that were built out here as we were going by the builder itself. And you can see that there's capabilities here that I can enable. I could add custom actions. I'm going to upload a file. So here is a lecture that I gave with some startup advice and I'm going to add that here in terms of these questions. This is a Dumb one. The rest of those are reasonable and very much things founders often ask. I'm going to add one more thing to the instructions here, which is be concise and constructive with feedback. So I will say three things to look for when hiring employees at an early stage startup. Now, it's going to look at that document I uploaded. It'll also have, of course, all of the background knowledge of GPT4. That's pretty good. Those are three things that I definitely have said many times.

And then what's even cooler about these GPTs is that they are launching a GPT store. This is like the app store for ChatGPT. It sounds like it's kind of replacing plugins like I mentioned earlier. But the coolest thing about what they're doing with this app store is that you can actually sell your GPTs inside of this app store.

You can list a GPT there and we'll be able to feature the best and the most popular GPT. Of course, we'll make sure that GPTs in the store follow our policies before they're accessible. Revenue sharing is important to us. We're going to pay people who build the most useful and the most used GPTs a portion of our revenue.

Now it'll be really interesting to see what people do with these GPTs. It seems like in order to really thrive as a GPT builder, you're going to need to create some proprietary sort of GPTs. Right? If anybody can go and create their own GPT GPT just using natural language and prompting it and telling it what it wants out of this GPT, what's to stop somebody from seeing a GPT that they really like in the store and just going and prompting chat GPT to build up a new one like the one they just saw. I feel like the only way you're going to be able to create actual revenue selling these GPTs is to have it with custom data inside of it. Now, people will get really creative with custom instructions. And I think the more creative you can get with custom instructions, the more likely you'll be able to create one of these GPTs that's really effective. But I feel like the real amazing GPTs are going to come from the people who upload custom data that have access to custom information and data. They upload it into their GPT and allow people to buy access to this custom data through their GPT. That's where I think it's going to have the most success for people. But I do think in the early days people are going to make some decent money selling access to GPTs that do fairly basic things. They're probably just going to get knocked off and cloned over time. Now, I am bummed out that I don't have access to build my own GPTs and ChatGPT yet. However, they are making the same kind of functionality available with the API and they're calling it Assistance API. And this, if you've got access to the GPT API and Playground, which I'll show you in a minute, you can actually do this right now, today, it seems to be fully rolled out.

The Assistance API includes persistent threads, so they don't have to figure out how to deal with long conversation.

So that persistent threads that he just mentioned, that's essentially like memory. Later on they showed off an example and you can see right here all of the threads. These are the discussions that have happened within one of these AI assistants and it can actually reference previous conversations for context to what you're asking now, basically meaning it's going to remember past conversations to inform future conversations. And once again, they did demo using this new AI assistant functionality inside of OpenAI's playground. They're creating an app called Wanderlust. You can see the instructions here are you're a helpful travel assistant that can write and execute code and has access to a digital map to display information. And then the model is GPT4 1106 Preview 1106, referencing today, the day this was announced, November 6th. This is the November 6th version of GPT4.

And here I'll also go ahead and select some tools, I'll turn on Code Interpreter and Retrieval and Save. If I say, hey, let's go to Paris. All right, that's it. With just a few lines of code, users can now have a very specialized assistant. Right inside the app, you can invoke multiple functions at once for the first time. So here if I carry on and say, hey, what are the top 10 things to do when I have the assistant respond to that again? And here what's interesting is that the assistant knows about functions, including those to annotate the map that you see on the right. And so now all of these pins are dropping in real time here. Next, let's talk about retrieval. And retrieval is about giving our assistant more knowledge beyond these immediate user messages. In fact, I got inspired and I already booked my tickets to Paris. So I'm just going to drag and drop here this PDF while it's uploading. I can just sneak peek at it. Very typical. United flight ticket. And behind the scene here, what's happening is that retrieval is reading these files and boom, the information about this PDF appeared on the screen. This is of course a very tiny PDF, but assistants can parse long form documents from extensive text to intricate product specs, depending on what you're building.

And that was pretty much all of the major announcements that they made at the OpenAI Dev Day of event, which were some pretty dang big exciting announcements. You can play around with the new GPT4 Turbo model using the API inside of OpenAI's playground. Now, this isn't free to use. You do have to have a credit card on file to use it. You can actually see that in my usage cost here. You can see I've spent, you know, a few pennies basically already. But let's jump back over to this Playground tab and you'll notice up in the top there's a little drop down with assistance or chat. Now, chat is sort of the window we were used to where you can enter a system prompt and then come over here, have a CH just like you would with Chat GPT. And if you come over to the model, you can now see that we do have access to GPT4 1106 preview. We can mess with all of these settings here and have a chat with GPT4 Turbo directly inside of the playground using the API. However, we also have.