ChatGPT’s Amazing New Model Feels Human (and it's Free)

So today, May 13th marks the beginning of a really interesting couple of weeks. Today we got some new announcements out of OpenAI, which came right before Google's making announcements tomorrow. For whatever reason, OpenAI loves to do that. They love to pick the timing of their events to try to overshadow Google as much as possible. And today's OpenAI event seems to be exactly what they were trying to do. I'm actually out here in Mountain View, California right now because I'm going to be at the Google Keynote tomorrow, but wanted to make sure that I got a video out as quickly as possible about what OpenAI just announced with their new model. Instead of calling this new model GPT 4.5 or GPT 5, they went with GPT 4O. And as it turns out, that mysterious chatbot that we've been playing with called GPT2 chatbot and I, I'm a good GPT2 chatbot, and I'm also a good GPT2 chatbot. We're all actually us getting the opportunity to test this new GPT 4.0. Now, while the model is improved a little bit, the biggest features about this model are the lower latency when having voice conversations. It seemingly has better multimodal capabilities. And instead of me breaking it all down, let's watch little bits of the keynote and sort of talk through some of these new announcements that OpenAI just made.

The big news today is that we are launching our new flagship model and we are calling it GPT4O. The special thing about GPT4O is that it brings GPT4 level intelligence to everyone, including our free users.

So this is actually a big piece of news here. Up until now, if you were on the free version of ChatGPT, you were using GPT 3.5. Well, this new state of the art model, GPT 4O, is now gonna be available both for plus and free users, which means anybody can use this new state of the art model completely for free. But if you're a plus member, you just get to use it a lot more.

A very important part of our mission is to be able to make our advanced AI tools available to everyone for free. And today we're also bringing the desktop app to ChatGPT.

So this is another big update that they made during this keynote is that we now get a desktop app. Now in their demos that they show here in this video, they're only showing the desktop app being used on Mac. They didn't really say if it was Mac only or if it was going to be Mac. And PC. My guess is that it's probably going to come out for both platforms. In the demo, they only showed Mac and they didn't really speak to that piece of it.

As you can see, it's easy, it's simple, it integrates very, very easily in your workflow. GPT4O provides GPT4 level intelligence, but it is much faster and, and it improves on its capabilities across text, vision and audio. So we're very, very excited to bring GPT4O to all of our free users out there. And for the paid users, they will continue to have up to five times the capacity limits of our free users.

So I'm going to go back real quick on the screen here and these are all the things now up here on the Top, right, that ChatGPT free, free users are going to get access to. You're going to be able to access the GPT Store and all the custom GPTs Vision, the Browse model, which allows you to search the Internet using ChatGPT, the memory functions and advanced data analysis, which used to be called code interpreter. Free ChatGPT members get that GPT4O is.

Not only available in ChatGPT, we're also bringing it to the API.

So basically, when they're saying they're also bringing it to the API, they mean that developers can actually work with this new model as well. However, that means that you can play with this model directly inside of the OpenAI playground as well. If you go over to platform.OpenAI.com playground and click on Chat in the left sidebar, you can see on this little dropdown, we actually have GPT4O over here. And one thing that's interesting to note is inside of the playground, we actually have the ability to upload images or link to images. I don't believe that was ever available inside of OpenAI's playground before. This is the first time I'm noticing it. Maybe it was there before, but I'm pretty sure they just added the ability to upload images here. With this new rollout of GPT 4.0.

Our developers can start building today with GPT 4.0 and making amazing AI applications, deploying them at scale. 4.0 is available at 2x faster, 50% cheaper and 5 times higher rate limits compared to GPT 4 Turbo. So we'll do some live demos.

I love that they're doing these demos live as well, because I think it's sort of a little bit of a message to Google. When Google put out their launch video for Gemini, they showed off all of these really cool capabilities and a nice prerecorded, polished video. And it turns out that a lot of what they were showing was not actually real time. They made it look like it was doing these things a lot faster than it actually was capable of doing them. OpenAI is saying, hey, check this out. We, we're gonna show it to you live in real time, no camera trickery. This is how it works. They also kind of made that little nudge in their blog post as well. About GPT4O. All videos on this page are at 1x real time. That note there was very intentional. It's a message to people who are comparing it to Google's launch guaranteed. That's why that's there.

Hi, I'm Barrett.

Hey, I'm Mark. So one of the key capabilities we're really excited to share with you today is real time conversational speech. And if you see there's this little icon on the bot right of the ChatGPT app and this will open up GPT4O's audio capabilities. Hey, ChatGPT, I'm Mark, how are you?

Oh, Mark, I'm doing great, thanks for asking. How about you?

Now, although they've been doing a lot of talking about GPT4O and this new model and how it's going to be in Chat GPT for free and it's going to be in the API, I think they really wanted to do this keynote today to show off this voice feature. This is very reminiscent of the movie her, where he basically has a chatbot companion that he talks back and forth with, who's voiced by Scarlett Johansson. And if you listen to this voice, a very similar sounding voice, just what they're showing off here makes me think that we're about to see an explosion of a high girlfriend apps because the conversation feels so much more realistic. And as they're having this conversation, one thing to note is the latency difference. You used to ask a question, there'd be a little pause for five or six seconds and then you get the response. Now the responses are a little bit more real time. It's feeling a little bit more like a real human to human conversation.

Hey, so I'm on stage right now, I'm doing a live demo and frankly I'm feeling a little bit nervous. Can you help me calm my nerves a little bit?

Oh, you're doing a live demo right now. That's awesome. Just take a deep breath and remember you're the expert here.

I like that suggestion. Let me try a couple deep breaths. Can you Give me feedback on my breaths. Okay, here I go.

Whoa, slow a bit there, Mark. You're not a vacuum cleaner. Breathe in for a count of four.

That's pretty crazy. That it actually could listen to your breathing and tell you you're breathing too hard and shows us that it is doing a pretty decent job of understanding some of the context of the conversation.

Going to breathe in deeply and then.

Breathe in and then for four, and then exhale slowly.

Okay, I'll try again. Breathing in and breathe out.

That's it. How do you feel?

I feel a lot better. Thank you so much.

They were having a little bit of Internet issues, probably because there's so many people all trying to connect to the same WI fi. The demo definitely was live and you do get a little bit of an extra delay that you probably wouldn't have normally just because of the connection issues.

You've used our voice mode experience before. You'll notice a couple key differences. First, you know you can now interrupt the model. You don't have to wait for it to finish your turn before you can start speaking. And you know you can just button whenever you want. Second, the model is real time responsive. That means that you don't have this awkward 2 to 3 second lag before you wait for the model to give a response. And finally, the model picks up on emotion. Right when I was breathing super hard there, it could tell and it knew. Hey, you might want to calm down a little bit. You're kind of running a little bit too fast. You know, it really has that capability across the board to perceive your emotion. Not only that, though, the model is able to generate voice in a variety of different emotive styles, and it really has a wide dynamic range. So my friend Barrett here, he's been having trouble sleeping lately, and I want you to tell him a bedtime story about robots and love.

Oh, a bedtime story about robots and love. I got you covered. Gather round, Barrett. Once upon a time, in a world not too different from ours, there was a robot named Bite. Was a curious robot, always exploring.

Started this story, but I want a little bit more emotion in your voice, a little bit more drama.

Got it. Let's add some drama. Once upon a time, in a world not too different from ours, there was a robot named.

No, no.

ChatGPT. I really want maximal emotion, like maximal expressiveness. Much more than you were doing before.

Understood. Let's amplify the drama. Once upon a time, in a world not too different from ours, there was a robot named Byte Bite.

So we can do this in a robo robotic voice now.

Initiating DRAMATIC robotic voice. Once upon a time, in a world not too different from ours, there was a robot named Byte. Bite was a curious robot.

So that's pretty cool. It can actually change the way it speaks, make it more dramatic, make it sound like a robot that could come in handy for things like bedtime stories and maybe meditation apps to get that real calming voice, things like that it seems like it'll be really, really good at. Now, the other thing they mentioned is that that latency has been sort of fixed where there's not as much of a gap when you talk and when you get the response. And it does seem like that gap has been closed, but it also kind of feels like maybe they're putting some filler words in there to close that gap. So you ask the question and it goes. Let me think about that for a second. Okay, here's what I think. And it will sort of add filler and then give the response. I believe that's what it's doing. They didn't really speed up the latency on there. They just made it start speaking before it was sort of ready to start speaking, if that makes sense.

Can you end the story but do it in a singing voice?

And so Byte found another robot friend and they live circuitly ever after.

Thank you so much.

I definitely feel a lot more relaxed.

We also want to show you the vision capabilities as the model can also see the whole world around us.

Yeah, absolutely. As you can see, you can chat with ChatGPT in real time, but you can also interact with it with video as well.

Hey, ChatGPT, hello there. How's it going?

It's going really well today. I'd really like your help solving a math problem.

I'm all ears. What math problem can I help you tackle today?

So I'm going to write down a linear equation on a sheet of paper and I'll show you, and then I'd love your help working me through it. But importantly, don't tell me the solution. Just help give me hints along the way.

Okay, I see it.

No, I didn't show you yet. Just give me help along the way. One second.

Whoops, I got too excited. I'm ready when you are.

ChatGPT, what equation did I write down there?

Ah, I see it now. You wrote down three x plus one equals four.

Yep, exactly. So what's the first step I should take to try to solve this?

The first step is to get all the terms with x on one side and the constants on the other side. So what do you think we should do with that plus one?

Okay, I'm going to try to subtract one from both sides and then I'll see what I get.

Great idea. Perfect. Now what do you get when you subtract one from both sides?

Okay, so chatgpt, this is what I ended up with. How does this look?

Spot on. Now you've isolated the term with x on one side and it looks like 3x 3. What do you think we should do next to solve for X?

I'm not sure. Can you give me another hint?

Of course. You have three x and you want to find the value of X. Think about what operation would undo multiplication.

Is it subtraction?

Close. But think of the opposite of multiplication, because 3x means 3 times times x.

Okay. I think it's division. Let me try to divide both sides by three and let me see what I get.

Bingo. Go ahead and divide both sides by three.

So, ChatGPT, this is what I ended up with. How does this look?

It looks perfect. You've solved it. And X equals 1. Nicely done.

Now, this isn't really showing off its math capabilities. That was a super, super simple math problem. I don't really know if it's greatly improved with math, although people were claiming the GPT2 chatbot was better with math, so it probably is a little bit better with math. This is more showing off the vision capability it can see as he writes the numbers on the paper and solves the problem. And really, again, this is showing in real time something that Google was kind of tweaking and speeding up and making it seem like it was able to do really fast, but it was really an edited video. So I think they're really trying to show you, again, look what we can do. But Google hasn't been able to do yet.

So, chatgpt, I really love that you taught the value of math to my friend Mark. And I wrote one last thing I'd love if you could take a look at.

Of course. I'd love to see what you wrote. Show it to me whenever you're ready.

Okay, so this is what I wrote down. What do you see?

Oh, I see. I love ChatGPT. That's so sweet of you.

Let me take out a computer for this. So I'm going to open the ChatGPT desktop app like Mira was talking about before.

Now they're going to get into the desktop app, which is pretty cool. One of the things that you can do with the desktop app is copy everything on Your screen, save it to your clipboard, and then ChatGPT will use that information for its chat. It's also got a button to share your screen, and ChatGPT can see everything that's going on on your screen and use that as context as well. Pretty cool. So check this out and to give.

A bit of background on what's going on. So here we have a computer, and on the screen we have some code. And then the ChatGPT voice app is on the right. So ChatGPT will be able to hear me, but it can't see anything on the screen. So I'm going to highlight the code command C it and then that will send it to chatgpt, and then I'm going to talk about the code to ChatGPT. Hey, ChatGPT.

Hey there. How's it going?

Yeah, it's going really well. I was wondering if you could help me with a coding problem today.

Of course, I'd love to help you out. What's the coding problem you're dealing with?

Okay, I'm going to share with you some code. One second.

Sure thing. Take your time.

Okay, so I just shared some code with you. Could you give me a really brief one sentence description of what's going on in the code?

This code fetches daily weather data for a specific location and time period, smooths the temperature data using a rolling average, annotates a significant weather event on the resulting plot, and then displays the plot with the average, minimum and maximum temperatures over the year.

Awesome. Thanks so much. And in the code, there's this function foo. Can you describe to me what the plot would look like if I had versus didn't have this function?

Sure. The function foo takes in a data frame X containing temperature data and a window size Y for calculating the rolling mean. It smooths the average temperature, minimum temperature, and max maximum temperature columns by applying a rolling mean over the specified window. With the foo function, the plot would show smoother temperature lines because the rolling mean reduces the noise or fluctuations in the data.

Awesome.

This makes.

Thanks so much. So, yeah, I definitely will want to apply this function and so I'm going to generate the plot and then show it to you. Give me one second.

Absolutely. Take your time. I'm excited to see the plot when you're ready.

So, so far ChatGPT has only been able to see the code, and now I'm going to run it and then I'll use the vision capabilities of the ChatGPT desktop app so it can actually see everything going on on the screen.

As Well, I don't really know if this is an improved coding ability inside of ChatGPT and GPT4. I feel like what they just showed it's been able to do. But what's really cool about it is the fact that they were able to copy the code and then the ChatGPT was able to read it, basically just because it was copied to the clipboard and give this sort of verbal explanation of what was going on in the code and what would happen if you changed the specific variable that he was referencing.

I'm sharing with you the plot now. I'm wondering if you can give me a really brief one sentence overview of what you see.

The plot displays smoothed average, minimum and maximum temperatures throughout 2018, with a notable annotation marking a big rainfall event in late September.

Fantastic. I have another question for you. Which months do you see the hottest temperatures and roughly what temperatures do those correspond to?

The hottest temperatures occur around July and August. The maximum temperature during these months is roughly between 25 degrees and 30 degrees. 77 degrees Fahrenheit to 86 degrees Fahrenheit.

What's going on with the Y axis? Is this in Celsius or Fahrenheit?

The Y axis is in Celsius. The temperatures are labeled accordingly, such as average temperature, minimum temperature, and maximum temperature.

If GPT4O is capable of real time translation. Mark, you want to try this one?

Sure. Yeah.

So it's really cool to see this translation feature because this is going to make communicating in other countries so much easier. You just open up your phone to the ChatGPT app and then you can have a conversation with anybody in almost any language. Check this out.

Let's do it.

I speak Italian, so you can speak English. Italian.

Let's try to do English Italian. Sure, let's do it. I would like you to function as a translator. I have a friend here who only speaks Italian and I only speak English. And every time you hear English, I want you to translate it to Italian. And if you hear Italian, I want you to translate it back to English. Is that good?

Perfetto, Mike. She wonders if whales could talk, what would they tell us?