GPT 5 Is Now In Training (Open AI GPT-5 Announcement)

So, ladies and gentlemen, we finally have another huge update towards OpenAI's next large language model, or most likely, multimodal model. And it seems that the leaks were right. You can see by this article, it says OpenAI chief seeks new Microsoft funds to build superintelligence. And this update is very interesting because, ladies and gentlemen, we are talking about GPT5. So in this article from FT, if we scroll down, you can see that they talk about many different things. Okay. And there's a lot to unpack here because there is a lot of data about GPT5 that we have discussed over the summer that many of you have missed. So make sure you watch the video until the end, because it wasn't just today's announcements where they've talked about GPT5. They've given many different hints about exactly what GPT5 is going to be. And trust me, you're going to get a really good scope as to exactly what GPT5 will be. One of the things that they talked about was the article says the company is also working on GPT5, the next generation of its AI model. Altman said although he did not commit to a timeline for its release, it will require more data to train on, which Altman said would come from a combination of publicly available data sets on the Internet, as well as proprietary data from companies. Now, before we Talk more about GPT5, of course Altman did say he didn't comment on a timeline for its release because, as you know, a lot of the times with these AI models, there are always things that perform well and there are always things that underperform. What I mean by that is, with the release of GPT5, they're going to have to ensure a very, very smooth rollout, and they know that it's going to be a massive hit. Which means that if they do set a specific date for this model and then they do miss that date, then there's going to be a lot of speculation in the air, which is why we don't usually get release dates for these things. For, for example, this robot that you're seeing on screen now is called Neo. And OpenAI is essentially backing this robotics firm. And we know that likely what's going to happen is that OpenAI's LLMs or their vision models are going to be embedded into these robots. Now, the only problem was that this company essentially said that they were going to release this during summer, and as of yet, we've received no updates for this robot that is supposedly going to hold the next generation of large language models or AI systems that are going to be embodied into physical robots. Which is why we're seeing many companies not commit on certain timelines. If you are thinking about timeline is going to be for GPT5, I can provide some kind of guidance based on what we did have for GPT4. So in a video that we did release four months ago, we did talk about release dates.

And then when we looked at the.

GPT4 release date, we realized that GPT4 had data collection in 2021 of September. It finished training in 2022 of August and the release was 2023 of March. So it seemed that it was somewhat like a two year cycle in which over the course of two years data is collected, the training is finished, and then of course there is a release. So I'm going to show you a clip from that video where I explain timelines and why GPT5 not that far away based on current estimates. And this is not just what I've said, it's based on what Sam Altman said himself. Now of course you can see in this video we spoke about them starting training in around December and I think that that was very, very accurate.

But this is where Sam Altman gives us a slight glimpse at when he's going to start training GPT5 or potentially collecting said data. So take a look at this clip.

What about you, Mr. Altman, do you agree with that? Would you, would you pause any further development for six months or longer? So first of all, after we finished training GPT4, we waited more than six months to deploy it. We are not currently training what will be GPT5. We don't have plans to do it in the next six months. But I think the frame of the letter is wrong. What matters is audit.

I remember this was four months ago. So if the model does come out in late to mid-2025, I think that that would be a very good accurate. Now, the article continues to state that while GPT5 is likely to be more sophisticated than its predecessors, Altman said it was technically hard to predict exactly what new Capab skills the model might have. Although this is good wording by Sam Altman, I don't think that this is entirely the case. We do know that it is going to have an LLM base and we do know that the image capabilities are going to be upgraded. Right now, what we've seen from GPT4 in its current state as it's been evolving is definitely far more sophisticated than we initially got with GPT4. So what is likely to happen is that Auman is just saying that we can't say exactly what new capabilities and skills it might have because of course, as you know, these things can change if certain things do don't work out too well. So he says, until we go train that model, it's like a fun guessing game for us. We're trying to get better at it because I think it's important from a safety perspective to predict the capabilities. But I can't tell you exactly what it's going to do that GPT4 didn't. This is kind of true because in previous statements, Sam Altman in interviews has talked about GPT5 and what the future models are going to be like. So I'm going to show you a snippet from another video where we talked about how Sam Altman did talk about the video for GPT5. So take a look at that because I think it's super interesting a law here.

We can predictably say this much compute this big of a neural network, this training data, this will be the capabilities of the model. Now we can predict how it'll score on some tests. What we're really interested in, which gets to the latter part of your question is can we predict the sort of the qualitative new things, just the new capabilities that didn't exist, exist at all in GPT4 that do exist in future versions like GPT5. That seems important to figure out. But right now we can say, you know, here's how we predict it'll do on this eval or this. There are a lot of things about coding that I think are a particularly great modality to train these models on, but that won't be, of course, the last thing we train on. I'm very excited to see what happens when we can really do video. There's a lot of video content in the world. There's a lot of things that are, I think, much easier to learn with video than text. There's a huge debate in the field about whether a language model can get all the way to AGI. Can you represent everything that you need to know in language? Is language sufficient, or do you have to have video? I personally think it's a dumb question because it probably is possible. But the fastest way to get there, the easiest way to get there will be to have these other representations like video in these models as well. Again, like, text is not the best for everything, even if it's capable of representing everything.

So with that, you can see that GPT5 on the base data that we currently have from interviews and statements, we do know that GPT5 is likely to have video. I do think that video is definitely one of the hardest things to do at the moment because even companies that are completely focused on video, like Runway and other companies like Pika Labs, haven't completely solved the video challenge yet, although there were some recent breakthroughs that we will discuss. I don't think video is a at the level to which a language model can input it in. I do think that video is possible because GPT4 can output MP4s in its current state, but the videos that are released aren't as good as they should be. But of course, as you do know, AI is evolving at a very rapid rate with tons and tons of decent research papers.

What was very interesting was at the.

End of the developer speech from Sam Altman, he said a statement that I think most people did overlook, and I'm pretty sure that most people did miss. He stated that what we launched today is going to look very quaint relative to what we're busy creating for you now. And I can't imagine that he isn't talking about the next generation large language model. I'm pretty sure that they're working on AGI for GPT5, although that is just a broad statement. I do think that with the capabilities we are now seeing in these large language models and combining all of these models and even what not even Sam Altman and OpenAI are doing, but what uses is doing. For example, if you saw our recent video on what people were simply able to do with the Vision API where models were literally be able to hooked up to computers and control computers and control phones and simply do a lot of tasks very effectively, like around 70 to 80%. I think that if OpenAI managed to streamline that and get that working very effectively next year when we do have GPT5, I think it's really going to have some AGI like capabilities. And the worst thing about this is that many of the large language models that are still currently being released don't even match up to GPT4's current capabilities. Which means that by next year if they do release GPT5 or at least some parts of GPT5, maybe a video version of Dall? E, it seems that the lead for OpenAI is just going to get further and further and further away. Now what's even crazier about this is that OpenAI is reportedly trying to poach Google AI talent with around $10 million pay packages as the race heats up, which essentially means that OpenAI knows that GPT5, if they're able to get that off the ground successfully, it is going to be the large language model or the multimodal model that completely decimates everything. So I'm pretty sure that since they're throwing so much money at this problem, they know that if they can solve it, they're going to be at the very, very top in terms of what technology companies are able to do now with regards to reasoning capabilities and the other stuff about GPT5s in terms of its technical abilities. I'm pretty sure you're all about to be shocked at what we do say next because there is still a lot of information that we do want to include into this video.

So potentially depending on the number of modalities that it is trained on, it could be larger or it could be smaller. But we do know that if GPT5 was just text based, the parameter count would be significantly smaller. That is because recent papers have shown that the effectiveness of your data matters much more when training your large language model than actually upping the parameter count. Let me show you some examples showing you that a smaller parameter count is more effective than a larger parameter count when you use high quality data to train your large language model. So if they're going to train GPT5, we can refer to this paper by Microsoft released recently. In this paper they talk about textbooks are all you need. And basically what they state is that.

When we had high quality data versus.

Low quality data, we increased the effectiveness of our large language model three times and the only thing we did was switch the training data. This means that if they switch the training data to three different methods that we're about to talk about, we could see a 3x jump in the quality and responsiveness of GPT5 even if the parameter count does not increase. So I can Simply summarize the PHI1 paper by saying that they had a large language model that was trained on 1.3 billion tokens and it achieved comparable results on par with other large language models that were trained on 16 billion tokens and 175 billion tokens, including GPT 3.5. So it did on par with or better with significantly less parameters. Which means that GPT5 doesn't need a large number of parameters to be effective. All it needs is high quality data. Additionally, something else that they're going to do which is likely to be done with GPT5 is resulting from this paper show you're working. So OpenAI did release a paper which talked about how they increased the ability of the raw version of GPT4 just by using a different method of prompting. So essentially this new method of training was they trained two reward models. One for providing positive feedback on the final answer to a math problem and another for rewarding intermediate reasoning steps. By rewarding a good reasoning, the model achieved a surprising success rate of 78.2% on a math test, almost doubling the performance of GPT4 and outperforming models that only rewarded correct answers. The approach of rewarding good reasoning steps extends beyond the mathematics and shows promise in various domains like calculus, chemistry and physics. So the paper highlights the importance of alignment and process supervision training models to produce a chain of thought endorsed by humans, which is considered safer than focusing solely on correct outcomes. Which essentially just means that when you get these large language models to think step by step, these large language models double their efficiency simply based on this chain of thought reasoning. Which means that the output that you're going to get is going to be as twice as good. Which means that GPT5 is likely to incorporate this into its mechanism for outputting prompts. Which means even if you put in a simple prompt, you won't need to say, let's think step by step. It will have that thought process originally and the output process is going to be much better. Then of course, we have another research paper which blows everything out of the water. So as we talked about before, the way in which you prompt GPT4 or the large language model can make the large language model improve by a 2x or a 3x. Okay. And with GPT5, I do know that they are trying to increase the capability. Now, this paper called Tree of thoughts increased GPT4's reasoning ability by 900%, which is the base model, increased by 900% just by changing the words that you input to it. So essentially in this paper, they talked about how they used a tree of thoughts prompting. Essentially what it means is that if you think about every time you can make a decision, there are about five different outcomes. The large language model was asked to rate every single decision from number five being the best decision or number one being the worst decision. Then every single time they went through that decision, they then continued that same process to the end and essentially ranking all the different outputs and essentially thinking about what is the best output that you could get by of course going through every possible output. And this of course increased the reasoning by 900%. So if we know that tree of thought is going to be implemented into GPT5 and which it most likely will be. This could increase GPT5's reasoning by a huge amount. So along with data and training very differently. The parameter size is of course hard to come by, but I do think that the quality of GPT5 will be absolutely incredible if we take a look.

At how smart GPT5 is going to be.

As we discussed earlier on in the video, there are countless different examples of research papers and new different research papers coming out every single week that showcase the ability to increase the effectiveness of large language models without changing anything. Now we do know that data and stuff like that is going to increase the capabilities, but one thing we haven't talked about is how is GPT5 going to perform currently? We do know that GPT4 was a huge leap in advancement from GPT 3.5 and GPT4 is absolutely outstanding. We do know that GPT4, for example, was able to pass the bar exam and was able to get around 90% on various various different tests that are benchmarks for artificial intelligence. So knowing this, it is currently estimated that if GPT5 manages to succeed in its ability to reason, think critically and include these trio thoughts way of thinking that it could theoretically achieve around 99% on pretty much every test there is. We know that it's already great at math, already knows every single subject that there is. The only thing we need to do is pretty much fine tune everything in one last cycle, which is why many people are thinking that GPT5 will truly be very close to AGI in addition to remember that GPT5 will have images embedded in it. And of course we know that the performance of GPT4 greatly increased when vision was acquired. So many of the exam questions that GPT4 took, they took them with vision and without vision. So they were able to see diagrams and some of them didn't have diagrams. And when they were able to see see these diagrams, GPT4 with vision improved significantly. Then of course we need to talk about the various limitations that GPT5 will implement. Because although GPT5 is going to be absolutely insane when you think about everything before that we discussed from higher context windows to image audio and of course to new ways of thinking and prompting, GPT4 and GPT 3.5 still struggle with the most basic concepts. And you might think that is a ridiculous statement, but please look at this TED talk where they document where AI is incredibly smart and also shockingly foolish because it cannot understand basic concepts. But let the video explain it better because it's going to do a better job so all the video is going to show you is a simple question, a common sense question that it doesn't take a genius to answer. But GPT4 continually gets it wrong.

AI is passing the bar exam. Does that mean that AI is robust at common sense? You might assume so, but you never know. So suppose I left five clothes to dry out in the sun and it took them five hours to dry completely. How long would it take to dry 30 clothes? GPT4, the newest greatest AI system, says 30 hours. Not good. A different one. I have 3012 liter jug and 6 liter jug and I want to measure 6 liters. How do I do it? Just use the 6 liter jug. Right. GPT4 spits out some very elaborate nonsense. Step one, fill the 60 liter jug. Step two, pour the water from 6 to 12 liter jug. Step three, fill the 6 liter jug. Again, step four, very carefully pour the water from 6 to 12 Liter jug. And finally you have 6 liters of water in the 6 liter jug that should be empty by now.

So with that, it's going to be interesting to see how they do solve this issue. I haven't been aware of any solutions just yet, but it will be interesting to see if they even focus on this because largely we do gloss over these problems and just focus on the interesting stuff. Next. Of course, now that you know that we don't really understand exactly what AI is doing, we also need to talk about emergent capabilities. Now, this is something that we've spoken about previously in the past, but you have to understand that GPT5 is likely to be a few echelons better than GPT4. This means that even if the parameter count is the same, a few emergent capabilities are going to be seen in GPT5 that we simply cannot predict that have been in GPT4. Now, one of GPT4's most emergent capability was theory of mind. And theory of mind essentially is where an AI is able to think about how other people are thinking in certain situations. Now, of course, this is particularly worrying if you are thinking about how an AI could potentially manipulate humans to get it to do things from it. Because of course, these large language models do have access to almost every piece of text on the earth. And that means it's not limited to other books about persuasion, manipulation and of course, persuasion tactics. Now take a look at this clip that perfectly explains emergent capabilities. You've likely seen it before. But for those of you who hadn't, you really need to understand because this emergent capabilities phenomenon is likely to be One of the reasons that they don't release GPT5 on time, because if there is an emergent capability, then the AI researchers or OpenAI will need to learn how to effectively contain it or potentially remove this capability.

Some people use the metaphor that AI.

Is like electricity, but if I pump even more electricity through the system, it doesn't pop out some other emergent intelligence.

Some capacity that wasn't even there before.

Right. And so a lot of the metaphors that we're using, again, paradigmatically, you have to understand what's different about this new class of gollum generative large language model AIs.

This is one of the really surprising things, talking to the experts because they will say these models have capabilities. We do not understand how they show up, when they show up, or why they show up. Again, not something that you would say of like the old class of AI. So here's an example. These are two different models, GPT and then a different model by Google. And there's no difference in the, the models, they just increase in parameter size. That is, they just, they just get bigger.

What are parameters?

Etha it's just like the number essentially of weights in a matrix. So it's just, it's just the size. You're just increasing the scale of the thing. And what you see here, and I'll move into some other examples, might be a little easier to understand, is that you ask the, these AIs to do arithmetic and they can't do them. They can't do them and they can't do them and at some point, boom, they just gain the ability to do arithmetic and no one can actually predict when that'll happen. Here's another example which is, you know, you train these models on all of the Internet, so it's seen many different languages, but then you only train them to answer questions in English. So it's learned how to answer questions in English. But you increase the model size, you increase the model size and at some point, boom, it starts being able to do question and answers in Persian. No one knows why.

We've already seen that clip on emergent capabilities. But I do think this is also going to show you exactly why AI has these emergent capabilities. You have to understand that although we can see the outputs of artificial intelligence models, models like GPT5 and GPT4, we still don't know what they're actually doing. So take a look at this tweet right here. It says this person tweeted. These engineers never speak a word or document anything. Their results are bizarre and inhuman. This guy trained a tiny transformer to do addition, then spent weeks figuring out what it was doing. One of the only times in history someone has understood how a transformer works. And transformers are essentially the building blocks of these large language models. Then, of course, you can see here, this is the algorithm it created to add two numbers. And you can see here, this is a large, simple, complex calculation that it's doing to add two simple numbers, which is pretty crazy, if you ask, which means that these AIs think completely different to us. This example shows us that this artificial intelligence thought about basic math as rotation around a circle, which goes to show that although it might tell us an answer, it doesn't tell us how it got there. And this is what's so scary about AI. We would never know that it's thinking about rotations around a circle when performing simple addition. But it is. Which means that we need to ensure that these artificial intelligence are completely aligned. Because if you release something like that into the public, the risks could be explicit.