Stunning New OpenAI Details Reveal MORE! (Project Strawberry/Q* Star)

So there have been a few more details revealed regarding the Strawberry update. Now this is rather fascinating considering there have been various rumors circulating Twitter, Reddit and various different social media platforms that all regard to AI development. So this video is going to try and clear up a lot of the information that's been going on in the space because I do know that there has been a ton of speculation and things have been rather confusing as of least. So a lot of this stuff is going to be coming from the account I ruled the world Mo and a lot of this is also going to be grounded in real factual evidence that we can of course verify. So I'm going to try to keep this video as non speculative as possible and as factual as possible without any hype or complete disregard for what is actually true possible. So one of the things that this account did recently tweet was that this account tweeted that there has been a GPT4O update. And and I'm going to talk about why this ties into a further point because there have been some recent revelations regarding GPT4O. So Diegoai tweeted that GPT4O got updated out of nowhere and I rule the world. Mo has been ushering a tweet on Twitter essentially stating that there is going to be some kind of update to GPT4O in the future. Now I don't think that this is the update that they were referring to. I think what they were referring to was of course an update that OpenAI may have pushed out because recently what has happened is that Google, if you check the benchmarks, they've taken the lead in terms of having a model that is rather good at answering humans requests and of course formatting them nicely and in terms of how the models tend to reason. And of course Claude 3.5 Sonnet has actually been in the lead for quite some time. So OpenAI pushing out an update like this isn't out of the ordinary. However, one of the things that I've recently tested with GPT4O and some people have tested GPT4O stating that look, it's marginally better, you can test this yourself. So in order to just quickly verify this or not, I did my own short test on GPT4O and I said how many Ls are there in the word lollapalooza? And they actually, you know, it got this question wrong. Now this doesn't mean that GPT4O is stupid by any means. It just basically means, you know, one of two things. Either a theory GPT4O model that they have isn't updated recently, as some may have suggested. And maybe that my edition, they just didn't roll it out to me because sometimes OpenAI can roll out different instances across different countries and different time zones. So maybe it's in America, maybe it's in Brazil. I honestly don't know where it's being rolled out right now. But for me personally, I've tested this. But this issue that people keep questioning the issue of can a model actually count the letters, whether or not? And of course, is this why they called it Project Strawberry or not? I don't think this issue is as big of an issue as people are making it out to be, because this is due to tokenization anyways, as Sean Ralston put it, that yes, of course there are some improvements, but if you actually understand how LLMs process text, they break it down into different tokens. So they don't see individual letters as we do. They see things as tokens. And then they use these tokens to like, you know, basically see how things have relationships with each other. And then, of course, they use that, in a sense, to kind of build a world model. So when you ask a model to count the number of letters in a word, it kind of struggles to do that. So you can see it says the way a word is tokenized can change depending on the context within a sentence. And of course, for instance, the word strawberry might be split into multiple tokens, such as StrawBerry, or even STRA and Wberry, depending on the model's tokenization rules. So running the text, and this is completely true, words are essentially, you know, done into tokens. So that's why some people say that this thing, you know, is not AGI, yada, yada, yada. But this doesn't really matter because as I've always said, if you can solve it with prompt engineering, I mean, what is, like, what difference does it make if you can change a few words and then you can actually get the right output? So I said write out each letter of the word, verify if it's L or not, and then count that. And then you can see right here, it actually managed to get the the word right. So this whole Strawberry, can it count the letters? Can it count the word? I don't think that this is a really important reasoning thing, because prompt engineering kind of solves this. And it's not like prompt engineering isn't going to be used in some of the most advanced reasoning cases where we do want real answers to things. And even if you're trying to use LLMs for certain applications, prompt engineering is going to be something that you use anyways. I think that this entire issue of can it count their letters or not, I think it's just a null point that doesn't really need much covering. But back to GPT4O. So essentially with this Strawberry account, essentially this account that was tweeting a lot of stuff, one of the things they did Tweet was a GitHub link. Now I was actually on Twitter at the time, I thought they were going to release something like an open source model, considering all of the recent pieces of information. But what they actually did was they put this date as GPT4O large for 2024, the 8th of the 13th. So what this is, this is a date for Tuesday. And essentially the reason that I think this is Tuesday is because, as you know, if you've been paying attention to the AI space, the majority of OpenAI updates or big tech AI releases are on a Tuesday or Thursday. So considering the fact that this date could be the release date for a model, this is rather fascinating. Now the reason I think this is even more fascinating is because of a recent tweet from someone who's actually, you know, got a long history of being a very credible AI leaker, and that is Jimmy Apples. And I don't really want to say leaker because he doesn't really leak stuff. He more so, you know, hints at, you know, next big model releases. So it's not like he's, you know, leaking entire things, but, you know, his hints have been rather accurate over the past few years. And he, he tweeted, you didn't count Sundar, and that's Sundar Pichai, the CEO of Google out, did you? You would be very wrong to do so. He's got a rematch next week. And essentially one thing that we've seen with the past AI model releases is that every single time Google has demoed something that's been absolutely incredible, it's always been overshadowed by OpenAI's releases. So for example, with, if you remember Project Astra, Google's amazing demo where they had a model that was really amazing in terms of what it was able to do, it was essentially an AI agent that's quite similar to GPT4O voice mode. But there was just a little bit more of a delay than OpenAI's one, and it was simply completely overshadowed by OpenAI's product release that was two days prior. So what many people are now speculating, including myself, is that potentially on Tuesday we might get some kind of release from OpenAI to overshadow what Google is going to release on the Thursday. And I think that this is quite likely considering A Jimmy Apples has a good track record of talking about AR releases and them being very accurate, and B, the fact that there have been a variety of different Google models released in experimental format, and I personally have enjoyed using them, and I've been using them every single day. So I think that this is not a surprise. And considering OpenAI's past track record, this wouldn't be a surprise if they do release something. Now, the thing is, the question is, will it be GPT4O large? Because the name GPT4O large suggests that GPT4O isn't a large model itself, which would mean that the current iteration of models that we do have access to are smaller than we do think. And this is where we get into some more information. So essentially right here you can see that this user in, you know, I'm just going to call it the Strawberry account, talks about how people are going to lose it when they realize GPT4O mini is the 8 billion parameter model, GPT4O is the 7 billion parameter model, and they're sitting on the 405 billion parameter model because their highly quantized goo is killing the competition with more still cooking. So essentially, I think here that GPT4O mini being the 8 billion parameter model is plausible, considering the fact that the model isn't as good, but it does well enough for various use cases and the fact that the model is extraordinarily cheap. Now, one thing that we have seen time and time again is that OpenAI have been a decent level above the standard level of research going on and the standard level of open models that we do have out there. So if we do kind of take that into account. Where llama. If we look at, for example, where llama 8 billion parameter is the llama 3.1, and then we think, okay, if OpenAI are, let's say, 30 to 40% ahead of standard closed models and even open ones, we can say that it is possible that we could get GPT4OH mini to be an 8 billion parameter model, that is definitely plausible, considering that that version that we're currently using is just the text base. So considering that they may have made certain breakthroughs or whatever, and considering when we look at the Phi 3 papers, which are a series of papers, Phi 1 all the way up to Phi 3, these papers kind of look at how models can get better with certain techniques and how smaller Models can be remarkably effective despite their size. So I don't think that this is impossible. And GPT4O being a 70 billion parameter model would be a real shock, because whilst yes, llama 370 billion parameters is a remarkably good model and it does really, really well, that would mean that OpenAI have done remarkable breakthroughs and that things are about to get even crazier. If GPT4O is a 70 billion parameter model, and it's not out of the question, it's not so far out of the question, but it would be a remarkable feat for OpenAI to accomplish that. And he does say that they're sitting on the large model because their highly quantized goo is killing the competition. And this kind of does make sense in a sense, in the sense that if you remember GPT4O, like people keep forgetting this. I'm going to bring this up again, because GPT4O, we don't actually have access to the full GPT4O model. One thing that people actually don't remember. And I'm going to show you guys this webpage, because this is something that I keep forgetting as well. And this is why I say people need to keep forgetting that OpenAI are still in the lead. And OpenAI have spoken about iterative deployment. Is this okay? If you're thinking, why are you talking about GPT4O large? This is a video about Strawberry. Just trust me, this is all going to link together. Just trust me, it's going to link together in a moment. So the 4o in GPT 4o actually stands for omni. And one of the things that people keep forgetting is that you can do a lot of things with a multimodal model. And if you look at all of the crazy things here, where you can take a logo, paste it onto an image, if you click see for sample, you can see, it can play audio, it can do all of these things. For example, there was character design. You can design a character, then you can have this character doing many different things, consistent characters, like having a model that can do all of this. Remember, we don't have access to the model. So maybe this is what OpenAI is going to release this Thursday. Considering the fact that we've got a situation on our hands where we're seeing that the Strawberry account is claiming that we just have the 70 billion parameter models. So, I mean, considering the fact that we don't really have the audio, we don't have the 3D stuff, we don't have this kind of image generation, and if we look at some of these, you know, other things, poster design where you can combine two images and it could make an instant poster. These are still tools that we don't have access to. So I think this is some of the stuff that people keep forgetting that is part of the GPT4O series. So this is something that, you know, like, look at the exploration of capabilities. I'll leave a link to this video because I made a video on this, I made a 30 minute video, you know, looking at all of this stuff. But GPT4O is actually a bigger model than we do think. So this tweet actually kind of has a bit more credibility considering the fact that GPT4O is a completely omni model, it's completely natively multimodal model. Now, if once again we look at the 400 billion parameter models, we know that 400 billion parameters in a model is achievable at current state of the art, because Llama 3.1, 405 billion parameters, the recent one we can see. And I'm looking at scales benchmarks here. The reason I'm not looking at the arena benchmarks is because the arena benchmarks, I wouldn't say they're contaminated, but I would say that there's not shady stuff going on. But just with how certain battles have gone, certain models are higher on the rankings than they should be, which doesn't give an accurate representation of how models will fare in, you know, user evaluation. So currently I'm just, you know, using this one because it's completely independent. But we can see here that llama 3.1 is right up there with state of the art, with a marginal increase in terms of abilities over other models. So right now we are getting to that stage where plausibly having a model that's 400 billion parameters being complete state of the art isn't out of the question at all considering Llama's recent release. Now, when we get onto the actual Strawberry stuff, this is where we get into some of the stuff that I was saying before, he says. AI explained has been close to this for a while, so I'd watch them for a cleaner take if you wanted it in. But this is what Elia saw. It's what has broken math benchmarks. It's more akin to reinforcement learning with human feedback than throwing compute at the problem. SUS column R is a very, very tiny OpenAI model. Using strawberry and Strawberry in larger models comes on Thursday, which is why I was talking about all the previous iterations of models and the kind of things that we're looking at now, it says think of it as an LLM fine tune to reason like a human. Hence why Sam liked the Level 2 comment and felt great about it and Elia did not. Here we are. Now, the reason I think this is so fascinating is because when we really take a look at what we're looking here, you know, if column R, SAS column R, the model that is currently on the arena area, the essentially chatbot area, where you can actually test out models and you can see the frontier, the next frontier of models and how they perform. If that model is actually like an 8 billion parameter model, 7 billion parameter model, and it's tiny and you, you know, it's performing state of the art in terms of just using whatever reasoning engine it's using. That is a true, true breakthrough because it's able to reason about problems that some of the other questions didn't get right. And like I said, if we're actually testing these things on actual reasoning through steps and spatial awareness and linguistic benchmarks and actual reasoning things, not just a trick question. Did you get the trick question right? I think that is the kind of thing that, with added scale would be where AGI is showing us that, okay, this kind of model is quite like an agile level system and maybe they just haven't scaled it yet. So I think this claim that SAS column R is a very, very tiny OpenAI model using Strawberry, it's not a bad claim because if the model was one that was a lot better in terms of the fact that it got every single question right, then I would be like, yeah, this is pretty insane because like I said before when I was testing the model myself, it was kind of just on par with the other state of the art. It wasn't ahead, you know, it showed some genius in some of them, but it wasn't that far ahead. But if it is a tiny model that's just as good as these other models, then that's completely insane. And like I said before, the way how the model did respond to me, it did respond like it just wanted to reason about the problems. It just wanted to reason about everything just the way it, like just internally I thought that the model was just trying to break down the problem in a certain way. And, and it just didn't seem like other models. It seemed like some kind of reasoning engine. That was just genuinely my experience with the model when I tested it over a bunch of different questions. And now this is why, with a reference back to this, of course. Hence why Sam liked the level 2 comment and felt great about it and Elia did not. If this is all true, we do have a model that is able to reason at the level as good as some of the state of the art models now. And it's a really tiny one. Maybe this is why, you know, ASI is within reach, because all they might need to do is, of course, scale this model. So with that being said, of course, remember, you know, it was only recently Elias left and said that superintelligence is within reach. So I think that statement, superintelligence being within reach is a really, really big statement. It's not like, you know, he went off to build AGI, he went off to build superintelligence, which means that maybe, you know, considering what he's thinking about, he's probably thinking, okay, if OpenAI can get to, you know, AGI here, then, you know, using some of their methods or some of the methods that I've worked on, we can definitely get to super intelligence. So, and I think that, you know, one of the things as well is that, you know, Ilya Sutskova is not like a hype person. He's not about building hype. But if he's saying that super intelligence is within reach, it doesn't mean that whatever they've worked on shows him that, you know, that super intelligence is not far off, which is an insane statement to make. So, yeah, that being a statement from someone that is, you know, arguably one of the linchpins of OpenAI success means that we're not far off from AGI, clearly, if superintelligence is within reach. Now, of course, there was also this level two tweet, which I've already referred to before. But level two is basically human level reasoning, which some people would say is AGI on the AGI scale, because, I mean, when you actually think about what AGI is, most people would say that it's human level stuff. And if we actually look at the actual levels, the levels from 1 to 5, level 5 being, you know, someone that can do the work of an actual organization. I don't think level five would be AGI. I think when you actually think about it, level five would be asi. Borderline, like, borderline asi. Because if a system can do the entire work of an entire company, that's not something that one human could do, even if, I mean, I guess you could, but like, when you really think about it, it's more leaning towards asi, if anything, if it's able to actually do that effectively and run non stop. Now of course there is also this tweet which is where it talks about huge model, soro, voice, video and safety, stating that I've referenced some model side based on Meta and Claude having small 8 billion, 72 billion and large. And this is a simple way to frame and means nothing except that a much larger version of FO is coming and when you try it will be noticeable jump that when we saw from going to three to four and it arrives next week with Strawberry. Now like I said before, I don't know if this is going to happen, but I do think that if Google manages to surpass OpenAI in public perception, they'll likely release something. Now one thing that I did want to touch on of course was safety. So essentially this GPT next, which we all know about, they state that it is ready to go, which does make sense based on the timelines and of course there's a variety of different things going on in the background. But it says it's difficult to say if competition will trump safety. Now it says the red teaming is finished and post training is done. The model is such an enormous leap in capabilities, it's becoming impossible to make the model safe. If you had this particular model unlocked, you could easily disrupt the world on an unprecedented scale. When you mix in voice, video, SORA agents and the eye watering capabilities, things heat up and they'll get the safety right and they'll rule it out, I'm sure. Now I don't know if the world is going to be disrupted on a complete scale if this model is released. GPT5, I think maybe GPT6, but I think one thing that we need to remember is of course OpenAI's preparedness framework. So the Preparedness Framework scorecard, this is from their recent GPT4O model card. And this is basically where they evaluate how the model is, the red teaming, how everything worked. And basically what we have here is their scorecard. So this is why I always state that I don't know if we're going to get systems like GPT7, GPT8 and people always like, what do you mean we're not going to get those systems? Think about it like this, okay? If you have a model that is really, really sufficiently smart, like it's really persuasive, it's autonomous, it's able to do a lot of things, we have to presume that these like every single model is going to be jailbroken. I don't think I've ever seen a model be released that hasn't been completely jailbroken. And the problem is that when you release a model and it's actually useful, the problem is that there's a fine line between a model being useless and a model not being able to be jailbroken. Because often what happens with the time is that a model, sometimes the more useless the model is, the less likely it is to be jailbroken. And with OpenAI, they have their models as products, they want them to be as useful as possible. And I've seen a ton of different jailbreaks before that are pretty hard to get around in terms of making sure it doesn't happen, but new jailbreaks are always going to be there. The point is that OpenAI have literally said themselves that only models with a post mitigation score of medium or below can be deployed and only models with a post mitigation score of high or below can be developed further. So they're basically stating that, look, they are only going to release a model if it stays at medium. If a model gets to high model autonomy, high persuasion or high biological threats or high cybersecurity, they won't release that model at all. Which means that even if we do get future models, we're probably going to get models that are somewhat dumbed down, because OpenAI have said before that they're not going to be releasing these models based on their preparedness framework, of course, and this is just a safety thing. So I do also think that as models get smarter and as they get more capable, it's going to be harder and harder to tame them because they're just that much smarter in terms of their reasoning, and it opens up the possibilities for widespread adversarial use. So this is just an issue that OpenAI have been facing with, regardless of what that Twitter account is saying. So I think overall, when we do take a look at this, look at this entirely, there's a few things that we can take away. Number one is that Google is very likely to release something next week. Considering the fact that they have been testing Frontier models on the chatbot arena for quite some time now, for I think around three weeks. And this is exactly what OpenAI did before they released GPT4O. I do think that the Strawberry model is likely a very small model that is kind of like a reasoning engine, because this is something that we've seen from Reuters. OpenAI have confirmed that Q Star was something and Sam Altman has in fact replied to this account and tweeted various times about this. And I do believe that OpenAI will of course release something depending on what Google release. And of course, I do also believe that when we do take a look at the safety issues surrounding these next frontier models, it is very likely to be hard for OpenAI to release these models, considering the capabilities of them. And I said before that one way that they could potentially release these models is, is they could make it so that you need a license in order to operate these models the same way that you need a license to work with certain chemicals. So that way if things go wrong with the model, they're able to independently track what the model was used for, what the chats were being sent, basically just like way to actually track the models and still use them in certain industries where they're not going to be used for dangerous things, but just a level of accountability so that we can say, okay, we've released this, you know, extremely capable model, but we're only going to allow it with people who have these licenses. And we're going to track everything that's said with the restrictions, of course. But of course you're not just going to release it completely to the general public, because individuals can do pretty much whatever they want without repercussions. So I think that's probably what will happen in the future, but I guess we're going to have to see. So with that being said, I think next week is probably going to be an insane week for AI, as long as Google manages to release something which it seems like they are gearing up to. So with that being said, I am eager to see what happens next week. And if I did miss anything in the video, don't forget to leave a like, don't forget to subscribe and I'll see you guys in the next one.