NEW OpenAI Details Reveal NEW ORION MODEL! (Project Strawberry/Q* Star/ OpenAI ORION Model)

So there has been so much information released regarding OpenAI's new model and even more information regarding a super secretive model that is called Orion. In this video, I'm going to dive into absolutely everything you need to know, as well as some things that you might have missed. So this article from the information clearly gives us a lot of information regarding the Strawberry model. And you can see right here that they give us some key details about some things that you may have missed. So, for example, right here you can see that the article starts by stating that as OpenAI looks to raise more capital, they're trying to launch the next AI product that they believe can reason through tough problems much better than existing AI. Now this is where it gets interesting, because of course, now apparently we're going to be getting the launch of the new AI codename Strawberry, which was previously called Q, pronounced Q. Now, this could be a release which might actually be on top of ChatGPT, and apparently the release date for this could be as early as the four. Now, the four refers to the following months, which are September, October or November. Now, I am placing my bets on the latter, which is November. But if there are some sneaky announcements from other companies such as Google or Anthropic, I can bet you that OpenAI is going to release this a little bit earlier. Now, you can see here that it says that Strawberry can solve math problems it hasn't seen before. And this is something that today's chatbot cannot reliably do. And it has been trained to solve problems involving programming, but it's not limited to answering technical questions. So one of the overall things you need to understand about Strawberry is, is that this model is a model that is a reasoner. This model has been trained to solve complex problems and this is like their reasoning engine that's going to help them to increase the ability of their other model. Now, the article talks about how when given additional time to think, the Strawberry model can also answer customers questions about more subjective topics such as product marketing strategies. Now, I think the most interesting thing about this is, is that when given additional time to think, this is a method that you know the article later references, which is called Test, time, compute or There are many different ways that you can actually give the model time to think, such as using different agentic workflows or different styles of prompting. But with this model that is a lot slower. I'm guessing they've got some kind of internal workings where the model thinks about the problem in an abstract way or whatever way it uses. It's able to generate a lot better answers, which by doing this. Now, you can see here that there is also a demonstration of Strawberry's prowess with language related tasks. So OpenAI employees have shown their coworkers how Strawberry can, for example, solve New York Times connections, a complex word puzzle. So if you don't know what the New York Times connections is, it's essentially a word puzzle where you have to draw connections between words of increasing complexity. So I'll show you guys an example of today's one. So what you can see here is that we have 16 different simple words. What you'll have to do is you'll have to kind of group these into four different categories. So, for example, you can pause the video now if you want to go ahead and do this, but I'm going to show you the first one. So, for example, you can see we've got thunder, then we've got raw, then we've got crash, and then we've got boom. So these are all sounds, all of these are loud sounds. So we could group those into a group of four. Then we would have to look at the remainder remaining words and group those into separate groups of four words. Now, this is quite interesting because I actually did some small tests on these LLMs, the current LLMs, and they currently don't perform that well at this task. And even when I asked Claude 3.5 Sonnet, for those of you who are thinking this task that Strawberry is so good at, I'm wondering if Claude 3.5 Sonnet is even good at this task. Well, I asked Claude 3.5 Sonnet and this model is currently state of the art, meaning that it is the smartest model that you can get your hands on. You can see right here that what we have is a model that can't really get the answer. Now, what's interesting was that when I asked ChatGPT, you can see that it managed to blab on about NBA teams. Then it did get the things that make a loud noise and it said things related to fire. And I'll show you guys the answers in a moment. But these aren't the correct answers. You can see here that it managed to get the loud sounds and it did manage to get the hair styling one, but it wasn't able to get the other areas and it did mess up some key elements. Now, the reason I'm showing you this is because if we can see that Strawberry, the model is able to do this rather well, then that means that this model is clearly much smarter than what other models are currently sitting on. And you can see right here, these are the answers. And none of the models managed to guess that this is chili pepper quality or these are kinds of cards. So this is something that I think is quite interesting because it shows us the increasing level of difficulty and how these models are able to reason through different problems. Now, right now there might even be some people that are probably figuring out ways to get an LLM to solve this. But I do think that if you can simply give a problem like this you to the model that is Strawberry and then it's able to immediately solve this issue, then I think that that is going to be something that's rather fascinating because it's going to allow us to truly understand where these models are in terms of their raw capability. Now, what we also have is more information regarding Strawberry and agents. So it says that the effort to launch Strawberry as part of OpenAI's Never Ending Battle to stay ahead of other well funded rivals vying for supremacy and conversational AI or large language models technology also has implications for future products known as agents that aim to solve multi step tasks. One of the main issues with why we don't have agents yet are essentially, you know, just one issue and the issue is down to reliability. What this means is that we need models that don't make any issues. Because if high reliability is crucial for AI agents performing long multi step tasks, because these tasks involve a series of actions that build on each other. So for example, if the AI makes a mistake early on, it can throw off the entire process leading to poor outcomes or even complete failure of the task. So for example, let's think about it like this. Imagine trying to bake a cake. If you mix the wrong ingredients or even set the oven to the wrong temperature, and at the beginning, it doesn't matter how well you follow the rest of the recipe, the cake won't turn out right. Similarly, if an AI makes a mistake at any point in a complex task, it can ruin everything that comes afterwards. And reliable AI ensures that every step is done correctly so that the task is completed correctly, just like carefully following each step of a recipe to get a perfect cake. And without this reliability that Strawberry aims to provide, the AI can't be trusted to handle important or complicated jobs with multi step tasks. Hence the reasoning we don't have reliable AI agents just yet. Now this is where we introduce Orion. So it says here that OpenAI's prospects rest in part on the eventual launch of a new flagship large language model it is currently developing, codenamed Orion and that model is seeks to improve upon its existing flagship large language model GPT4, which it launched early last year. But by now other rivals have launched LLMs that perform roughly as well as GPT4. Now, this is rather true. We actually do have models that are currently better than GPT4 and on GPT4's level. So interestingly enough, it's surprising that OpenAI isn't rushing to get this model out the door yet. Now, this model, codenamed Orion, interestingly enough, was actually leaked sometime last year. Now, for those of you who don't pay attention to Jimmy Apples, you may have missed this tweet in November of 2023 when he tweeted, let's conquer the cosmos. Mood Curious Jimmy now, this image might seem random to anyone that views it, but this image is actually the constellation of Orion, one of the most recognizable and prominent constellations in the night sky. So the fact that Jimmy Apples tweeted this in November of 2023 and if you're thinking, why are you even talking about a random Twitter account tweeting a random image? This account has previously tweeted many different things regarding OpenAI, and these things were months before they occurred. If you're in the AI space, you're going to know exactly what I'm talking about. But it's just interesting that this was in the works for for so long, and it's only now that we're starting to get the first details of how these products are being developed. Now, you can see here that it says it isn't clear whether a chatbot version of Strawberry that can boost the performance of GPT4 and ChatGPT will be good enough to launch this year. The chatbot version is a smaller, simplified version of the original Strawberry model, known as Distillation, and it seeks to maintain the same level of performance as a bigger model, while being easier and less costly to operate. So I think what OpenAI are trying to do here is that they of course have this specialized model, the Strawberry model, and this is the model that, you know, of course they could really release it and it would be absolutely incredible in terms of its reasoning capabilities. But they're stating that they aren't sure whether or not it's going to be released. I'm not sure whether it's safety reasons, but whatever. But the point is, is that they are trying to distill the model down so it works as well as the bigger model, while being easier and less costly to operate. You have to understand that the inference for these models that are really smart. It takes a long time to get the answers. And this was something that we saw with Google's models where we had to wait a few minutes to get answers for certain really advanced systems. And I am wondering if they're going to release this distilled version of this Strawberry model and apply it to ChatGPT or GPT4 to improve its reasoning capabilities that surpass the other models at this moment in time. Now what we also have here is the fact that this involves the process of distillation, which is something that is becoming more and more common in the AI space. So you can see here from Google's blog, it basically talks about how most fine tuned large language models contain enormous numbers of parameters. Consequently, foundational large language models require enormous computational and environmental resources to generate prediction. Note that large swaths of those parameters are typically irrelevant for a specific application. Now what they're talking about here is that when you distill a model, it creates a smaller version of a large language model. And the distilled large language model generates predictions much faster and requires fewer computational and environmental resources than the full LLM. However, the distilled models predictions are generally not quite as good as the original LLMs predict. And it says that recall that LLMs with more parameters almost always generate better predictions than LLMs with fewer parameters. So overall, it seems that OpenAI is using this entire process here, known as distillation, to try and sort of get some of the key reasoning abilities of Strawberry out to the mass market. Now I think that this might either be released on top of GPT4 as a way to edge its model to be even better than the competition. But one of the things that I found super interesting about this entire article was the fact that this article doesn't even mention GPT5 1. So all of these things that we're talking about, these are all separate to GPT5. So it seems that these models that we're going to be getting are varied in terms of their capabilities and in terms of their size. Looks like we're going to be getting GPT5 and advanced GPT4 and potentially Orion on top of that too. Now with distillation, there is some more interesting stuff as we speak here about the article. And it says that with distillation it can be used in a chat based product before Orion is released. And it says this shouldn't come as a surprise given the intensifying competition among the top AI developers. And it says we're not sure what a Strawberry based product might look like but we can make an educated guess. And one obvious idea would be incorporating Strawberry's improved reasoning capabilities into ChatGPT. However, these answers would be more accurate. They would also be more slower. Now, the reason for this is of course, when you have a model that is thinking through multiple steps, if you give it a harder question, often the model has to just take more time to think about that problem, as would any kind of human who's dealing with any kind of harder problem. And this is going to be kind of interesting to see how this evolves in the future, especially considering we're getting advanced chips that allow for faster inference. So I am wondering to see if we're going to be getting these improved reasoning capabilities into ChatGPT later this year, because there aren't any current talks of GPT5 being released this year. And considering that that is going to be a flagship product and there are many delays surrounding the model. Considering all of the things going on around OpenAI, I wouldn't be surprised if OpenAI just incorporated a distilled version of Strawberry into the current version of ChatGPT, just so that they don't lose in terms of their position at the leaderboard. One thing you have to know about OpenAI is that they love the number one spot on the LMSYS arena. And that benchmark kind of serves as an anchor for people to position themselves for where they think the best current model is. When we actually look at what Strawberry might be suited for, you can see that it talks about how it's not going to be suited for applications where users expect immediate responses, like OpenAI's Search GPT engine, which is where you want certain responses that are almost immediate. But. But of course, what they are using this model for and what the real applications are going to be for is for sensitive use cases like fixing non critical coding errors in GitHub. So this model is going to be a model that is quite slower, but it's going to be a lot more accurate. And this is going to be what people tend to use in areas where they need accuracy over speed. That's why we have models like Google's Flash and of course GPT4O mini, where you just want response quickly and you're not that interested whether or not the model is right or wrong. You just really want to know if that model is going to give you a response. But with these more advanced models, the ones that are more advanced at reasoning, what we would want is these models to have a lot more accuracy because they're dealing with more sensitive Use cases and of course, things like programming where the accuracy is more important than the speed. Now, what's fascinating about all of this is that they actually talk about synthetic data here and they talk about two strawberries. So essentially what we have here is that we have the bigger version of Strawberry, OpenAI's secret model, the bigger version of Q being able to generate training data for Orion. And I think the implications for that are absolutely incredible. So it said OpenAI is using the bigger version of Strawberry to generate training data for Orion, said a person with knowledge of the situation. That kind of AI generated data is known as synthetic. It means that Strawberry could help OpenAI overcome limitations on obtaining enough high quality data to train new models from real world data, such as text or images pulled from the Internet. One of the key things that most people have spoken about in terms of these models and their limitations was the synthetic data area. Now, I recently did a video, it isn't released yet, but it's a deep dive into all, all of AI's limitations for the future. And one of the limitations was of course, the data wall. But in many cases the data wall isn't really going to happen because number one, we have synthetic data. And although some individuals talk about model collapse, which is where models trained on synthetic data tend to eat their own tail and then collapse isn't really an issue, especially if you manage to filter the data with a high quality AI model or a human. And we also have the fact that we haven't actually exhausted all of the data that currently exists. So the fascinating thing about this is that there is the raw Strawberry model, which is apparently large enough to generate probably great reasoning examples that is going to be used in training Orion. And we do know that high quality data is really important for models to succeed with their reasoning tasks and of course, high quality output. So you can see that the bigger version of Strawberry is generating this training data for Orion. Orion. Which brings me to a thought pattern where I'm now thinking about how effective is this model, the Strawberry Q Star model, in terms of its reasoning capabilities, because it's able to literally generate data for their next iteration of models called Orion. So that's going to be absolutely incredible in terms of those implications. Now what they also talk about is reducing hallucination. Using Strawberry to generate higher quality training data could help OpenAI reduce the numbers of errors its models generate, otherwise known as hallucinations, said Alex Gravely, CEO of agent startup Minion AI and former chief architect of GitHub. Copilot said, Imagine a model without Hallucinations, a model where you ask it a logic puzzle, and it's right on the very first try. The reason why the model might be able to do that is because there's less ambiguity in the training data. So it's guessing a lot less. And I think that if a model is able to reduce hallucinations severely, this is going to actually increase the rate at which AI is adopted, because hallucinations are essentially errors where the model guesses or makes up things. And this actually reduces the ability for models to be used in certain applications where it would be rather effective. But hallucinations are errors, and in some industries you can't really have any errors or the error rate needs to be so low that these models just can't be used in the workplace because a high degree of reliability is completely needed. Now, what's also fascinating about this model is the fact that they actually also showed this model to national security officials. You can see right here that it says, earlier this month, CEO Sam Altman tweeted an image of strawberries. You know, without fanning the flames of speculation about an upcoming release. But they also gave demonstrations of Strawberry to national security officials and this summer set a person with direct knowledge of those meetings. So we can kind of start to gauge the picture of how good this Strawberry model is. Because if this model is one that is a able to generate enough high quality training data for the smaller models like Orion, and if they are saying that, look, this is the model that led to this entire kerfuffle at OpenAI, and if they're saying that now that this model has been shown to national security officials, a person with direct knowledge of those meetings said, then that definitely means that this Strawberry model is one that is truly, truly advanced. And this truly makes me wonder about the capabilities of this model, because it seems that this model is so advanced that they're now taking certain precautions. Now, there is the other argument, which is, of course, this might not be true. We could have Sam Altman taking a different approach, approach, considering the fact that there were departures of several safety leaders earlier this year, some of whom claimed that Altman didn't care much about safeguarding the tech as they did. You can see here that it also states that by demonstrating an unreleased technology to government officials, OpenAI could be setting a new standard for AI developers, especially as advanced AI increasingly becomes a national security concern. So I've always said that government intervention was always going to happen. And this is also something that Leopold Aschenbrenner has said would eventually happen. To AI labs. This is something that he said spoke about in his PDF document, AI and the decade Ahead. What was also interesting is that I didn't see this exactly, but it said that Sam Altman stated that we feel like we have enough data for this next model. Sam Altman apparently said at an event in May, which was likely referring to Orion, and he says they've done all sorts of experiments, including generating synthetic data. Now we actually also look at why this would be a lucrative application, because AI that solves tough math problems could be a potentially lucrative application given that existing AI isn't really great at math heavy fields such as aerospace and structural engineering. And it's a goal that has tripped up AI researchers who have found that conversational AI ChatGPT. And it's a goal that has tripped up AI researchers who found that these large language models like ChatGPT are prone to giving wrong answers that would flunk any math students. And improvements in mathematical reasoning could help AI models reason better about conversational queries such as customer service requests. Now, what's also interesting was that this article showed us these other models that are also working on their reasoning engines. We've got Google DeepMind, who are working on developing AI models to solve math and geometry problems. We've got Anthropic, who are highlighted through their reasoning capabilities and interpreting charts and graphs in their latest model, Claw 3.5 Sonnet, which is just completely remarkable in terms of how it performs. You've got these other companies here, like Imbue, that are mainly focused on developing AI agents to code and complete tasks. And of course we've got Cognition Labs, which is developing AI agents to code, and of course Magic, which is doing the exact same. So there is clearly a competitive arena for these large language models to compete. But it's rather interesting to see which company is going to come out on top. Now, of course you can see that this article also speaks about how to improve the model's reasoning. Some startups have been using a cheap hack that involve breaking down a problem into smaller steps. And though the workarounds are slow and expensive now, essentially what they're referring to here is the other method that people currently use, which is reflection. So reflection is basically the process of where you have an LLM, generate a response, and then you ask it to criticize that response, and then, you know, basically give feedback to itself. So says, for example, after a customer asks an AI app to draft the aforementioned blog post, the app could automatically trigger extra queries. The customer wouldn't see such as asking the LLM behind the app to rate how well it did and where it could improve. And this is the Socratic method of teaching a student to think critically about their belief or arguments. And this generally leads to better responses. So something that you could always try with your models. Now, another thing that these models do in terms of their reasoning capabilities, and we don't know what method Strawberry uses, but we'll get onto that in a moment. But it says if a developer wants to take a page from Google's book, they might try a technique called sampling. And sampling search is really, really effective. And this is something that has led to superhuman AI systems. So this is where during sampling, a developer increases an LLM's ability to generate creative and random answers by asking the same questions dozens or even a hundred times and then picking the best answer. So it's like, you know, for every answer, considering that, you know, these models aren't completely accurate, eventually sometimes they do get it right. And then what you try to do is pick the reasoning steps or the answer that seems to fit what you'd want to see. For example, it says a coding assistant app might ask an LLM for 100 different answers to the same coding prop, then the app could run all those pieces of code, then see which ones produce the correct answer and automatically select the most concise codes, add the final answer. So this is how some of these systems work. And you can see right here that this is actually how AlphaCode 2 works, which is better than 85% of programmers. So basically it's sample generation. It generates up to a million diverse code examples and of course most of them are going to be worthless and then it just basically selects the one that are the best. Now, I do remember that there was some discussion basically stating that whilst yes, this could lead to superhuman systems, the problem with this is that when you actually look at how humans do this, humans don't search over that many spaces in order to get to the answer. Like if you look at how AlphaGo actually won, it was searching over many different opportunities and different moves. But if we look at Lisa Dole, he's a human and the human mind we know was only searching around between 50 to 100 different solutions, whereas AlphaGo was searching through. I'm not sure if it was thousands or millions, but it was certainly like a lot more. And there is this discussion that whilst yes, looking and reasoning through generating a million different diverse code samples and then just running the one that works, this does work. But there is clearly something about humans, something about the way that humans reason through problems that is just way more efficient than these computers. And if we can find a way to actually get these computers to truly reason like humans do, in the sense that they only need to search through maybe 50, maybe not 50,000, then we could get models that are literally just 10 times smarter than us. Because, you know, we're not just sifting through a million different solutions and seeing which ones work. We use what humans do, and I'm talking about really smart humans, and then use those outputs. So you can see right here that this article actually also talks about what Ilya saw. You can see here, it says that Strawberry has its roots in research. It was started years ago by ilya Satskova, then OpenAI's chief scientist, and he recently left to start a competing AI lab, which is Safe Superintelligence. And before Ilya Satskova left, OpenAI researchers Jakub Pachocki and Simon Tidor built on Sutskova's work by developing a new math model called Q. And this was where we got the first inklings of something crazy going on at OpenAI. Now, this breakthrough and safety conflicts. Of course, this came just before the board, led by Elias Sutskever, fired Sam Altman before quickly rehiring him. And this is where last year, leading up to Q, OpenAI researchers developed a variation of a concept known as test time computation. And this is where you boost LLM's problem solving capabilities, where you essentially give the model more time to consider all parts of a problem after you've asked the model to execute. And at that time, Elias Atskova published a blog post related to this work. So basically, test time compute is something that is rather effective. And this is where you make models instead of making them bigger, you essentially make it think harder when it's trying to solve a problem. And this paper I'm going to show you guys from Google. So this one right here was where they tried two approaches and they basically first let the AI revise its answers, kind of like how a student might check their work. Then B used a separate AI to judge which answers look promising and then explore those more. And they found that different strategies work better for different kinds of problems, and easy problems need one approach and hard problems need another. And by choosing the right strategy for each problem, they were able to get better and better results whilst using less computer power. And for easier problems, this think harder approach sometimes worked better than just using a much larger AI. And here's where we can see the research that they were talking about that was published the last year. You can see May 31, 2023, improving mathematical reasoning with process supervision. You see, it says we've trained a model to achieve a new state of the art in mathematical problem solving by rewarding each correct step of the reasoning process. And this is where you call it process supervision instead of rewarding the correct final answer. So it's basically just focusing on the next step. And as long as you get the next step right, then that is where you know you're awarded. So it said, in addition to boosting performance relative to outcome supervision, process supervision also has an important alignment benefit. It directly trains the model to produce chain of thought that is endorsed by humans. So I'm wondering that if this is what Q is, if this is what the Strawberry model is, because we can see here that when we look at the math problems and when we spoke about how Q Star was able to solve math problems and how it was able to do that rather effectively, maybe this is where the research is in the sense that they built something else on top of this that was a lot more effective. Because what we can see here is that process supervised revision actually performed increasingly well. And you can see that over time of, you know, as there were more and more samples, this test performance was going up and up and up. So it seems like, you know, research was probably built on top of this and this was able to make the model just a lot more effective. It does say here though, and I think this is why probably we haven't, you know, seen the model released yet, but it says it's unknown how broadly these results will generalize beyond the domain of math. And we consider it important for the future work to explore the impact of process supervision in other domains. And if these results generalize, we may find that process supervision gives us the best of both worlds, a method that is more performant and more aligned than outcome supervision. So this is rather fascinating because this, of course, you can see all the authors, you can see Jan like and Ilya Sutkova, and, you know, I can't believe I didn't see this article before, because when you look at Q, this is something that was of course, a small model that was able to, you know, solve math problems really well. But I'm guessing that this process supervision was something that is related to test time and compute and the other research regarding Strawberry. Let's just do a conclusion. So to conclude, Strawberry, which was formerly Q is a new AI model focused on improved reasoning capabilities can solve math problems and complex word puzzles. This might be integrated into ChatGPT later this fall and is currently being used to generate training data for other models. Well, it might be. Now, Orion is the next major model which doesn't reference GPT5 and this aims to improve upon GPT4 and is currently in develop and it may use generated data from Strawberry in order for its training. Now, essentially, we have a situation on our hands. OpenAI is working on multiple AI models to stay ahead of competitors. Strawberry is focusing on the reasoning, and Orion's meant to be the next flagship model after GPT4, or I'm not sure if it's going to be GPT5, which could be a completely different model. And of course, we do know that there are currently two versions of Strawberry, the distilled version, which could be applied to ChatGPT, and of course, we do have the raw version, which is currently excelling at reasoning and other abilities which could be used to train models like Orion. So this is going to be completely fascinating to see where OpenAI is. It seems that OpenAI are just so far ahead in terms of the competition. If they've got GPT5, they've got Orion, they've got Strawberry distilled, they've got the original Strawberry and they've got all of these other things in the work. So it's going to be kind of interesting to see what OpenAI has up their sleeve. And if you enjoyed this video, I'll see you in the next one.