OpenAI Employee ACCIDENTALLY REVEALS Q* Details! (Open AI Q*)

So there was a recent tweet that was actually deleted from someone that works at OpenAI that has got people a little bit rattled and it's got the community wondering why the tweet was deleted in the first place. And if this tweet actually relates to OpenAI's infamous Q star model, which they refused to talk about. So this video is going to include some speculation, but there are several key points that I do want to make which shows why this tweet could be rather interesting. And of course, everything will be linked in the description. So essential. Essentially, we have a tweet here that is from Noen Brown, and he is a prominent figure in the field of artificial intelligence, known for his contributions to developing AI systems capable of playing poker at superhuman level. And his work has significantly advanced standing and capabilities of AI in imperfect information games, a category that includes not just poker, but potentially extends to real world applications like negotiation, cybersecurity and even strategic decision making. And he's done really, really well for him. Now he currently works at OpenAI. And what's absolutely crazy was this tweet right here. So he stated that you don't get superhuman performance by doing better imitation learning on human data. And essentially right here he's stating something very quickly and then he deleted the tweet. Now, what he could be referring to here is a variety of different things, but the main speculation that many people have come up is the fact that he's talking about, you know, the planning model which OpenAI is working on that is allegedly Q Star. Now, in addition, there are some additional tweets that you might want to see from Noam Brown. So we can see right here that this is the earlier tweets from 2023 when he was talking about joining OpenAI. He says, I'm thrilled to share that I've joined OpenAI. For years I've researched AI self play and reasoning in games like poker and diplomacy. I'll now investigate how to make these methods truly general, if successful. And take a look at this. We may one day see LLMs that are a thousand times better than GPT4. And that is a crazy statement, but there is a very, very interesting clip where he actually talks about why this is not only feasible, but very, very fascinating. So he also adds onto the tweet, he says, In 2016, AlphaGo beat Lisa Doll in a milestone. But key to that was the AI ability to ponder for one minute before each move how much did that improve it for AlphaGo? Zero. It's the equivalent of scaling pre training by 100,000x. And we can see that this really did increase the ability of it by 100,000 times in terms of the equivalent scaling pre training, which is very, very incredible. The point here is that what he's talking about is the ability to get more out of a model than in a more efficient way. And this is something that is not really widely discussed. And he also says all those prior methods are specific to the game, but if we can discover a general version, the benefits could be huge. Yes, inference may be a thousand times slower and more costly, but what inference cost would we pay for a new cancer drug or proof of the Riemann hypothesis? Improved capabilities are always risky, but if this research succeeds, it could be valuable for safety research as well. Imagine being able to spend a million doll on inference, see what a more capable future model might look like. It would give us a warning that we otherwise would lack. And he dives further on into this, basically stating that look, in certain tasks you don't need the model to respond almost instantly. What you do prefer is you do prefer accuracy over speed. And in order to give the model more time to think, which is something that we've seen in recent research papers such as Quietstar, where LLMs were given times to have an internal monologue and it improved them by quite a bit. It is something that of course could essentially be applied here. Now, something that he does talk about is of course this. He expands on this and I'm going to show you all a clip from an interview where he dives into this concept a lot further. And remember, this is the guy that is working on planning and many people suspect that this is one of the lead researchers that are working on Q.

Star planning in these games. So I mentioned in Go, if you add planning, it increases. It's like the equivalent of increasing your model and training by a hundred thousand x. And the same thing is true in poker. If you add search in poker, it's the equivalent of increasing the model size and training by a hundred thousand x. And I think you can do something similar in language models. It's not really clear how you do it yet, but I think that there is an opportunity there and I think this is really important because you know, if you look at the cost of how of know how much it costs to train these language models today, it's really expensive and we're going to see it scale up. I'm sure the models are going to get bigger and bigger and trained for longer. You're not going to be able to scale them up by a hundred thousand x at least for the foreseeable future. And in a AI paradigm where scale is the key thing, there's a question of like okay, well if you can't scale up the model during pre training beyond a certain point, then how do you scale it up further? And I think the answer is you scale up the amount of inference cost. The language models typically these days they respond very quickly. If you ask it a question, it can give you an answer in milliseconds or maybe seconds at most. But for a lot of applications you don't need a response immediately. You can wait a minute or an hour or even a week sometimes to get a response you can think of. If you ask the model to write a contract for you, a legal contract, you don't need that in, in five seconds, you can wait like a minute for a really high quality answer. And you know, thinking like if you wanted to write a novel that could take a full week. But I think, you know, if it's, if, if it's, and obviously the inference cost would be a lot higher in that case, but if it's something like writing the next Harry Potter, then that seems totally worth it to spend, you know, a thousand times the inference cost or to find a new life saving drug or to prove the Riemann hypothesis or any of these things. There's a lot of applications where it would be worth it to spend orders of magnitude more on inference in order to achieve the equivalent output that you would get from a model that's orders of magnitude bigger.

So that is a smaller clip from a larger interview. And it's important to note that this interview was in 2023. So these thoughts and theories aren't really recent. We're now in 2024 and of course since then we don't really know how much his thoughts have progressed in regards to planning since the amount of research that you know, OpenAI may have done, it could have changed in that time. And for a quick refresher on Q Star, this is the essentially details. Of course OpenAI made a breakthrough before Sam Altman firing soaking excitement and concern. And with Q Star there were two main concep and Q. The breakthrough actually also did speak about synthetic data. And I'm going to link this back in a moment, but it says Sutskever's breakthrough allowed OpenAI to overcome limitations on obtaining enough high quality data to train new models. According to the person with knowledge, a major obstacle for developing next generation models. And the research involved using computer generated rather than real world Data like text or images pulled from the Internet to train new models. So essentially right here it talks about synthetic data, which is data generated by the AI itself. And there are a variety of different ways you could do this. And that's why with this tweet from Noam Brown, the thing of him saying you don't get superhuman performance by doing better imitation learning on human data. Many are now speculating that this links back to Q, because of course Q was something about planning and of course it was something about synthetic data. So this could have been a tweet regarding that. Now, of course, the Q breakthrough. There are several pieces of information Talking about how OpenAI have likely solved planning agentic behavior for small scale models. And this is something that we've seen happen a lot recently in the industry. We're seeing a lot of agents and this is pretty much the year of agentic AI and we're about to see a lot more models be able to plan and reason. But there is also some other important points about how these models are going to be trained and how this kind of data even works. So one of the things that even confirms some of this data is of course Yann LeCun and I did cover this in a previous video before, but he actually does talk about how one of the main challenges to improve LLM reliability is to replace autoregressive token prediction with planning. Pretty much every Top Lab Fair, DeepMind, OpenAI is working on that and some of us have already published and seen the ideas and results. It is likely that Q star is OpenAI's attempt at planning. They pretty much hide Noam Brown of Liberator Poker to work on that. And of course this is something that is pretty much true because all the top labs I've looked at so many different research papers, different articles, they are all pretty much working on planning for this year. Now what's also fascinating about this is that if you've been paying attention to the space, maybe you have seen some demos, maybe you haven't, but there have been several recent demos in which we've seen AI systems being able to plan and being able to do these kind of things are making them orders of magnitude more effective. For example, this is mesa's kpu. This is their AI agentic system which is very, very effective at planning and doing tasks. You can see that currently it is being able to reason, and this is a system that is built on top of the GPT4 stack and it is able to reason very, very effective. Now you can see the different reasoning steps that it takes right here. And you can also see that just like Norm Brown talked about, he spoke about how with these kind of AI systems, if we have an AI system that is, you know, the inference is a lot slower and the inference does cost a lot higher. Even though that is the case, we do get a higher degree of accuracy. And we can see that with these models reasoning in multi step fashion, we can see that they're able to increase their ability to reduce hallucinations and perform the tasks more effectively. And this benchmark that I saw on the MESA kpu, maybe you didn't see this because this didn't blow up that much, but it was very, very, very surprising. And not only did we see demos of this, and I did an entire video explaining mesa's KPU and how the entire demo did work and some of the benchmarks that I'll likely add to the end of this video, but there was also, as you all know, Open, not open Devin, but there was also Devin, which is of course the world's first AI software engineer. And like with what we just saw with mice's kpu, you can see that Devin also has some internal scratch pad where he is essentially allowed to plan. We can see he's got the planner right here and we can see that he, you know, initially after gaining a prompt, he manages to flesh out some kind of plan and then all he has to do is go through this plan and then execute and write the code. And this is once again built on top of the GPT4 stack. Now why is this all very fascinating? Number one, we have to remember that this is something that is still in its early phases. Number two, we know that OpenAI is already workers. And of course, number three, we know that if OpenAI are working on something like this and they already are using a model like GPT5, we know that that is going to be absolutely incredible. Now, I'm not sure if GPT5 is going to be natively agentic, which just means that it's going to have these kind of planning capabilities natively built into it, or there's going to be separate versions of the GPT series, which is more than likely considering iterative deployment, but the point is that things are about to get really crazy when we do see Q star like systems that do involve planning, or systems that are able to do multi step reasoning, multi step thinking and multi step planning to achieve long term goals. So this would be something that is rather, rather fascinating and, and I genuinely can't wait to see how some of this does work, so there are many different concepts to explore here, but let me know what you think about this tweet. Do you think that this is a reference to Q Star? Why do you think that this tweet was deleted? Could it just have been something that is not that much of a big deal or was it something to do with Q Star and Noam Brown accidentally realized that he's talking about synthetic data and maybe something that he shouldn't have? Either way, I don't know, but there is a lot of speculation in the Reddit thread below that I'll leave a link to and there are a lot of few comments that do link to. Of course, planning and being able to use an AI system in order to achieve certain goals that are much more effective. And this is something that we've seen time and time again in AI, so it will be interesting to see where things go. And if you did enjoy the video, I'll see you all in the next AI update.