OpenAI'S Q-STAR Has More SHOCKING LEAKED Details! (Open AI Q*)

So, ladies and gentlemen, there has actually been a new Q star leak, although quite fishy, but there's a lot of information floating around the Internet that I would just rather put out here into a video and let me know what you all think about it because it's rather fascinating. So let's take a look at the new Q star leak and all of the information surrounding it. Okay, so this is a Pastebin and it was tweeted by someone, but I honestly cannot find the original tweet anymore. I'm not sure if the person deleted their account, but essentially what we have here is a tweet about OpenAI's Q star system. Now I do want to say, as is with any leak, you always want to be a little bit skeptical because it isn't confirmed. They didn't really say where this came from, but like I said, it's just been floating around on the Internet. But there is some actual real information that I will get to later on in the video with as to why people are giving this some kind of credibility due to other sources talking about this quite, quite as much. So let's take a look at this. Okay, so it says Q is a dialogue system conceptualized by OpenAI designed to enhance the traditional dialogue generation approach through the implementation of an energy based model. Okay, and we're going to get into that in a moment. It says, distinct from the prevalent autoregressive token prediction methods, Q aims to mimic a form of internal deliberation akin to human thought processes during complex problem solving such as chess playing, where a deeper analysis of potential moves such as leads to better decision making compared to rapid less considered responses. This model shifts focus towards the inference of latent variables, reminiscent of construction probabilistic models and graphical models fundamentally altering how dialogue systems operate. So I'm guessing that here and I will get onto this later on in the video, but you can see that they're basically changing how these LLMs think. And I guess you could say that they're moving from token prediction methods, like I said here, to something that is similar to human thought. That's basically what they are saying using these energy based models. And I will get into the technicals of that in a moment. But it says energy based model for dialogue generation. And it says at the core of Q is the energy based model which operates by assessing the compatibility of an answer to a given prompt through a scalar output. This output signifies the energy of the response where a lower value indicates higher compatibility, which is essentially just a better answer and a Higher value indicates a poorer answer. This mechanism allows Q to evaluate potential responses holistically, moving beyond the sequential predictions of tokens to understanding the relevance and appropriateness. So essentially, if you think of it like this, these energy based models, they assess the compatibility of an answer where the lower value indicates better answer and a higher value indicates a poor answer. Then essentially they talk about optimization in abstract representation space. The innovation in Q lies in its optimization processes conducted not within the space of possible text strings, but in an abstract representation of space. In here, thoughts or ideas are represented in a form that allows for the computational minimization of the EBM scalar output, akin to finding the least path of resistance in a landscape. And this process involves a gradient descent, a method for finding the minimum of a function applied to iteratively refine these abstract representations towards those that yield the lowest energy in relation to the prompt. It continues to go on. Here it says from abstract thought to textual response. Once an optimal abstract representation, one that minimizes the EBM's output is identified, Q employs an autoregressive decoder to transform this abstract thought into a contextual coherent response. This step bridges the gap between non linguistic conceptual understanding of the dialogue system to linguistic output required for human interaction. Now here's where they talk about the training of the system. They says that the EBM within Q is trained using pairs of prompts and responses, adjusting the system parameters to minimize the energy for compatible pairs while ensuring that incompatible pairs result in higher energy levels. And this training process can incorporate contrastive methods which where the system learns to differentiate between compatible and incompatible pairs, and non constructive methods which involve regularization techniques to control the distribution of low energy responses across all possible answers. And they state here, you know, whatever this is, okay? Implications for dialogue system QSTAR's approach leveraging EBMs for dialogue generation represents a significant departure from traditional language modeling techniques by optimizing over an abstract representation space and utilizing gradient based inference. And Q introduces a more efficient, reasoned and potentially more powerful method for generating dialogue responses. And this system not only promises improvements in the quality of generated text, but also offers a blueprint for future advancements in AI's ability to engage in human like reasoning and conversational interaction. So essentially why a lot of people were talking about this is because essentially there was a paper put out by OpenAI, I think it was around 2019, it was called implicit generation and modeling with energy based models. And essentially like the paper said, you know, these energy based models are a system that has a really unique way about thinking of, you know, and solving problems. And essentially when you ask some questions, well, when you try to figure out questions with the system, they don't just give you the first answer that comes to mind. Instead of they, you know, they consider a lot of possible answers but they don't think about these answers in terms of full final solutions right away. Instead they start out with like rough ideas or concepts of what the final answer could kind of be. And then they have a special way of judging how good each one of these ideas are. So you can kind of think of it like each idea has a certain amount of energy and the best ideas have the low energy while the not so good ideas have high energy. And then of course you take the idea with the lowest energy, you refine it and make it better until you can't reduce its energy anymore. And then once they have this best idea, then it's translated into words and that becomes the final answer to your question. And you know, the process of starting with many rough ideas, judging them based on their energy, and then of course refining the best one, which is essentially what makes these energy based models powerful. And you know, essentially a lot of people are saying that this might be the future of planning because it's considered very flexible and can handle complex situations where there are many solutions. And you know, traditional planning methods often struggle when there are too many possibilities to consider and they actually might take too long to find a solution or get stuck on a not so good answer. And you know, Sam Altman did actually talk about that in an interview he did with Bill Gates say that, you know, and maybe this is actually referencing this kind of model. He did state that, you know, in fact, if I can find the clip, I will put it in here. But he did say that there are 10,000 answers that, you know, ChatGPT can give you and when it gives you 10,000 answers, one of those is going to be good. And usually you'd want to get that one every single time. And, and that is something that they're working on. If I can find that clip right now, it's not actually in the presentation but eventually video.

Clearly people really want that. We launched images and audio and it had a much stronger response than we expected. We'll be able to push that much further. But maybe the most important areas of progress will be around reasoning ability. Right now GPT4 can reason in only extremely limited ways and also reliability. You know, if you ask GPT4 most questions 10,000 times one of those 10,000 is probably pretty good, but it doesn't always know which one and you'd like to get the best response of 10,000 each time. So that'll be that. That that increase in reliability will be important. Customizability and personalization will also be very important.

If I can find it, I will add it. But essentially what he's talking about is something that's eerily similar to this. Now that I think about the clip, and I'm guessing that, you know, whichever kind of system you do use, whichever tree search, you know, energy based system, whatever system it is to get your final output, essentially they're just trying to optimize whatever it is for reasoning. So these energy based models obviously do offer a promising approach to that. But the paper that they did put out, it wasn't actually for text based outputs. This one was for graphics, I think it was for video and images. So that's why people are actually talking about this, because it was something that people were deciding to tweet about. Now the reason why people were also talking about this as well is because Yann Lecun was on a podcast and he actually spoke about energy based models and a lot of people are starting to say, and if you don't know who Yannickun is, well respected scientist at Meta's Fair. So that's essentially the AI division. And he actually spoke about this on a podcast with Lex Friedman. And I'm gonna basically talk you guys through this because essentially why this leak is so skeptical is because this was on like the 9th of March or something, and then that leak was on like the 11th. Either way, it was two days after. I'm not sure about the dates, but it was two days after. So some people are stating that it's just a summarization of what Yann Lecun said. So let's just take a listen to that.

Really? What? So basically the principle is like this. The prompt is like observed variables. And what the model does is that it's basically a measure of. It can measure to what extent an answer is a good answer for a prompt. Okay, so think of it as some gigantic neural net, but it's got only one output and that output is a scalar number, which is, let's say zero if the answer is a good answer for the question and a large number if the answer is not a good answer for the question. Imagine you had this model. If you had such a model, you could use it to produce good answers. The way you would do is produce the prompt and then search through the space of possible answers for one that minimizes that number, that's called an energy based model.

But that energy based model would need the model constructed by the LLM.

Well, so really what you need to do would be to not search over possible strings of text that minimize that energy. But what you would do is do this in abstract representation space. So in sort of the space of abstract thoughts, you would elaborate a thought. Right, Using this process of minimizing the output of your model, which is just a scalar, an optimization process. Right. So now the way the system produces its answer is through optimization by minimizing an objective function, basically. Right. And we're talking about inference, we're not talking about training. Right. The system has been trained already.

So yeah, that was Yann LeCun's statement on energy based models. You can watch the full podcast with Lex Friedman. But like I said, the reason why many people were discrediting this leak is because someone also tweeted out. Someone actually just basically tweeted out a description of Q Star. And essentially what they said is that, you know, it's not actually a description of Q Star at all, rather it's an auto generated explanation by Claude of Yann LeCun's EBM project. As you see, it looks very similar indeed. And I'd be very skeptical about these claims of OpenAI's leaks as it seems to be just summarizing what Yann LeCun said. So essentially he's just saying, look, this kind of text document, it was just used by Claude Opus to summarize the document. So it's stating that it is probably fake. However, I do want to say some other things because I think it is pretty interesting. Number one is the fact that on the same podcast, Sam Altman said this a couple of days ago. I think it was 48 hours ago. One can dream.

OpenAI is not a good company at keeping secrets. It would be nice. We're been plagued by a lot of leaks and it would be nice if we were able to have something like that.

Can you speak to what Q Star is?

We are not ready to talk about that.

See, but an answer like that means there's something to talk about. Very mysterious. So, yeah, that, that is Sam Altman's statement on Q Star, which goes to show that even if this leak is fake, which it probably is, it does mean that, you know, of course there's something there, which I'd say OpenAI clearly have some kind of breakthrough. I don't know what kind of breakthrough it is, but clearly they're working on something that is There. Now, of course, the original letter was pretty crazy. It had numerous insane, crazy claims which are, you know, I guess you could say on par with, you know, threatening to humanity and. And the original claim as well. Another one of the claims as well, and this is from the Information article, this core principle of if Q Star actually does utilize EBM principles or similar methodologies to evaluate and select the optimal solution for unseen math problems, this approach could be theoretically possible and the model would effectively search through the space of possible solutions, minimizing the energy or compatibility of its answer, leveraging patterns and principles. It's seen during question. So, I mean, it's something that starts to get into the area, which is definitely pretty speculative because we don't know if this is something that OpenAI is actively working on. We don't know if they're using this. And that last paper, they hadn't updated it for quite some time. So it will be interesting. I hope there is more information about this. But like I would say, you know, even if this is fake, someone at Fair, someone at Metis AI Division also did tweet that, you know, look like the Q Star rumors align with a series of works on our latent space planning slash search areas. And then he links to seven different research papers that reference that. And then he says, you know, also check my previous comments regarding Sora, which is OpenAI's recent video model. And he states that doing planning search in a learnable latent space rather than original space has a unique advantage which reduces compound error and planning horizon. And there is strong evidence that, you know, this representation actually really helps in these scenarios. And he says that that is the area that META are looking into. And of course, you know, this is something that we've seen with of course, Yann Lecun, and now he's obviously confirming it. So I'm guessing that this is something that they're working on. And of course he also talks about this in Sora and he said, you know, one interesting component on SORA is that the video is not created by next frame prediction, but by constructing and redefining latent token sequence first and then decoded them back and doing planning search in latent space rather than original space has a unique advantage. And of course this is why they say this is beyond traditional RL formulation, which assumes fixed state action representation that may hinder optimization efficiency. So I'm guessing that they're trying to use more effective methods to provide problems to the solution, because video generation is one of those things that's really, really difficult and OpenAI, as always, have come up with a new method, but it seems that meta's hot on the heels in terms of their energy based models as well. So either way, we're going to be getting more information about these EBMs, because it seems like META is working on that, with Yann Lecun and his team confirming that. And even if opening I don't publish a research sometime soon, we know that META are going to publish there. So either way Q whatever it is, it's definitely kind of fascinating, definitely kind of interesting. And all we can do right now is of course not speculate too much, but we can just wait until we have more information.