Did OpenAI FAKE AGI ? (Controversy Explained)

So yesterday was a monumental day in AI because many were speculating that OpenAI basically achieved AGI and many that worked at OpenAI also made this claim. Now, recently we had a very interesting discussion on Twitter that claims that this AGI benchmark demo that we saw isn't what we think it is. So it all started when we got this Tweet. I'm getting GPT4 vibes from this announcement when in March 2023 we had some massive progress. And how did they get such a jump in ARC AGI? Then this person, Niels Roj, responded saying they did this by training 75% on the training set. And then this refers to something the OpenAI showed where they shared that they trained on the O3 training set and they tested on 75% of the public training set. And it says that they have not shared more details. Now. Now this is basically referring to this, which is on the Ark AGI webpage where they look at the O3 results. And it says here that it says, Note on tuned OpenAI shared that they trained the O3 we tested on 75% on the training set. And it says that they have not shared more details and they have not tested the ARC untrained model to understand how much of the performance is due to the ARC AGI data. Now then we also had a notable AI critic, Gary Marcus, who said, wow, if this is true, this raises serious concerns about yesterday's announcement. And this is of course referring to the fact that if the model was, quote, unquote, trained on the training set, then it would have seen some of the examples. That was the presumption. Now, interestingly as well, we also had this. Someone also said that they had an Altmanian slip in that 03 presentation. The engineer said that they specifically targeted the ARK benchmark. And Altman immediately corrected him, saying that they didn't do anything special. And Altman has no need or reason to say anything unless it's a lie, also has nothing to say. Now, essentially what he's referring to here. I'm gonna show you guys the clip in a second. But he says Altmanian slip, which is quite like the word Freudian slip, which is like where you say something that you're not meant to, and it is apparently a verbal or physical error that incurs when an unconscious thought infers with what someone was meant to say. Now take a look at this small clip here because I do think this is a little bit interesting, but as this video evolves, you'll start to see that the claims don't hold much ground.

You know it's also a benchmark that we've been targeting been on our mind for a very long time. So excited to work with you in the future.

Worth mentioning that we didn't we target and we think it's an awesome benchmark.

We didn't go to SPIS chose this.

Is just the General O3 but yeah really appreciate the partnership. Yeah this was a fun one to.

Do and that clip was basically referring to where the engineer basically said that they targeted the benchmark but Sam Altman was stating that they didn't. Now one of the co creators of the ARK AGI benchmark has actually responded to this and he says raising availability on this note we added to address the arctuned confusion said that OpenAI shared that they trained the O3 we tested on 75% of the public training set and he said this is the explicit purpose of the training set. It is designed to expose a system to the core knowledge priors needed to beat the much harder private evaluation set. The idea is each training task shows you an isolated single priority and that the evaluation set requires you to recombine and abstract from those priors on the fly. Broadly the evaluation task requires utilizing three to five priors and the evaluation sets are extremely resistant to just memorizing the training set. And this is why O3 is impressive. Now he also stated here that the other reason that he doesn't think that this matters is that it's nearly certain that ark's training set and the evaluation set, mind you, was included in the pre training for GPT 3, 4 and 4.0 because those have been hosted on a public GitHub repo since 2019. And the craziest thing about that is that those other models don't seem to perform that well on the ARC AGI benchmark, which means that if it was in the training pre training data then that means that O3 would likely actually.

Be a different breakthrough.

But because if that is the case then we should have seen those other models like GPT4 or 4.0 actually perform well on those benchmarks. Now someone else here also said where's your result without this data set? What's the scientific value of this number? Have you explicitly call out this limitation? What do we learn from this? And he said that we want to do the ablation study. We just didn't have enough time before the announcement. And I still don't think this takes anything away from the fact that O3 is a uniquely capable system and this is Something that I do agree with when we do take a look at what O3 is capable of, which I will mention later on in the video. Now you can see here that this is where Francis Chollet also decides to respond. The other individual who also created the ARC AGI benchmark says that this is the point of the training set though to train your model on it. It would have been more impressive if the model had no prior exposure to the ARC data, but the fact that the model was ARC adapted via the training set absolutely does not invalidate its score. So you can see here that the creators of the benchmark also do not think that this invalidates its score. Now of course we are going to have notable AI critics like Gary Marcus that are going to say stuff when it comes to criticizing AI. Like I've always said, I always do think that it is important to have a even discourse in AI because that is how you actually make progress with rather than just being in a hype bubble. That is how you actually look at your flaws and then of course decide to see, okay, if this isn't real reasoning, how do we achieve the real reasoning and get to true AGI? Now of course there is some more information because a lot of people really have missed the mark with this one. Speaking of AI, have you actually wondered.

How large language models like the ones we talk about consistently actually work under the hood? Well, this is where today's sponsor Brilliant comes in.

Brilliant is where you learn by doing.

With thousands of interactive lessons in math.

Data analysis, programming and AI.

And what I personally love about Brilliant is that they break down complex concepts into intuitive hands on problems that you can build real understanding from the ground up. Now their courses on AI and programming are particularly fascinating. You'll start building programs on day one with their built in Python editor and they have an excellent course that lets you peek under the hood of LLMs to understand the concepts powering today's AI technology. Instead of just watching passive lectures, you're actively solving problems and building intuition, which.

Has been proven to be six times.

More effective for learning. And what I especially appreciate is how Brilliant helps you develop a consistent learning habit.

In just a few minutes a day.

You can actually level up your programming and AI skills, whether you're commuting, taking a break or are waiting for one of your AI models to train.

Now if you want to try it out, the first 30 days are completely free when you sign up with my link brilliant.org AI Grid or alternatively you can either scan this QR code on.

Screen right now or just click the link right there.

You can see Here, Rune, an OpenAI employee or member of technical staff, says that, oh man, they trained on the train set.

It's all over now.

And he's basically using sarcasm to say that this is not the case. Now when we actually take a look at what's going on here, we can take a look at what this individual here who is conducting research at OpenAI says he stated, okay, that the model we used for all our O3 evaluations is fully general. A subset of the ARC AGI public training set was a tiny fraction of the broader O3 training distribution and we didn't do any additional domain specific fine tuning on the final checkpoint. And this is in response to a tweet from someone else that works at OpenAI and who says to anyone wondering if the high ARC AGI score is due to how we prompt the model? Nah, I wrote down a prompt format that I thought looks clean and then we used it. That is the full story. So you can see here that the individuals working at OpenAI have clearly stated that they didn't do any additional domain specific fine tuning on the final checkpoint. I know that a lot of people would have thought that, okay, because it says tuned, so that means they fine tuned the model on that benchmark. But this is not the the case.

We could also see someone here, say, were anyone on the team aware and thinking about ARK and ARC like problems.

As a domain to improve at when.

You were designing and training O3? And they said that this is the distinction between succeeding as a random side effect, one succeeding with intention, which is of course, you know, potentially referring to the fact that you can target a benchmark, which is sometimes what tends to happen when people are trying to build these LLMs. You can see here, he says, no, the team wasn't thinking about Ark when training 03, because people internally just see it as one of the many other thoughtfully designed evaluations that are useful for monitoring real progress. And we can see someone also said here that, you know, what does tuned mean here? So he says it's a strange way. This is another, you know, researcher at OpenAI says it's a strange way of denoting that we included ARC training examples when we were training O3. And it isn't some fine tuned version of O3 though, it's just O3.

And this is of course a very.

Very big difference because if you fine tune something on, you know, certain pieces of data that is completely different than to just having it in its training data. So those are Two different things. And that is definitely a very strange way of denoting that, which I think brings some confusion. And of course I'm guessing that the team internally weren't really trying to target that ARC AGI benchmark. It was just an unintended side effect. Now whether or not that's true, you're going to have to be the judge. Now with all of this information, with some people stating that yes, they did the benchmark, no they didn't because of the examples, yada, yada yada.

But, but we clearly see that, you.

Know, we have both creators of the benchmark basically saying that, you know, this doesn't invalidate its score.

And of course that the evaluation sets.

Are extremely resistant to just memorizing the training set, which is why O3 is impressive. I still think that even if, okay, which I don't believe anyways, that even if this was just pure memorization, which like I said, I don't believe, O3 is still a remarkable system because of this. So this is the Epoch AI Frontier Math benchmark. And the previous state of the art systems managed to get just 2%. You can see here, O3 manages to get over 25%, which is pretty incredible when we start to think about what this means for the system. Now for those of you who don't realize how big of a jump this really is and how incredible this result is. Terence Tao, which is often considered one of the greatest living mathematicians who is a Fields medalist and a professor at the University of California, ucla, this guy, okay, said that this benchmark, you know, these are extremely challenging and I think they will resist AI for several years at least. This is what he said when he looked at the benchmark when it first came out. And we can already see that O3 is a system that manages to get 25%. Remember, this is OpenAI's second iteration of the model and it's able to do that well on a really challenging benchmark. And the craziest thing about all of this is that when we look at this benchmark, it's a thing that is really resistant to memorization. But because there is not much data out there, take a look at what they actually said in the video they.

Posted to their channel.

And I think most people did miss this because the video has, I think around 300 or 200 views.

Existing benchmarks for complex scientific reasoning are close to saturating. We need something new to be able to tell how much progress we're making towards expert level ability. And that's why we've Built Frontier Math. We worked with over 60 mathematicians worldwide, professors, IMO writers and fields medalists to produce hundreds of original extremely challenging math problems.

These are actually genuinely hard problems. One of the things that will be, I think difficult is a lack of training material like pretty niche. It's probably not that well documented.

Yeah. So I took a look at the 10 problems you sent. I think of the. I could do the analytic number three ones in principle and then the others I don't know how to do, but I know who to ask.

The problems of Frontier Math were range in scope from Olympiad style puzzles to research level challenges and span across all the major fields of mathematics. They're also beyond the capabilities of current AI. We've tested the most advanced systems available and each succeeded in less than 2% of these problems.

In the near term. Basically the only way to solve them, short of having a real domain expert in the area, is by a combination of a semi expert, like a graduate student in a related field, paired with some combination of a modern AI and lots of other packages and things like that.

Although Frontier Math measures advanced mathematical reasoning, we use problems with closed form answers like integers, so they can be automatically verified without significant advances in the math libraries of proof assistants. This is a challenging design constraint for the mathematicians writing these problems whose research is almost entirely conveyed through proving theorems. An AI capable of solving this challenge will have a drastic effect on mathematics.

So that right there should show you guys that even if there are maybe.

Some minor issues, okay, and very, very.

Minor issues with the Arc AGI benchmark, a system managing to get 25% on the benchmark.

When we saw Zero1 Mini and Zero1 Preview get around 1%, we know that this system is a drastically different system in terms of the capabilities. Now whether or not you agree that is completely up to you. There is always going to be AI skeptics, which I think there should be because it's, it allows for criticism and.

Like I said, it allows for us to achieve real progress outside of the bubble of hype. But I wanted to make this video to clear up the fog in the air because there was a lot of.

Discussion about whether or not this demo.

Was faked, whether the benchmark was real.

And this video should just provide you.

Guys with some insight with as to.

What this benchmark truly measures. With that being said, if you enjoy.