OpenAI Might Be In Big Trouble...

So OpenAI actually might be in some trouble and that is related to their recent model. Now, I personally think the recent model is really decent, but many people don't think so. Currently there is this idea that GPT 4.5 simply isn't that good of a model and apparently OpenAI is destined for failure. So without further ado, let's dive into is OpenAI in trouble and are they slowly falling off a cliff? So one of the recent articles that I came across that was recently published by Futurism was this one right here. It says OpenAI may have really screwed up with GPT 4.5. It says that the hype is dying and this is in reference to the fact that GPT 4.5 was meant to be a frontier model. Now, currently it's not a frontier model in the sense that it's the best in all categories, but I think maybe the marketing surrounding the model didn't do it any justice. One of the things that they actually did was stating that GPT 4.5 is the largest and most knowledgeable model yet, while managing the expectations and cautioning saying that it's not a frontier model. And so I think there was definitely some marketing issues like I was talking about before because, you know, they're essentially trying to have it both ways. You can't call it the largest and most knowledgeable model whilst also stating that, look, it's not going to be the best model. Now I've done a video on GPT 4.5 that you guys may want to check out and but this article has its point. So they say that probably because the company knew what the public response was going to be neutered, they actually reference AI critic Gary Marcus calling the LLMs a nothing burger to an anonymous expert who told Ars Technica that the model is a lemon. And it appears that the amply hyped new model is seriously lacking in the type of juice that made the original ChatGPT or its follow up GPT4 be become enormous cultural and financial touchstones. So in this article where they're saying that OpenAI may have screwed up with GPT 4.5, I on the wider spectrum don't think this is that much of a bigger issue because I do know what OpenAI are trying to do and I'll talk about that later on. But I do think that overall it doesn't look good from an outsider's point of view because it's like they were hyping up new models and then they released GPT 4.5 and the public reaction is quite bad. Now we also need to take a look at GPT 4.5 and what they were saying before. If you remember what earlier articles from places like the Wall Street Journal and Bloomberg were saying, they actually had a lot of information regarding leaks. So they spoke about how the project, called GPT5 and codename Dorian, had been in works for more than 18 months and and intended to be a major advancement in the technology that powers ChatGPT. The closest partner investor, Microsoft, had expected to see the model around mid 2024. I remember that would have been seven or eight months ago now. And apparently OpenAI actually conducted at least two large training runs, each of which entailed months of crunching huge amounts of data with the goal of making Orion smarter. And every single time they did this, apparently new problems arose and, and the software felt short of the results researchers were hoping for. That's what people close to the project were saying. And they say at best, Orion performs better than OpenAI's current models, but hasn't advanced enough to justify the enormous cost of keeping the new model running. And I think this is pretty crazy because we've got a situation on our hands. Apparently GPT 4.5 in this article says that this was actually meant to be called GPT5. And I'm guessing that if we deduce that they were talking about a model that was quite expensive, but the model didn't live up to capability capabilities, then maybe they actually just changed GPT 5 to GPT 4.5. Because you have to remember that, you know, GPT 4.5 was, you know, codenamed Orion. And apparently now this is what they were saying about GPT5. So my best guess is that maybe OpenAI did actually make GPT5, which is currently GPT 4.5. And I'm guessing that since GPT5 was probably a disappointment in terms of not getting the results that they were expecting, maybe this was like, look, since it's not that big of a deal in terms of the performance you're expecting, what it's going to be able to do. I'm guessing we should probably call this model GPT 4.5. And all of this data, when they're talking in quotes about GPT5 here, this actually looks exactly like what probably happened to GPT 4.5, which is the model we currently have now. And it is really, really expensive to run if you're not in the. I think it's protein, you actually don't even have access to it. And it is a good model, but certainly I don't think it's worth the $200 a month for marginally better performance. Now, of course, you can see right here we've got GPT 4.5. So this is where Chubby says, judging by the mood, GPT 4.5 is the first big failure of OpenAI. Too expensive, too little improvement, and often inferior to GPT 4o in creative answers in Community test. And I think that this statement is half true and half not because of course, it's probably the first big failure of OpenAI, but I do believe that it's not inferior to GPT4O, and I'll get into that in a moment. But Gary Marcus did actually quote, tweet this by saying, look, I've been trying to tell you guys about this for some time now. Gary Marcus said here that it's a big surprise only to you and the many other OpenAI fans who refuse to listen to me when I patiently and endlessly explained that this would be exactly what would happen for the rest of us. This was essentially inevitable. Now, Gary Marcus has been, you know, quite critical of OpenAI for some time now. He's openly spoke about, you know, the things that he believes OpenAI has done wrong in terms of them, you know, not having any moat, of course, them having major problems. Here you can see he dives into some really, really key details on why he believes that OpenAI is in serious, serious trouble. He says, of course, they've got a brand name which is good, but. But GPT 4.5 is hugely expensive, which I do agree with. And he says, even so, it offers no decisive advantage over competitors, which means they had zero moat, which is, you know, on the grand scheme of things, it is actually quite true. Scaling them hasn't gotten them to AGI. I do think that that is true, but at the same time, AGI will come later. The GPT5 project was a failure. Some people are wondering, is that all they've got? The deep seq war led to a price war. Of course, I will speak about this as well. And OpenAI is of course losing money. And one of the key things he actually does talk about is that many top people have left. And I do think this is unfair, fortunate, because these people are essentially multi, multimillionaires because, of course, they've had equity in OpenAI. And I do think that some of them maybe just want to pursue creative projects in AI rather than, you know, working ruthlessly at one of the most competitive startups. And we can actually see what Gary Marcus says in a recent interview where he talks about the fact that OpenAI might not be doing so well now. It's of course always, I'm going to always add this caveat is that Gary Marcus is a known critic of the AI industry. He's definitely critical of a lot of things that I don't think sometimes is warranted. Like sometimes he does make good points, but sometimes it does seem like the statements that he makes don't come from a place of genuine criticism, but rather. I'm not even sure where it comes from, but just wanted to add that because it's important to understand viewpoints.

I think that OpenAI is highly overvalued. I think we just saw their business model sort of blow up yesterday or over the last few days with Deep Sea basically giving away for free what they wanted to do, charge money for. Also, I think that Deep Seek is more open than Open AI and that's going to be attractive to talent. I don't think things are looking great. I think $157 billion valuation is hard to bear out when you're losing $5 billion a year or thereabouts.

Now, this is why I said I didn't really agree with Chubby's statement. I agreed with half of it, but the other half I just didn't really agree with that much. So basically the reason I said that is because Carpathy did some community testing and and 80% preferred the answers of GPT4O instead of GPT5. But the only reason I said that this doesn't make sense as a valid test. Whilst yes, it is kind of a test. The only problem wrong with this is that this is a very small sample size in the sense that you only have five questions and five questions that Andrej Karpathy has asked isn't a large enough sample size to be judging a model. It's like me saying 5 I've got flannel like 5 questions I've asked in AI model pick between A or B. And the reason I stand by that statement of just stating that look like if I had five prompts that I showed and people just preferred one model over the other. The reason you use other methods is because the larger sample size you have, the more holistic view you're going to have. So for example, let's say we looked at this, this was actually released while I was making this video and we actually see that GPT 4.5 managed to top the LMSYS arena. So we can see right here the GPT 4.5 preview was an area where OpenAI actually managed to get themselves to be number one. And this is pretty crazy because it was only a week ago that we had Grok3 become number one across all of the categories. And here we can see that we had 3,000 votes, which is of course a much bigger sample size in terms of having users talk to the model and view the model in terms of their actual responses. And considering that this is something that OpenAI have essentially said is going to be number one, clearly we can see that this makes sense. So if we want to look at other categories, we can see that in coding, math and hard, we can see that it is number one, which is genuinely quite surprising considering other models are also pretty decent. So I don't know what it is about the vibes of the model, although that is a very vague term. The vibes clearly are real. And despite some user testing, even by the goats like Andrej Karpathy, it still doesn't, you know, account for the rapid testing that we can do here, there, where, you know, you can see exactly, just exactly how these results are. So as well, with Star Control, we can see the GPT 4.5, it actually manages to beat out the other models by a large jump. So we can see that this is a pretty decent jump in terms of overall ELO rating, which is rather surprising. Now, remember the Deep Seek thing, which as Gary Marcus said, is something that kind of, you know, messed up OpenAI's launch. I do think that certain industry partners are leaving OpenAI due to them being able to use open source models. One other thing that I don't remember who said this, but they basically said OpenAI doesn't have a moat in the sense that if someone can offer a foundation model for the same price or cheaper, then individuals that work with them would work with other companies that are able to do that and have it completely open source or basically for free where they can host it themselves. And basically we recently had that confirmed when figure the company that produces humanoid robots. In an interview, the CEO actually said that, look, everything that we are currently using now is open source. And they actually also terminated their agreement with OpenAI recently because of course they had a breakthrough and they just believe that they are the very best at.

Embodying humanoids with AI trajectories to go and grab something, right? And that doesn't sit like in the LLM. There's no idea what that looks like. There's no robot data in there. So yeah, we basically train, we use open source models today, but with our own models internally and own data collection, we do internally on robot, and we basically build basically the foundation models yourself ourself. And we've been doing that almost exclusively, basically ourselves for a year. And we think we're the best in the world at doing this on robotics. I don't think there's anybody who's demonstrated better AI learning on robots than we put out publicly.

And now, if we're going to talk about other things, we also have to talk about deep research hallucinations. Now, this video isn't an attack on OpenAI, but it's to put a critique on certain things and ask the question, are OpenAI genuinely in trouble? Now, Basically, Deep Research is, you know, OpenAI's flagship tool that I've seen a lot of people talk about and they say that the tool is really, really good. But there is currently a flaw in Deep Research that I don't want to say makes it unusable, but in certain aspects, I would argue that what would be the point of using this if this is, you know, the case? So basically, this person said that this is the worst hallucination I've seen from a state of the art LLM for a while. And I think that this is something that, you know, is basically true for all LLM. So this isn't just, you know, OpenAI's fault. This is something that across every single research tool, so it says Deep Research made up a bunch of stats and analysis while claiming to compile a data set of thousands of articles and supposedly gather birth year info from, you know, from each author from reputable sources. But, you know, they did the research and none of this stuff is true. So this is something that is of course a big issue because you have to ask the question, and someone on Reddit posted this, and I will say that this is probably the best way to put it, because this, you know, is incredibly dangerous if you have a research tool that you know, gives you hallucinations, because this renders the entire point of deep research rather pointless because you will need to go and personally verify every single source to confirm the accuracy. And I think that that is true. What is the point of having a research tool that does research for you? But you need to go ahead and confirm that those sources are legitimate because you might as well have just done the research yourself. Now, of course, maybe it saves you time or whatnot, but isn't the point to have everything solidified and verified so that you can say, okay, I've got this research report. I now only need to Skim through this and I've got the main details. And like I said, this is something that goes for all deep research tools. I would like to see an update where they actually have the sources and they have like an AI that's able to click the link, it's able to verify the source. So, you know, there's like green ticks on verified, verified source. Or maybe if it doesn't know about certain sources, it has some kind of, you know, not sure about this source or whatever, because that is, you know, something that, you know, renders this, you know, quite useless, I would say. And this actually makes me a bit apprehensive to use it, because if I'm using an AI tool to do research and then that research comes back and there are some hallucinations, that is going to be, of course, a big issue now. Remember ages ago when they were stating that these models could begin to automate large portions of the economy and that they believe that companies that train the best 2025-2026 models will be too far ahead for anyone to catch up in subsequent cycles? Was it just hype? Because right now we've got a situation on our hands where even people that are training the biggest models, they aren't that much ahead than the competition. And. And it seems like things are currently changing on their hands. And it was Ilya Sutskova that said, you know, pre training as we know it will end, of course, computers growing, but the data is not growing. So there will be new innovations on this side. And I'm actually quite bullish here, the fact that these large models are not producing better reasoning. I think that now the real work in AI has started because the easy pickings have gone like before. It was just, okay, just make it bigger and it becomes smarter. But now they have to think about new innovative ways, which means we're actually probably going to get some really smart systems in the future because people aren't just staring down one tunnel, they're exploring multiple solutions. So another thing that I did think was a really, really bad look for OpenAI. And like I said before, I'm not bringing this up to critique OpenAI, but does this mean that OpenAI are in trouble? And this was something that, you know, I don't want to say it upset me, but it definitely did throw me off because I didn't expect this at all. So this right here is Frontier Math. And essentially there was an AI benchmark that was, you know, they basically spent a lot of time on this benchmark and they basically said it was a math benchmark. And they basically said, you know, this benchmark is something that if any AI is able to solve any of these problems, this AI is ridiculously smart. So basically, OpenAI commissioned Epoch AI to produce 300 advanced math problems for an AI evaluation that form the core of the Frontier Math benchmark. And the only problem was that OpenAI secretly funded this benchmark linked to the O3 model. So OpenAI, they basically paid for this benchmark to get created and they actually had access to the problems and the solutions of the test set. So that was something that the individuals who were creating the benchmark, they actually didn't disclose that until O3 was released. So there was this huge debacle of Frontier Math and this was something that basically put a stain on that benchmark. Because one of the key things with O3, whilst it does do well in terms of certain benchmarks, one of the benchmarks that was really surprising because the jump was from 2% to 25% and people were like, whoa, how on earth did that happen? Basically, if that benchmark is invalidated, people are going to now start to question the integrity of other benchmark scores. And I'm also somewhat doing the same because it's now like, okay, there is an incentive for not opening eye to lie, but for them to maybe just game the benchmarks a little bit so that their models can seem even better. I mean, these companies are trying to attract investment and of course doing so would definitely boost that. So Gary Marcus also talks about this. It says OpenAI's impressive results on O3 should be taken with a huge grain of salt. They had access to the problems of solution. We don't know what they trained on, we don't know what problem tailored validation or data augmentation techniques might have been built in. And the problem is right now, we're still waiting on this. Like, I'm truly still waiting on this, is that Epoch AI have not been able to independently evaluate the model using their holdout set. And the holdout set is basically a bunch of questions that they haven't simply shown the model yet. So for me, I would like to see different benchmarks where OpenAI's models haven't seen it before and they're able to still reason well. Now, I will say that I'm not completely bearish on OpenAI at all. I still think that the company is doing amazing things and I still think that they are going to do well in terms of the fact that right now they are focusing on asi, which I'LL explain in another video, but the question is, did OpenAI screw up by releasing this model? Did GPT5 not live up to expectations? I'd love to know your thoughts and theories, but my honest opinion is yes, GPT 4.5 or GPT 5 may have been a flop, but I would say that it is the case that this model is number one, more of a Vibes model. Number two, it's not a reasonable model and number three, the new paradigm, the one which we're slowly ending has give birth to. Of course the new paradigm which is test time reasoning or test time compute in which there is still a lot of areas that are currently unexplored. So I would say OpenAI still have a lot of Runway because they are ones that, you know, pioneered that area and I think that they will still lead due to their connections and partnerships. But of course there is now competition. We have other countries like China on the case of being very very hard to deal with in terms of the competition. So it will be interesting to see what goes on with these other companies. And with that being said, do not forget to leave a like do not forget to subscribe and I will see you guys in the next one.