China DROPS AI BOMBSHELL: OpenAI Is WRONG!

They're basically saying that, look, OpenAI have got this world model wrong and you guys need to pay attention to this video. So of course, paper here, yada yada, yada. But what is the video that they made? So they made this video, okay? And it says, how far is video generation from a world model, a physical perspective? So this is coming out of this company that you can see right here, called bytedance. So the video starts as follows. So if you click the video, you can see that it actually complements Sora. It basically says that, look, SORA can create incredibly realistic videos to which it can. Everybody has seen Sora. We know exactly what this system is capable of. It's capable of incredible stuff. But then this is where they start to get a bit skeptical, okay? And trust me that when you actually see what this research is saying, it's not just about sora. It's about the underlying technology behind it and how certain things work. It says video generation models theme, okay, seem the keyword there being seen to generate the world. But let's take a closer look, okay, so let's continue watching. And this is essentially just referencing what OpenAI have put on their blog themselves to. So this isn't taking anything out of context, but this is exactly what OpenAI have said on their blog post themselves about Sora. So pay attention. It says from OpenAI, our results suggest that scaling video generation models is a promising path towards building general purpose simulators of the physical world. That is what OpenAI themselves have said about Sora. So they're basically stating that they are going to be using potentially SORA or some variation of that to get to AGI because of course you need to be able to simulate the physical world if you want to act and, you know, essentially exist in the physical. But things might not be as they seem. So let's take a look at what this research continues to state. So let's continue to watch this video because this video is super, super fascinating. So this is where they say or not, does SORA really understand the physical law? Okay, remember, it's not just about sora. It's backfilled. SORA is potentially a gateway towards AGI. And if you have something that can generate the physical world in a really cool way, that means that you really do have a world model. And a world model is essential for developing artificial general intelligence systems. So here you can see that they state that does SORA really understand physical law? And this is essentially a video from the Sora model where sometimes it hallucinates and it Messes up. So this is where you can see like this glass just jumps up and it just does something random. And they're basically saying that look, the answer is no, these are not world models. They do not simulate physical reality. It is something completely different. But this is where things start to get interesting. Okay, so it says we conduct systematic study on this problem in synthetic scenes at scale. So things are going to get a little bit complicated just for 60 seconds. But just stay with me, trust me, it's worth it. So they said that they have a 2D physics simulation engine which is going to be used to generate the synthetic videos. Okay, so that's pretty simple to understand. And, and the video generation model is trained to predict future frames, quite like how traditional models are doing the same thing. Okay, so this is nothing that's out of the ordinary. But video generation models are importantly trained to predict the future frame. So it's basically predicting the next frame or whatever's going to happen next. And now this is basically where they're saying that, look, they have unlimited data scaling, which is really cool. And now also this allows them to create and use the data to test different settings. So you can change the speed, you can change the velocity, you can change different things. Okay, and, and what this allows you to do is this allows you to have things that are inside the distribution. And when they say inside the distribution versus outside of distribution, all they mean is that like, okay, let's say we take a look at things that are inside the distribution. We can say that this means that it was on the test data versus if it wasn't on the test data. How do these models perform? And if models cannot perform well on stuff that is out of distribution, it is quite likely that they are very unable to generalize well. And this is something that we really do need. If systems could only perform well on in distribution, it means these systems are only performing retrieval. Think about it like this guys. You can drive your car to another state even if you haven't seen the roads before. But AI systems really struggle with those tasks because they are out of distribution. So that's why this kind of thing is really essential for getting AI. So now that you've understood this concept. So of course you look at here and this is where they have this graph which is like combinatrix or something like that. And then they basically look at someones that are trained, trained, and then, you know, half of those are tests. So this is where we look at their observations. Now remember I said that we have in distribution outside of distribution and the combination of both things. So in distribution they said that they have perfect recall because of course, remember this is stuff that it was already trained on. So if it's in the distribution, if it's in the test data set, if you've seen a model that has been trained on 5,000 million cars flying, you're going to look at cars flying if you prompt it, and then you're going to be able to see that it's able to record that perfectly. However, when we look at stuff that is outside of the distribution, things don't perform well. And even in the combination we see that there is this kind of scaling law that is a little bit different. So let's take a look at exactly what is said and when you see exactly how this works, you'll understand that why these failure modes are truly significant to the development of AI. So it says we observed many interesting failure modes, so let's take a look at what they observed. Okay, so we have the real versus the generated. And this is, is where things start to get crazy. Now they post first post these without any context, so you can't really understand what's going on. But as soon as they give you this information, it shows you why these models are failing to generalize out of distribution in scenarios. So it says the key message here is that the model has perfect in distribution generalization, but fails to generalize in out of distribution scenario. So let's take a look at what they've done here. So you can see that in distribution there's basically no kinds of error. These errors are super small. But, but outside of distribution, when you're trying to prompt it for things where it hasn't potentially seen before, we can see that the error is large and random, which is not good at all. This is not ideally what you want to see with a system that uses this kind of architecture, because this is going to be the system that most companies are using to try and get to AGI. Now, of course, OpenAI could be working on something different, but based on public information, we understand that things aren't looking good so far. So you can see here it says that we train the model with multiple scenes combined of multiple objects and procedures. So, so you can see the real simulation and the generated, which is. It looks really cool. Okay, now take a look at the scaling laws for the combination of these. Okay. And it says with larger scaling laws and model data coverage, the ratio of physically plausible videos increases significantly, which does make sense. Okay, but this is the thing that gets me. Okay, this is the thing that when I saw this I was like, okay, maybe these systems aren't as smart as they saw. And some of the people that are truly speculative on AI have a real point. And it seems that every single month more information comes out and it seems like these guys are getting more and more credibility. It says here that videos are generated by case based retrieval and replay, not by simulating dynamics. I'm going to say that one more time, guys. It says the videos are generated by case based retrieval and not by simulating dynamics. That means that these models don't understand the actual physics. The only thing that these models are doing is retrieving the content that they have been trained on. And you have to understand that, guys, if that is actually true, if the research what they're saying, if their conclusions are accurate, then that would mean that these AI systems are not as great as we think, which has severe implications. So if these videos are just retrieving and not actually simulating, then it means that we would need an entirely new architecture to get to AGI. But if you don't believe what they're stating, take a look. There's one example that might just blow your mind. So it says, for instance, okay, we train the model with scenes, I didn't even get to read that, where the ball goes left and right at fast speeds. Okay, so you train the ball to go left and right at fast speeds. And you've tested it with low speed towards the right. But the model produces the following result instead of actually going towards the right. So you've trained the model to predict what happens when this red ball is here. So this is the training data. Okay, so you train the model with scenes where the ball goes left and right at fast speed. Remember that the ball goes left and right, right at fast speed. Then you decided to test with low speed, the ball only going towards the right, but what do you see? The ball then goes towards the left. Okay, which is just absolutely insane because it only saw what was in the training data. So if you want it to go to the right, this is essentially where you can see this happens because the model record the data set cases, which actually has cases where it's going towards the left. Now this case example is pretty confusing. It's not that confusing, but there's a better example that they do have. And, and it basically says that the video is generated by referring to the similar training data, not by the physical rules. And that's even if you use an accurate world simulator. So whatever they used to generate the Physics initially was accurate, but the model that they're using, okay, when you train a model, even if it's on accurate physics, like our real home world, just because it's seen that training data doesn't mean it's fundamentally understanding exactly what's going on. There's another example that you need to see. It says, we even discovered the generative models preference order over different attributes. Take a look at this, because this is just absolutely insane. This is when it's going to click into your brain that, oh, my gosh, maybe these models are actually just advanced retrieval mechanisms instead of actual intelligent beings. So it says when doing retrieval, the model has an internal priority over different attributes. Okay, so this is pretty crazy because it basically says that, look, these models are prioritizing certain things when it's retrieving. And it basically means that you can kind of predict what the model is going to do based on what you've retrieved or what you've stored in your database. So basically, the model will look at first the color, then the size, then the velocity and the shape. It doesn't even look at the physics at all. It doesn't really care. It just looks at those things. So this is the example that blew my mind, and I was like, oh, my gosh. So it says that, for example, in the training data, okay, all the circles are red while all the squares are blue. So this is the only training data that they have. All the circles are red and all the squares are blue. Pretty cool, right? Okay. Very simple, very easy to follow. So let's take a look at what happened when they use the model. So when the test starts with a red square, which is, of course, what they didn't see in the training data. Remember, in the training data, the only red object they saw was a circle, okay? And of course, the only square that they saw was a blue square. So you can see that this thing. Let's take a look at what happens when you have a red square. So if we have a red square and we decide that, okay, we want a red square, and it actually turns into a circle guy. That is absolutely insane, you guys. They've trained it, okay, to use the physics, of course, of this red circle and this blue square, but the training data favors the color of the object, so much so that when it looks at a red square going left to right, it actually transforms into a circle instead. Meaning that it's not predicting the physics of objects moving left to right. It is predicting the fact that a red circle is moving from left to right, which Means it's not focused on the physics, it's focused on the color of the object, which is absolutely insane. So this is why we get hallucinations. And this actually makes sense now, guys, this is something that of course is out of distribution. You can see, we got this, this is something that we never saw before. And you can see exactly what happens when you have something that is out of distribution. So this is something that is absolutely crazy. And it says in retrieval, okay, the color determines the shape instead of the opposite. Which has remarkable implications for other video models that use similar architectures. So I'm not going to lie, guys, this is pretty incredible research because it basically shows some things that we may not have known before. And here's the takeaway that they do say, okay, that this is where the video generation model behaves well. Only in distribution. So only on training data it's going to perform well. Out of distribution is poor. Which means if you're trying to prompt something in a text to video model, the reason why you can't get good results is not because your prompting strategy is probably because the model hasn't been trained on that. So it's probably simply because it just doesn't have enough training data. And when it doesn't have enough training data, it performs poorly. And it says combinatorial generalization is possible at scaling and it generates mostly via case based retrieving and combination, but not understanding of the physical law. So basically, if you want these models to actually predict the physical law, you cannot use this architecture at all because this is not what they're doing. They are actually retrieving it. Now this completely incredible because this means that there's a series of knock on offense. So of course they have the paper. Okay, and this video was super, super helpful because it does have the informations. But you can see right here, guys, that this means that like, you know, right here we have the situation where it means that, okay, these things are just completely not okay. And of course maybe there's going to be some research that comes out after, but it seems that scaling of video generation models, you know, as a promising path towards building general purpose simulators of the physical world. Apparently this means that this is not the truth. And I think it is due to the generative architecture. Of course, maybe I'm at my depth even just by saying that. But guys, this video is a complete bombshell because so many people have been critical of the generative tire landscape. And this is another thing that is just like, whoa. Okay, this is pretty insane now. Crazy Even crazier, okay, is that someone commented on Twitter and said, have you found a solution to this problem? And you can see that they've responded saying, unfortunately, we have not. Actually, this is probably the mission of the entire AI community, which is absolutely insane. Okay, now, of course, some AI critics have commented on this, and of course, what would we be doing if it wasn't, you know, a critic video. When we're talking about Gary Marcus, you can see here, he says, first P marker confirms deep learning is hitting a wall. I mean, Gary Marcus has been saying that for quite some time, but he says, this is exactly what I've been telling you about AI since 1998, and over and over again, the difference between within and without of training, space generalization is all, until we solve this, we won't get to AGI. So basically trying to say that, look, okay, if we don't have a system that can generalize out of distribution tasks, we are not going to get to AGI because AGI is a system that is going to be able to generalize out of distribution tasks. Because I guess, of course you can always put more data in. But that kind of architecture, I mean, I don't want to say it doesn't scale to AGI, of course you could use unlimited synthetic data. But the point is, okay, is that when you're trying to replicate human intelligence, that comes from the human's ability to be able to do things they haven't seen before and not fail to understand the physical law. Think about it like this, okay? Remember how in the video, and this is how it kind of blew my mind, okay, this is when I really understood it, okay? So remember how in the video, okay, like we saw, okay? And this is why this research is so important, okay, Remember how in the research, this is what we saw, okay, Let me bring it back to a specific example. So in the research, we saw, okay, this, okay? And I'm going to just explain this one more time because it just recently hit me again, okay? We saw the fact that, like, all red items are circle and all blue items are squares. So all circles are red and all squares are blue, okay? But if a human, if you understood this, okay, and if you moved it from left to right, okay, when you were predicting what happens next, if it moves left to right, like if as a human you were given this, okay? Like if as a human you saw this and then you, you saw this, okay, your then prediction wouldn't be that this then turns into a circle, okay? Like that wouldn't be your prediction. Like you wouldn't think if something moves left or right, it then turns into another object. But because this system does think that, I mean, I guess that these systems aren't learning. And as this research says, it's simply via case based retrieval, which is rather fascinating. Now, of course, like, remember when I talked about, you know, like I said, you know, two, three weeks ago I did that video on the Apple paper, which is basically where they said we found no formal reasoning in language models. The behavior is better explained by sophisticated pattern matching so fragile that in fact changing names can alter results by 10%. I don't know about you guys, but if you're studying for a test and they change the names, I don't think your results would drop that significantly. Obviously, 01 does perform a lot better. But of course we need things that are really effective at performing on tasks that they haven't seen before. And changing names really shouldn't result in that much of a drop. 10% is really significant. If you look at how models change from iteration to iteration, a lot of these AI companies are really happy when they get a 10% improvement. So if you can change the name on a benchmark and it manages to drop 10%, that's pretty bad. Now one of the things that I wanted to talk about is of course, of course, Yannel Khan, because I'm guessing that right now he's feeling pretty vindicated because this kind of research is something that backs up the kind of thing that he's been talking about for quite some time now. May include a video clip. I'm not really sure. I don't know how many people watch these videos that I post on Yann Lecun. Because Yann Lecun is seen unfortunately as an AI skeptic, just skeptic. Just because he doesn't really focus on LLMs and generative AI, he focuses on an entirely different architecture called objective driven AI. Now basically what he says is that objective driven AI may actually reach human level intelligence one day. And that when you actually look at what objective driven AI is, he basically says that this kind of AI is where they use world models, joint embedding, predictive architectures which are not generative. And with this, then you can actually get to systems that understand the physical world, have persistent memory, can reason, and can perhaps plan hierarchically. So, okay, this is Yann Lecun's entire theory. This is what he's working on at Meta, and this is his big thing. I may include a clip, I'm not sure because there is an entire 30 minute talk. I did a long video on it.

I have Yann leCun's Objective Driven AI and this is essentially the main architecture that will essentially be artificial general intelligence. Now this is quite the different architecture compared to current standard LLMs and, and even quite different from the O reasoning, considering it's an entirely different new system. So I'm going to try to use a simplified breakdown because Yann Lecun does talk about this for 10 plus minutes and I gotta be honest, it is. So basically, instead of just reacting to data like how current AI systems, which are LLMs, respond based on patterns, objective driven AI works more like a thinking process. It would allow the AI to imagine different possible future scenarios and basically make plans based on that. Now the reason that this is truly important is because the goal is to move beyond AI that can only perform specific tasks like predicting the next word in a sentence, and move towards AI that can figure out how to achieve goals in new situations, even if it's never faced those exact scenarios before. And that's something that AI has a really big problem doing. So how this objective driven AI works is that the AI has a world model which is essentially a mental representation of how the world works. And, and then it combines this world model with goals, objectives, and then optimizes its actions to achieve those goals while considering any constraints like avoiding danger. And instead of just going through preset actions like following a script, it can adjust and adapt based on what it learns or what changes in the environment, which is quite more like how humans plan. So I've added this graph by Google Gemini that showcases exactly the key differences between LLMs and objective driven AI. And I think this is a useful graphic that you might want to screenshot because it just simplifies the understanding. Next we have the V JEPA architecture. Now this is something that was actually open source earlier this year and around February. This is something that meta are openly trying to build upon with the open source community and are still developing. But basically what they're trying to do is they're trying to get a system that can predict things as efficiently as humans. If you know humans, humans don't have to do things millions of times to them to get it right. They can do things a few times and implicitly understand exactly what's going on. And that's what V. Chepper is doing. So I'm going to play for you this first video by Meta, which is a really simple understanding that's going to show you exactly what's going on and then you're going to hear Yann Lecun actually talk about why generative architectures don't work for predicting certain things, which is really interesting because I think the space needs this kind of input because I think once we start to criticize ideas, I think that's how we can actually lead to some kind of improvement.

Today, machines require thousands of examples and hours of training to learn a single concept. The goal with JEPAs, which means joint embedding Predictive architectures, is to create highly intelligent machines that can learn as efficiently as humans. V JEPA is pre trained on video data, allowing it to efficiently learn concepts about the physical world. Similar to how a baby learns by observing its parents. It's able to learn new concepts and solve new tasks using only a few examples without full fine tuning. V JEPA is a non generative model that learns by predicting missing or masked parts of a video in an abstract representation space. Unlike generative approaches that try and fill in every missing pixel, V JEPA has the flexibility to discard irrelevant information, which leads to more efficient training. To allow our fellow researchers to build upon this work, we're publicly releasing V jepa. We believe this work is another important step in the journey towards AI that's able to understand the world, plan reason, predict and accomplish complex tasks.

You cannot predict which word is going to follow a sequence of words, but you can produce a probability distribution of all possible words in a dictionary. But when it's video frames, we do not have a good way to represent probability distributions over video frames. And in fact, the task is completely impossible. Like if I take a video of this room, right? I take a camera, I shoot that part and then I stop the video and I ask the system to predict what's next in the video. It might predict that there's going to be the rest of the room at some point. There's going to be a wall, there's going to be people sitting, the density is probably going to be similar to what's on the left. But it cannot possibly predict at the pixel level what all of you look like, what the texture of the world looks like, and the precise size of the room and all things like that. There's no way you can predict all those details accurately. So the solution to this is what I call joint embedding predictive architectures. And the idea is to just give up on predicting pixels. Instead of predicting pixels, let's learn a representation, an abstract representation of what goes on in the world and then predict in that representation space. Okay, so that's the architecture Joint embedding, predictive architecture. These two embeddings, take X, the corrupted version, run it through an encoder, take Y, run it to an encoder and then train the system to predict the representation of Y from the representation of S of X. Now the question is how you do this, because if you just train a system like this using gradient descent back propagation to minimize the prediction error is going to collapse. It's going to say it's going to learn a representation that is constant. And now it becomes super easy to predict, but it's not informative. Um, so, but that's the difference that I want you to um, remember the difference between generative architectures that try to reconstruct predictors, autoencoders, generative architectures, mass total encoders, whatever, and then the joint embedding architecture where you make predictions in representations space. The future I think is in those joint emitting architecture. We have tons of empirical evidence that to learn good representations of images, the best way to do it is to use those joint emitting architectures. All attempts at trying to learn representation of images using reconstruction are bad. They don't work very well. And there were huge projects on this and claims that they work, but they really don't. The best performance obtained with.

So for those of you who want a quick, you know, comparison on what the differences are between V jepa, which is meta's approach and of course the populate generative approaches, V JEPA is basically where you have, you know, efficient learning, uses 1.5 times to 6 times less training data through abstract understanding. It can learn tasks without retraining the entire model and it excels at detecting detailed objects, interactions. Now one of the limitations is that it only works well with short videos up to seven seconds. When you compare that to the physics based video generation, which is where it can complete, you know, full physics based video predictions, it does have some limitations. Limitations such as it tends to memorize rather than understand physics and distribution limitations, struggles with scenarios outside the training data and of course it requires some retraining. Yeah, I'm not gonna lie guys, this is definitely one of the craziest videos that I think I've made in quite some time. And this kind of research is truly, truly impactful because they're basically saying that look, if we actually want to get to AGI and we actually want to get to real world physical video simulators, we're going to need an entirely new different approach because this thing is just relying on training data to predict the future action. So clearly. They're basically saying that, look, OpenAI Sora is good, but it's just wrong in terms of its architecture approach if you're actually trying to get to a world generator. Which is a pretty crazy conclusion to come to. But I'm not going to lie, guys. After seeing the research, I can completely understand. I think these guys are onto something. So with that being said, let me know what you guys think. I do apologize for my okay.