Metas New A.I Statement Just CONFUSED Everyone!

There is no such thing as AGI because we can talk about human level AI, but human, human intelligence is very specialized, so we shouldn't be talking about AGI at all.

So Meta's chief scientist has said something that has ruffled some feathers in the AI community. Yann L is a member of Meta's AI team and he leads a division that is working on something different. But his recent comment has, like I said before, made some people wonder what's really going on with this AGI thing. Just take a listen.

We should be talking about what kind of intelligence can we observe in humans and animals that currently systems don't have. And you know, there's a lot of things that currently systems don't have that your cat has or your dog, and they don't have anything close to general intelligence. So the problem we have to solve is how to get machines to learn as efficiently as humans and animals. That is useful for a lot of applications. This is the future because we're going to have AI assistance that, you know, we talk to help us in our daily lives. We need those systems to have human level intelligence. So, you know, that's, that's why we need it. We need to do it right.

So his statement there was actually rather fascinating. He said that we shouldn't even be talking about AGI at all and we should be talking about what kind of intelligence we observe in humans and animals that currently AI systems don't have. It's a very, very interesting perspective and it makes the AI industry one of the most interesting ones because, you know, people just don't agree on certain things. Now this AGI debate on what AGI is and what AGI isn't. There's a lot more that is explained in a recent article by the Financial Times. You can see here, he says Meta's AI chief says that large language models will not reach human intelligence. Yannin argues that current AI methods are flawed as he pushes for world modeling vision with super intelligence. And take a read at this because I think it's important to have different and other AI perspectives that show you that the path that everyone is currently on might not be the path that we use to get to super intelligence. And remember, this is one of the most respected AI researchers that, that have been in the field for a very, very long time and has made numerous contributions to the field. So take a look at this. Meta's AI chief said that the large language models that power generative AI products such as ChatGPT would never achieve the ability to reason and plan like humans, as he focused instead on a radical alternative to create superintelligence in. In machines. Okay. And I think it's pretty crazy that he's focusing on something that will create super intelligence and is basically stating that LLMs have, I wouldn't say plateaued, but they're not the kind of intelligence that you can effectively scale. Take a look at this. Okay. He said that LLMs have very limited understanding of logic, do not understand the physical world, do not have persistent memory, cannot reason in any reasonable definition of the term, and cannot plan hierarchically. And I honestly say you do have to agree with some of the things that he says. I mean, very limited understanding of logic. This is something that, if you've actually tested GPT4's version, not the version of logic, but if you've tested, like, how the logic actually, I guess you could say, is understood sometimes it doesn't truly understand certain things. Like, for example, if you ask it to count the number of Rs in Strawberry, it gets it wrong. But if you ask it to count it out letter by letter, it then gets it right. If you present certain logic problems that are very simple to a human, sometimes GPT4 will get it wrong, and then sometimes it will get it right. Okay? And that is apparently a limitation of these large language models. Now, in addition, of course, they do not understand the physical world. And I think understanding the physical world, how things interact and how they are, is of course, an important component of how we learn and how we digest information. I mean, it's a lot more that than these LLM systems have because we have the senses of touch, we have smell, we have sight. I mean, all of these, you know, senses and LLMs, what do they really have? They just have text, which is, I guess you could say, very limited in terms of, you know, how much intelligence you're allowed to grab. And, and of course, we have, you know, persistent memory. This is something that is really, really important if we want a system to be able to effectively become smarter and effectively learn. So persistent memory is something that, you know, I guess you could say they're making some strides in it. But I guess the only way we're going to know is if there are any research papers that are published surrounding this area. And of course, if any of the big labs do state something like that now. So, I mean, there are a lot of limitations on LLMs that most people don't think about. If you're actually trying to get to AGI, like all of these things, understanding of logic, understanding of the Physical world, a persistent memory, you know, being able to reason in a decent definition and being able to plan hierarchically. Those are things that, you know, when you think about it really like you really break it down. The average human can do these things, you know, with, I guess you could say, not ease. But it's not impossible for the average human to do this. I mean, humans just have this innate understanding of basic logic of the physical world and of course a persistent memory. Now there are also some other things that were said. We do have the fact that he is developing an entirely new generation of AI systems that he will hope will power machines with human level intelligence, although he said that this vision could take 10 years to achieve. So he's basically, I wouldn't say given up on generative AI, but he's working on something completely different. And one of the key indicators of this is that every time Alarm 3 paper is released, sometimes people say congratulations Jan Lerner. But he always states that, hey, it wasn't actually me that worked on this. We're working on something completely different. And he describes it here and I'm going to show you guys a video where basically he talks about this, but he describes it here. He says, this is the article in the Financial Times where I explained that autoregressive LLMs are insufficient to reach human level intelligence or even CAT level intelligence, but alternative architectures. And this is where the meat of the video comes into play. Objective driven may reach human level intelligence one day. They use world models based on jepa, joint embedding predictive architectures which are not generative. With this we may have systems that understand the physical world, have persistent memory, can reason, can plan, and perhaps hierarchically. So this is where we take a look at, you know, his new architecture that they're working on, which is, I guess you could say, the path to superintelligence.

AGI Today machines require thousands of examples and hours of training to learn a single concept. The goal with JEPAs, which means joint embedding predictive architectures, is to create highly intelligent machines that can learn as efficiently as humans. V JEP is pre trained on video data, allowing it to efficiently learn concepts about the physical world. Similar to how a baby learns by observing its parents. It's able to learn new concepts and solve new tasks using only a few examples without full fine tuning. V JEPA is a non generative model that learns by predicting missing or masked parts of a video in an abstract representation space. Unlike generative approaches that try and fill in every missing pixel. V JEPA has the flexibility to discard irrelevant information, which leads to more efficient training. To allow our fellow researchers to build upon this work, we're publicly releasing V jepa. We believe this work is another important step in the journey towards AI that's able to understand the world, plan, reason, predict and accomplish complex tasks.

Very fascinating. Okay, something, something very, very interesting that you need to know. So of course we know that V Jepa right here is, you know, self supervised learning. You know, it's trained using unlabeled data, meaning that it doesn't need a predefined labels to learn. This is bas this the process of self supervised learning, which essentially is just more efficient. And it kind of mirrors how humans naturally learn from their environment. And with V. JEPA you have this abstract representation. So this is higher level understanding. So instead of focusing on every tiny detail, V. Jepper learns to understand videos in an abstract way. For example, it can recognize actions like picking up or putting down a pen without needing to analyze every single pixel in the video. And the model essentially learns by masking parts of video. So the model learns by predicting the missing parts. And it helps and this helps basically develop a deeper understanding of sequences and interactions with the video. So basically he kind of describes this right here with us to why this is a lot more effective than what we are currently doing.

And so we started experimenting with systems that could do video prediction. We got nowhere. I mean those system can sort of predict a little bit in the video, but the kind of representation of the world they learned from this is useless. And what I kind of completely changed my mind about three, four years ago by realizing that the systems that work best to learn image representations are not generative. They're not systems where you take an image and you corrupt it and you restore it or video. There are systems that are called joint embedding. So you take an image and a corrupted version of it, and then you train encoders, neural nets to encode the image so that you can recover the representation of the fool from the representation of the corrupted. And that's not generative. You're not reconstructing pixels. It turns out you can't reconstruct pixels in images, just too complicated. So you have to use those non generative architectures. So the future of AI is non generative.

So you can see here that he's clearly stating that the future is of course non generative. Now it's pretty interesting with us to, you know, his further statement on LLMs because in a recent interview called the scaling theory on the Podcast, he basically said that LLMs are useless. So take a listen to this. I mean, I don't want to discredit him by stating that he said LLMs are completely useless. They're useful for of course, text generation. But he talks about in terms, of course reaching AGI, they're not as useful as people might think.

A good guess is that the ones that will be players there are the ones that can invest in research, long term research, not just fine tuning current systems, but like, you know, how do we change the paradigm? Which is basically what I've, I personally work on the next generation AI systems. To people like me, LLMs are the past, essentially, they're kind of boring now. Of course they're very useful and there is a whole industry that should be built around them. I'm not saying the opposite, but if you're interested in the future, there's kind of a lot more that will happen in the future and it may completely change what we perceive. I mean, it's entirely possible, for example, that system will become a lot smarter with a lot less data. Humans are not exposed to nearly as much text data as current AI systems yet we're still a lot smarter than those systems. Right?

And here is another important clip, and this is basically why he says LLMs suck and hear what they're good for. Because I think this clip where he actually explains it and breaks it down with us to why we're fooled by their fluency and they don't truly understand how the world works, I think he's stating that, you know, basically the current consensus, the current, I guess you could say collective consciousness on how great LLMs are, is, I guess you could say mistaken. You know, people are fooled by the current state of things.

They really are not good for anything. And so the idea that somehow we're going to scale those up, I mean, they're very useful, but they're not good for reaching human level intelligence. So the idea we're going to scale those up and train them on even more text, which we don't have because basically they're already trained on the entirety of the public text on the Internet. The idea somehow we're going to scale them up and reach human intelligence, it can't possibly work. So AGI is not around the corner, if you believe in AGI. And so we are easily fooled by their fluency into thinking that they are smart, but they really aren't. They're useful. There's no question we can.