Metas AI Team Just Revealed Their Secret To AGI

Yann Lecun has given a recent talk that is probably one of the most insightful talks recently given in AI, because it talks about the future of artificial intelligence and how we're actually going to get to AGI. He talks about the timelines to AGI, the architectures that we need, and of course, he's going to start this conversation with what the future of AI assistants are going to look like.

Okay, so I'm going to talk about human level AI or how do we get there and how we are not going to get there as well. So, first of all, we do need human level AI because there is a future in which most of us will be wearing smart glasses or wearing other types of devices, and we'll be talking to them. And those systems will host assistants. Maybe not just one, maybe a whole collection of them. And what that will cause is that all of us will have basically a staff of kind of smart virtual people working for us. So it's like everybody would be a boss.

This is where Yann Lecun actually talks about what we need in order to get to advanced machine intelligence. Some could call that AGI, some could call that asi. But I think it's really important that we look at the things that we currently don't have. Things like persistent memory, a bunch of other things, and these are the kinds of things that future systems that are really, really smart and are on the level of artificial superintelligence. This is going to be the base level of things that we're really going.

To need, just not of real humans. And we need to build this for basically amplifying human intelligence and making people more creative, more productive and everything. But for this, we need machines that understand the world. They can remember things that have intuition, have common sense, things that can reason and plan to the same level as humans. And despite what you might have heard from some of the most enthusiastic people, current AI systems are not capable of any of this. So that's what we need. Systems that learn to basically model the world, have mental models of how the world works, and every animal has one search model. Your cat certainly has one that's more sophisticated than any AI system ever built or ever devised. Systems that have persistent memory, which current LLMs don't have. Systems that can plan complex action sequences, which is not possible with LLMs today, and systems that are controllable and safe. So I'm going to be proposing an architecture for this that I call objective Driven AI. I wrote kind of a vision paper about this that I posted about two Years ago and a lot of people at FAIR are basically working towards implementing that plan. FAIR used to have a combination of long term blue sky research and more applied projects. But Meta a year and a half ago created a product division called Genai focused on AI products and they do applied R and D. So now FAIR has been sort of redirected towards the longer term next generation AI system. We don't do LLMs basically. So the success next.

This is where Jan Lecken actually speaks about how we need something. We're basically missing something because we keep running into Moravec's paradox where things that are easier for humans are extraordinarily difficult for computers. And things that are, you know, really, really good for computers, things that computers really excel at, like mathematics and advanced calculations, these are things that humans really do struggle with. So we need a way to solve that by tackling this in a different method.

And we're still missing something big to reach human level intelligence. And I'm not necessarily talking about human level intelligence here, but even your cat or your dog can do amazing feats that are still completely out of reach of current AI systems. How is it that any 10 year old can learn to clear up the dinner table and fill up the dishwasher and the 10 year old can learn this in one shot. There's no need to practice or anything. 17 year old can learn to drive a car in about 20 hours of practice. We still don't have level 5 self driving cars and we certainly don't have household robots that can curb the dinner table and fill up the dishwasher. So we're really missing something big. Right? Otherwise we would be able to do those things with AI systems. So we keep bumping into this thing called the Moravec paradox, which is that things that appear trivial to us, that we don't even consider intelligent seem to be really, really difficult to do with machines. But like high level complex abstract thinking like manipulating language seems to be easy for, for machines or things like playing chess and go and stuff like that.

So next we have one of the most fascinating pieces of data, no pun intended, but this is where Yann Lecun actually talks about how our world model needs to be trained on a lot more data than we think. And he basically says that it would take an LLM like it would take 350,000 years for human to read the amount of data that we need, and especially at like 250 words a minute. And of course you can see that as a human child is awake for 16,000 hours that's just more data than any LLM has seen, despite these bigger training runs. So he's basically saying that, look, we think we've got a lot of data, but when we compare it to systems that are actually doing really well, like humans and animals and these things that are actually really good in the world, when we actually look at every single image as a frame, as a piece of data, that's actually so much data that means that we just are going to need a lot more of it. Okay, so it's like to think about how we're even going to do that.

Okay, so maybe one reason for this is the following. An LLM is typically trained on 20 trillion tokens. A token is basically zero is like three quarter of a word on average for a typical language. So that's 1.5 10 to the 13 words. Each token is about 3 bytes typically. So that 610 to the 13 bytes, it would take on the order of a few hundred thousand years for any of us to read through this. Right? That's the totality of all the text available publicly on the Internet essentially. But then consider a human child. A human child, a 4 year old, has been awake a total of 16,000 hours, which by the way is 30 minutes of YouTube upload. We have 2 million optical nerve fibers, optic nerve fibers coming to our brain. Each fiber roughly carries about 1 byte per second. Maybe it's 1/2 byte per second. Some estimate it's like 3 bits per second. Doesn't matter, it's an order of magnitude. So that data volume is about 10 to the 14 bytes, roughly the same order of magnitude as the LLM. So in four years a child has seen as much visual data or data as the biggest LLM trained on the entire publicly available text on the Internet. So that tells you a number of things. That tells you first that we're never going to reach anything close to human level intelligence by just training on text. That's just not going to happen. Then a counter argument is okay, but visual information is very redundant. So first of all, this 1 byte per second per optic nerve fiber, that's already a 100 to 1 compression ratio compared to the photosensors you have in your retina. We have on the order of 60 to 100 million photosensors in our retina. And that gets compressed using neurons in front of your retina to 1 million nerve fibers. So there is already 121 compression. Then it gets to the brain and then it's expanded by a factor of 50 or something like that. So I'm measuring compressed information, but it's still very redundant and redundancy actually is what self supervised learning requires. Self supervised learning will only learn something useful from redundant data if the data is highly compressed, which means it's completely random. You can't learn anything, you need redundancy to be able to learn anything. You need to learn the underlying structure of the data. So we're going to have to train systems to learn common sense and physical intuition by basically watching videos or by living in the real world.

Next is where we have Yann leCun's Objective Driven AI. And this is essentially the main architecture that will essentially be artificial general intelligence. Now this is quite the different architecture compared to current standard LLMs and even quite different from the O reasoning, considering it's an entirely different new system. So I'm going to try to use a simplified breakdown because Yann Lecun does talk about this for 10 plus minutes. And I gotta be honest, it is. So basically, instead of just reacting to data like how current AI systems, which are LLMs, respond based on patterns, objective driven AI works more like a thinking process. It would allow the AI to imagine different possible future scenarios and basically make plans based on that. Now the reason that this is truly important is because the goal is to move beyond AI that can only perform specific tasks, like predicting the next word in a sentence, and move towards AI that can figure out how to achieve goals in new situations, even if it's never faced those exact scenarios before. And that's something that AI has a really big problem doing. So how this objective driven AI works is that the AI has a world model which is essentially a mental representation of how the world works. And then it combines this world model with goals, objectives, and then optimizes its actions to achieve those goals while considering any constraints like avoiding danger. And instead of just going through preset actions like following a script, it can adjust and adapt based on what it learns or what changes in the environment, which is quite more like how humans plan. So I've added this graph by Google Gemini that showcases exactly the key differences between LLMs and objective driven AI. And I think this is a useful graphic that you might want to screenshot because it just simplifies the understanding. Next we have the V Jepper architecture. Now this is something that was actually open sourced earlier this year and around February. This is something that meta are openly trying to build upon with the open source community and are still developing. But basically what they're trying to do is they're trying to get A system that can predict things as efficiently as humans. If you know humans, humans don't have to do things millions of times to them to get it right. They can do things a few times and implicitly understand exactly what's going on. And that's what V. Chipper is doing. So I'm going to play for you this first video by Meta, which is a really simple understanding that's going to show you exactly what's going on. And then you're going to hear Yann Lecun actually talk about why generative architectures don't work for predicting certain things. Which is really interesting because I think the space needs this kind of input because I think once we start to criticize ideas, I think that's how we can actually lead to some kind of improvement.

Today, machines require thousands of examples and hours of training to learn a single concept. The goal with JEPAs, which means joint Embedding Predictive Architectures, is to create highly intelligent machines that can learn as efficiently as humans. V JEP is pre trained on video data, allowing it to efficiently learn concepts about the physical world. Similar to how a baby learns by observing its parents, it's able to learn new concepts and solve new tasks using only a few examples without full fine tuning. V JEPA is a non generative model that learns by predicting missing or masked parts of a video in an abstract representation space. Unlike generative approaches that try and fill in every missing pixel, V JEPA has the flexibility to discard irrelevant information, which leads to more efficient training. To allow our fellow researchers to build upon this work, we're publicly releasing V jepa. We believe this work is another important step in the journey towards AI that's able to understand the world, plan reason, predict and accomplish complex tasks.

You cannot predict which word is going to follow a sequence of words, but you can produce a probability distribution of all possible words in a dictionary. But when it's video frames, we do not have a good way to represent probability distributions over video frames. And in fact, the task is completely impossible. If I take a video of this room, I take a camera, I shoot that part, and then I stop the video and I ask the system to predict what's next in the video. It might predict that there's going to be the rest of the room at some point there's going to be a wall, there's going to be people sitting, the density is probably going to be similar to what's on the left, but it cannot possibly predict at the pixel level what all of you look like, what the texture of the world looks like and the precise size of the room and all things like that, there's no way you can predict all those details accurately. So the solution to this is what I call joint embedding predictive architectures. And the idea is to just give up on predicting pixels. Instead of predicting pixels, let's learn a representation, an abstract representation of what goes on in the world and then predict in that representation space. Okay, so that's the architecture, joint embedding predictive architecture. These two embeddings, take X, the corrupted version, run it through an encoder, take Y, run it to an encoder, and then train the system to predict the representation away from the representation of S of X. Now the question is how you do this, because if you just train a system like this using gradient descent back propagation to minimize the prediction error, it's going to collapse, it's going to say, it's going to learn a representation that is constant. And now it becomes super easy to predict, but it's not informative. Um, so, but that's the difference that I want you to um, remember the difference between generative architectures that try to reconstruct predictors, autoencoders, generative architectures, masked autoencoders, whatever, and then the joint embedding architecture where you make predictions in representation space. The future I think is in those joint embedding architecture. We have tons of empirical evidence that to learn good representations of images, the best way to do it is to use those joint emitting architectures. All attempts at trying to learn representation of images using reconstruction are bad. They don't work very well. And there were huge projects on this and claims that they work, but they really don't. The best performance are obtained with architecture on the right.

Next. What is fascinating is that we get the first instance of what will happen once these systems are truly here. So this is where Yann Lecun gives his ideas and opinions on what the future is going to look like. And I think it's always important to look at what those who are considered some of the most skeptical of current AI, what they view the future to be like. Because their opinions are the least hypey, meaning that this is potentially the most realistic look at the future we're going to get.

So if we succeed in doing this, okay, we're going to have systems that really will mediate all of our interaction with the digital world. They will answer all of our questions, they will be with us at all times. They will basically constitute a repository of all human Knowledge. And this feels like an infrastructure kind of thing, like the Internet. Right. It's not like a product, it's more like an infrastructure. These AI platforms must be open source. I don't need to convince anybody from IBM here because IBM and Meta are part of something called the AI alliance, which promotes open source AI platforms. But I really thank Daayo for spearheading this and everybody at IBM. So we need those platforms to be open source because we need those AI assistants to be diverse. We need them to understand all languages in the world, all cultures, all value systems. And you're not going to get that out of a single assistant produced by a company on the west coast or the east coast of the us. This will have to be contributions from the entire world. And of course it's very expensive to train foundation models, so only a few companies can do this. So if those companies like Meta can provide those base models in open source, then the entire world can fine tune them for their own purpose. That's kind of the philosophy that Meta has adopted in IBM as well. So open source AI is not just a good idea, it's necessary for cultural diversity, perhaps even the preservation of democracy. So training and fine tuning will be crowdsourced or will be done by the ecosystem of startups and other companies. And this is really what has jump started the ecosystem of AI startups is the availability of those open source AI models. How long is it going to take to reach human level AI? I don't know, could be years to decades. There's a huge variance and there's many problems to solve on the way and it's probably almost certainly harder than we think. It's not going to happen in one day. It's going to be like progress, progressive evolution. So it's not like one day we're going to discover the secret to AGI and we're going to turn on a machine and immediately we'll have superintelligence and all of us will be killed by a superintelligent system. No, it's not happening this way. Machines will surpass human intelligence, but they will be under our control because they will be objective driven. We give them goals and they fulfill those goals. It's just like many of us here are leaders in industry or academia or whatever. We work with people who are smarter than us. I certainly do. There's a lot of people working with me who are smarter than me. Doesn't mean they want to dominate or take over. Right.

So of course, I think this talk was rather fascinating because it talks about AGI and Future Intelligences and says that they are not right around the corner, but they are going to be many years away and they are much harder than we do think. And I think that's quite fascinating, considering earlier this week we did get an interview from Demis Asabis saying that AGI is at least 10 years away, whilst other individuals of leading companies are saying that Superintelligence or AGI is around two to three years away. So, I mean, we're living in probably one of the most uncertain times considering the fact that the industry's experts are completely divided on these timelines. So, with that being said, let me know what you guys think about the future of aggressive.