Experts are STUNNED! Meta's NEW LLM Architecture is a GAME-CHANGER!

So Meta might have just introduced the future of large language models. Well, I guess after this video you won't be calling them large language models anymore. This is what we call large concept models, language modeling in sentence representation space, basically stating that, look, LLMs aren't exactly the best thing to use. And in this new paper by Meta, they detail large concept models. This is something that is, I guess you could say, a little bit different to LLMs. And it's really cool when you realize the thing that it is tackling. So we all know what LLMs are, but if you know how LLMs work, you'll know that they work through the process of tokenization. So these systems actually work by predicting the next word and they do that via the token. So you can basically think of them as advanced autocomplete, although I do know some researchers are going to hate me saying that. But this is essentially how it works. And I've used the GPT4O tokenizer visualizer to show you what the characters actually look like to the LLM system. And this is genuinely why there was that big debate as to why these LLM models weren't able to get the famous question, how many Rs are in strawberry? There are two Rs in the word strawberry. Of course there are three. But this is essentially why these LLMs weren't able to get the answer right, because they of course just saw the word Strawberry as one entire token, leading to this big confusion. Now this is essentially where another AI researcher, he actually talked about this entire thing. He says, I'm more and more confident that tokenization will be gone, stating that the era of these tokenization models might actually be going away sometime soon. And he says here that humans don't actually think in tokens. Tokens are hard coded abstractions in LLMs that lead to weird behavior. LLMs can solve PhD level math questions but cannot answer. Is 9.9 bigger than 9.11? And he says now Meta is shifting LLMs to large context models or concept models, changing next token prediction to next concept prediction. And the concept is treated as a sentence representing an abstract idea or action. So they talk about how, despite the undeniable success of LLMs and continued progress, all current LLMs miss a crucial characteristic of human intelligence, explicit reasoning and planning at multiple levels of abstraction. The human brain does not operate at the word level only, which is very true. We usually have a top down process to solve a complex task or compose a long document. Of course, we first plan the higher level like the outline, and then Step by step, we add these details at lower levels of abstraction. And one may argue that LLMs are implicitly learning hierarchical representation, but they're stipulating that models with an explicit hierarchical architecture are better suited to create coherent long form output. And I think this is one of the key things here that, you know, LLMs do miss some reasoning steps. And I think maybe this is why certain models reason better than others, because maybe they're already using this architecture. Of course, that is some huge, huge speculation. But we do know that certain companies like Claude have a ridiculous level of response quality and many are stumped as to how they're managing to do it. Now, basically they actually talk about this example here and don't worry about this huge block of text. I'm going to explain this to you in a simple, simple way. It basically says that imagine a researcher giving a 15 minute talk or presentation, which is kind of like what I'm doing now. And in such a situation you don't usually prepare detailed speeches by writing out every single word they're going to pronounce. And of course this is true. It wouldn't really flow well and it wouldn't sound that good. It would really sound monotonous. Instead, they actually outline a flow of high level ideas that they want to communicate. Usually on a PowerPoint presentation they'll just have like three or four words, maybe with some images, and then they'll talk about those. And basically they also state here that if they give the same talk multiple times, the actual words being spoken are actually going to be different and they could even be in different languages. But the higher level abstract ideas are going to remain exactly the same. And this is the same when writing a research paper or essay on a specific topic. And this is exactly what humans do by preparing an outline that you know, is of course a structure, and then they refine it iteratively. And of course, this is something that humans do when processing a large document. Instead, they use a hierarchical approach and they remember which part of a long document they should use. The, the search for a specific. And of course, this is something that humans do when looking through words in a large document. Now this is the example that they actually use here. So this left side diagram that you actually see shows how the model transforms a story about Tim into a very simple form. You can see pink dots actually show us the detailed story. So it shows us how Tim wasn't athletic. He thought that that would change if he joined a sport. He tried out for several teams. He didn't make the Cut for any of them. And then he decided to train on his own instead. You can see that the higher level abstraction here you can see the higher level idea and the blue dots show how it's summarized into a key concept. And it says that because he was not very athletic, Tim could not join teams to improve. So he decided to train on his own. And then this is how on the right hand side we can see how the LCM processes this language like a sandwich. So we can see this bottom layer which is the concept encoder, which is of course fixed. The this is where you take the regular words and you convert them into these concepts so you can think of these as complete ideas. Then this is where we got the middle layer, the large concept model. This works with these concepts and processes and understands them like understanding the main ideas without worrying about specific words. And then of course we get to the top layer, which is the concept decoder, which is actually it takes the processed concepts and turns them back into regular words that we can actually read. This is quite like similar to taking a story written in English and converting it to universal idea symbols that any language could understand and then processing those ideas and converting those ideas back into words, which could be, you know, in any language. Think of this bottom layer right here as, you know, an idea language. And then this is where you think about those ideas and then this is where you translate it back into human language. And this is actually really different to the current AI models that work directly with words. This system works with complete ideas or concepts instead. Now what's actually fascinating about this is that they state that to some extent the large concept model architecture actually resembles the JEPA approach, which is from Yann Lecun, as you know, one of the leads at Meta AI. And this V. JEPA approach is something that also aims to predict the representation of the next observation in an embedding space. And this is actually really interesting because V. JEPA is a fascinating architecture. This is the architecture where they talk about a new architecture for self supervised learning and how they actually try to learn the same way that humans do.

The goal with Jeppas, which means joint embedding predictive architectures, is to create highly intelligent machines that can learn as efficiently as humans. V JEPA is pre trained on video data, allowing it to efficiently learn concepts about the physical world. Similar to how a baby learns by observing its parents. It's able to learn new concepts and solve new tasks using only a few examples without full fine tuning. V JEPA is a non generative model that learns by predicting missing or masked parts of a video in an abstract representation space. Unlike generative approaches that try and fill in every missing pixel, V JEPA has the flexibility to discard irrelevant information, which leads to more efficient training. To allow our fellow researchers to build upon this work, we're publicly releasing V jepa. We believe this work is another important step in the journey towards AI that's able to understand the world, plan reason, predict and accomplish complex tasks.

Now, if we actually do take a look at the results, we can see that LCMS actually generated coherent and meaningful expansions. LCMS also tended to avoid excessive repetition compared to LLMs like LLAMA 3, 18B and LCMS followed instructions better, generating controlled length expansions. Overall, I do think this is one of the most fascinating things because tokenization has really been a big problem with LLMs, and we know that these very small problems in AI, like how many Rs are in Strawberry are quite frustrating when we're trying to look at what these LLMs can do. Considering that we know these models are pretty smart. Let me know what you guys think about this research. If you guys think this is the future of LLMs or what's going to be next, or maybe this is going to be some kind of hybrid architecture, I do think META is always pioneering innovative research, which is why I chose to include it into today's video. So if you enjoyed the video, I will see you in the next one.