Ilya Sustekever Finally Reveals Whats Next In AI... (Superintelligence)

If you think about the evolutionary history of humanity, so 4 billion years ago there was a single cell, some kind of a replicator. Then about a number of billions of years you had various different single cellular organisms. Then about a billion years ago you had multicellular life. Several hundred million years ago you had maybe reptiles. 60 million years ago you had mammals. 10 million years ago you had primates. 1 million years ago you had the homo sapiens. Then 10,000 years we had the writing. Then you had the farming revolution, then the industrial revolution, the technological revolution, and now finally the AGI, the super intelligence. It is the final, the ultimate challenge. It can create a life of unimaginable prosperity, which Sam alluded to. But it is also a great challenge.

So this is the article that actually has the information where Elia Sutskova finally speaks after a long hiatus. And this article is from Reuters, where it essentially talks about how OpenAI and others are seeking a new path to smarter AIs as current methods hit their limitations. Now, if you're wondering who Reuters are and if they are actually a reliable source of information, if you actually remember what happened quite some time ago, Reuters were actually the company that was able to give everyone this piece of information, which I don't want to say broke the Internet, but it was really important to understanding where the pace of AI development was at the time. And this was the article that basically warned of Q Star and it said that ahead of OpenAI's days of turbulence, basically there was a letter to the board which warned the directors of a powerful AI discovery that could threaten humanity. And I remember this is why Q Star started going completely viral. But in this article what they focus on is that they focus on the, the company, OpenAI and the entire AI industry. So they're basically saying that, look, these companies are facing unexpected delays and challenges in the pursuit of ever bigger LLMs by developing training techniques that use more human like ways for algorithms to think. And if you've been paying attention to the AI space, you know exactly the kind of methods that we're using right now, which are essentially where we have the models think at inference time in, in order to generate a more coherent and a more accurate response. But what the crazy thing is is that of course this new paradigm, although it does exist and although it's going to be, you know, scaled up, of course they're talking about the GPT series and of course you can see they're basically stating that. But now some of the most prominent AI scientists are speaking about the limitations of this. Bigger is Better is philosophy. So this is the part where they talk about how when ChatGPT was released, the main thing that people were focused on was essentially the fact that scaling up current models through adding more data and computing will consistently lead to improved AI models, which was basically the law that the more data you add to the model, the better these models get in terms of their coherence, in terms of response and in terms of how smart these models get. But of course we have to remember that it seems now that there are several limitations to this bigger is better philosophy, and which of course does make sense now. It isn't only Reuters that have actually commented on this phenomenon. If you watch my video from two to three days ago, you remember I spoke about this article from the Information and this basically spoke about how OpenAI's next series of models, which is of course Orion, how that model isn't reliably better than its predecessor at handling certain tasks are torn into the employees and apparently, you know, Orion is performing better at language tasks, but isn't performing better at coding tasks. And the fact that once again, this Orion situation could test a core assumption of the AI field known as the scaling laws, and that this is why companies are now moving towards this different area where they're going to be improving reasoning after their initial training. Now, of course, most of you guys might be wondering, what exactly did Ilya Satskova say? But of course this is what he actually said. He actually said, and I think this statement is really important because Ilya Sutskeva is one of the innovators in the field of AI and he was really integral to OpenAI's success. And he basically said that the results from scaling up pre training, which is where they have the phase of the AI model that uses a vast amount of unlabeled data to understand language patterns and structures, apparently this area of growth has actually plateaued. So apparently this area is something that is not doing too well at the moment and that the growth here is slowing down. And the crazy thing about this is that OpenAI have now said this. Elias Satskova is now saying that, look, pre training isn't as effective as it was when we were in the early days. And the crazy thing about this is that even if you've been paying attention to some of the things that are going on in the industry at other companies too. A couple of days ago there was this article from the Verge that basically spoke about how even Google are facing these kinds of issues. And the Verge basically commented on the next version of Gemini and apparently Gemini is going to be released widely soon. And it says here that I've heard that the model isn't showing the performance gains the Demis Hasabis led team had hoped for, though I would still expect some interesting new capabilities. And the chatter I'm hearing in AI circles is that this trend is happening across companies developing leading large models. So this is something that we can see right here that shows us that, look, this isn't just an open AI situation. This isn't just an Ilya Satskova company situation. This is a phenomenon now leading into AI. If you're wondering why some of these companies are facing many, many issues with scaling up these models. So some people do believe that scaling up these models won't lead to greater intelligence at all, but just better memorization machines.

If you scale up the size of your database and you cram into it more knowledge, more patterns and so on, you are going to be increasing its performance as measured by a memorization benchmark. That's kind of obvious, but as you're doing it, you are not increasing the intelligence of the system one bit. You are increasing the skill of the system, you are increasing its usefulness, its scope of applicability, but not its intelligence. Because skill is not intelligence. And that's the fundamental confusion that people run into, is that they're confusing skill and intelligence.

Now, I don't think this is as bad as it might seem to be, because there's also additional information that I need to share with you. So the article continues to talk about how Satskava was widely credited as an early advocate for achieving massive leaps in generative AI through the use of more data and compute in pre training and power and all that kind of stuff when they created ChatGPT. And of course now he left to start his own company, save Superintelligence. However, this is what Eliya Sutskeva is saying. He's saying that, look, the 2010s were the age of scaling. Now we're back in the age of wonder and discovery once again, and everyone is looking for the next thing. And Sutskova said scaling the right thing now matters more than ever. Which means that we've entered the next paradigm of AI, which means that it's quite likely we're on the bottom of this S curve growth once again. And I will explain that to you guys in a moment with a small diagram. But this is basically what that means. So when we do look at what the kind of growth that we do have, just bear with me for one second. So this is the kind of graph that I'm going to show you guys in a second. But basically there's S curve. So of course you have this first area where you start, you go up and then of course there's a decline plateau. And of course I want you to focus on this area right here where there's these bubbles of innovation as these S curves go up. And then basically this is what we're seeing with these LLMs right now. So if you guys do remember the areas where we had GPT2, of course we had GPT3. And then of course the reason we had that major, major jump is because with S curve growths you get that major jumps from GPT3 to that GPT4 area. And it seems that now leading up to the area where we're going to go to Orion, it seems that there is that slow peter off as we get to the top of the benchmarks and as things start to slow down. But remember, that doesn't mean that things are slowing down because of course now we are at that new paradigm where you can see that once again we have that bubble of innovation. So here you can see that there is a new paradigm which is the 01 series. We've of course got the O2 series. And then things are going to start to get crazy once we get to the O3 and potentially O4 series before once again potentially slowly petering out before another S curve growth again to potentially lead to asi. So this is essentially how a lot of graphs do depict the kind of growth that we do have. Of course, I just created this as a visual demonstration to show you guys exactly what's going on. But when we look at the Times 2025, 2026 and 2027, this could be the short paradigms for where we have these S curves growth up, S curves growth up and then these bubbles here and here where we switch innovation. So of course, right now, of course we're switching innovation to the test time compute paradigm. And I do wonder what comes after that, because it might lead to some asi. Now, Ellis Tziva also talks about what superintelligence is.

Why did we choose to use the term superintelligence? The reason is that super intelligence is meant to convey something that's not just like an AGI. With AGI we said, well, you have something kind of like a person, kind.

Of like a co worker.

Superintelligence is meant to convey something far more capable than that. When you have such a capability, it's like, can we even imagine how it will be. But without question, it's going to be unbelievably powerful. It could be used to solve incomprehensibly hard problems. If it is used well, if we navigate the challenges that superintelligence poses, we could radically improve the quality of life. But the power of superintelligence is so vast.