The Future of AI Might Be…
A spoken YouTube commentary by Dr. Zsolnai (Two Minute Papers) about recent AI advances and whether they could lead to superintelligence. He contrasts early models (GPT-2) with modern systems (GPT-4), highlights a perceived core gap—reasoning and data efficiency—and explains how video-game-like simulations and reinforcement learning can provide abundant feedback and training signals. The video focuses on OpenAI's O1 system (not framed as GPT-5), which performs iterative "thinking" and uses reinforcement-style feedback; this could enable a self-improving loop that accelerates reasoning abilities, though robustness and failure modes remain concerns. The video ends with a sponsorship plug for Lambda Labs.
Chapters
We are living in the age of AI and it is crazy how far we have come. But what is coming is perhaps even more incredible. You see, back in the day we talked about OpenAI's GPT2, which could complete your sentences. Amazing feat at the time, but today not so impressive. But over time we got to GPT version 4, which is so much smarter and is a proper AI assistant with hundreds of millions of users all around the world. And, and our question is, where are we going? Is it just going to keep improving? Is creating a super intelligent AI possible? And what do video games of all things have to do with that? Well, most experts I hear about are saying that we have a key puzzle piece missing for achieving superintelligence and that is reasoning. You see, these are neural network based techniques that can learn to recognize a mug, but not as a human would recogn a mug. Many modern techniques require thousands and thousands, if not hundreds of thousands of images of a mug to understand what it is to a human. We just show them one and they immediately understand the concept. And that is an example of reasoning learning from very little data. So how do video games come into the picture? Well, one way it can help is if a computer game is nearly as detailed as reality, then we can just drop an AI in there and let it learn for as long as it wishes. Infinite self driving car scenarios, infinite worlds for robots to train in, you name it. That is kind of starting to work today. So that is fantastic. Two, they can play computer games themselves. Like DeepMind's early AI played and mastered Atari Breakout and then many other Atari games. And it gets better. Today they can even start out in a Minecraft world and get to a respectable amount of achievements in these games. It makes a series of actions with the controller and at the end if it got a good score, it can assign that good score to the whole chain of actions it just did. If the score is not great, it can learn that this chain of actions was not very successful. This is a technique often referred to as reinforcement learning that can be combined with a neural network for looking and understanding the video stream of the game itself. Please remember this because this might be the key to creating an incredibly capable AI. So how do we do that? I would like to ask for a little more of your patience, but I promise we'll get there soon. So this video game thing does not necessarily help AIs to reason. Computer simulations create tons of data for them to learn on. But remember, reasoning is learning from very little data current AIs are not too good at that. Until now OpenAI released their O1 system which is not GPT5. It is not just the next ChatGPT in line, it is a completely different kind of system. So what is different and where are we going with this? Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Well, when you ask a question, previous methods give you an answer almost immediately. However, the new O1 takes a little time to think. Here is my friend Dr. Jim Fan's fantastic visualization of it. Okay, but does that help? Is that a good thing? It surely does help us humans. A tough exam takes not several seconds, but often several hours. If we take longer, we do better. But machine intelligence is not human intelligence. So is thinking longer good for them too? Look, it is not good. It is fantastic. If we let it think for longer, the results can go from 20% accuracy to even 80% accuracy at a great cost. By the way, as this is a logarithmic scale. But that is a groundbreaking observation. However, it doesn't help with everything. If you ask a simple question, it might not make a difference. However, if you want to break a cryptographic cipher, solve tough brain teasers, do market research. It is fantastic. They just showcased three excellent and one not so excellent example of that. These are new real use cases that can help people in the real world and they are almost irrelevant. I will tell you why. 1. Here it helps with expanding your company within Europe and you see that it is thinking. Now of course this is not the same kind of thinking as we humans do. It is a slightly anthropomorphic, a humanified term if you will, and it talks about market entry strategy, risk analysis and financial planning. What is incredible is that if there is some part that you like, like for instance Berlin sounds good, just ask a follow up question and go deeper. 2. It can also help you code up a web app for your users to create their profiles. It thinks for only a couple of seconds and you get a full table of contents and a piece of code. Then you can even specialize it to your infrastructure. For instance, if you use Microsoft's Azure cloud, it can help with that too. 3. It can also help you make calculations about a financial instrument called Covered Cause. I highly recommend it in case you are looking to lose a ton of money really quick. And 4 if you are a fellow scholar, just ask how to make dogs a little healthier, what nutrients are important and what it recommends we do in terms of research when creating new kinds of dog food, I got to say I am a little less Impressed by this, because the task is not that specific. So general information can do a lot of heavy lifting here. So O is absolutely incredible. It can help quantum physicists with their research and do many other things the regular ChatGPT AI can only dream of. So finally, let's pop the question. Can this O one thing reason, and can it lead us to some kind of super intelligence? No. At least that is what scientists at Apple, yes, Apple say. They say, not so fast. These systems can't reason why. Well, if you take some relatively simple tasks and add more details to a question that seem relevant but are in fact completely irrelevant, basically junk. I wonder what does it do with current AIs? It shouldn't change anything. Of course, it is irrelevant information. But that's not what happens. Look, it throws them off pretty reliably. This applies to previous AIs, but then interestingly, it also applies to the new O1 system too. Or you can change a few numbers or names in the same tasks and they suddenly perform a bit worse. Not what you would expect from an intelligent being. So they say these AIs can't really reason. They are just complex pattern matching machines. They do that really well. But ultimate ultimately they are just matching patterns. Now, there is a philosophical question whether what humans are doing is anything more than complex pattern matching too. Sometimes I am thinking the question shouldn't be whether machines can think, but whether humans can think. So is this where things fizzle out? We won't get to more intelligent systems ever? Is it impossible? Well, not so fast. Don't despair, fellow scholars. Here is a big reason why it might still happen. And finally, here is the video game thing I promised you see, OpenAI's 01 is not just a neural network, but it is also a reinforcement learning system as well. We saw earlier that it builds up a chain of thoughts and then at the end we can tell the AI whether it did well or it did not. But wait a minute, do you see where this is going? That is exactly like a video game. Remember, in a video game you make a series of actions and you get a score. And if you got a high score, then you can look at the whole series of actions and it essentially becomes new training data for the AI that it can learn from over time. We say we like this answer and we didn't like that. And it can assign scores to its chain of thoughts and get tons and tons of data on how to improve data to improve reasoning. And it sees the whole thing as trying to solve a little video game. It's like the earlier AlphaGo AI which simulated many actions into the future during a game and if it led to a winning position, it learned from the whole chain. So it gets smarter. And if it gets smarter it will create better actions. And then it learns from those better actions, it gets even smarter, creating a potential flywheel effect. We don't know if that is exactly what is going to happen. This is how research works. But that seems to be OpenAI's grand plan. And here ultimately OpenAI's 01 doesn't learn how to play a game, it learns to be intelligent and helpful instead. And this also means something really surprising. We keep looking at these results, but they are completely irrelevant. Why? Because this is the foundation of a new kind of system that will be able to learn faster from its mistakes over time. Yes, this is the worst version of O you'll ever see. And that is what might lead to an incredible intelligent AI in the future. And we probably don't even have to wait for long. The pace of progress in AI research is staggering. It is probably learning and getting better right as you are watching this video and it cannot wait to help us with our scholarly problems. What a time to be alive. And if you feel inspired and you wish to run your own experiments on Nvidia GPUs, check out Lambda Lambda is a cloud service for large scale AI projects and it is my absolute favorite. Seriously. Try it out now@lambdalabs.com papers or click the link in the description. If you use Lambda, you can run your own experiments on super powerful hardware and you also support two minute papers.
