NVIDIA’s New AI: Insanely Good!

This is a historic paper, Groot N1, which is going to set off a robotics revolution by giving us. Yup, an open foundation model for humanoid robotics. All open, all free for all of us. Wow. And everyone's favorite, the super cute robots are coming, hopefully soon. Okay, so what is going on here? Is this important? Well, when it comes to robotics, not so long ago even a company like OpenAI said, Nope, I am outta here. Why did they do that? Note that you may see some eye candy here, but we are fellow scholars here, so rest assured that what you hear in this video is grounded in a proper research paper multiple, as you will see in a moment. And we are not shy to talk about limitations either. So when it comes to robotics, even OpenAI said, I am outta here. And they did that because it was expensive. And more importantly, they had a huge data problem. You see, training a chatbot today is very easy because we have the whole Internet at our disposal. It's just text, textbook, courses, everything. Just read and learn it. But for robots moving around in the real world, not so much. But wait, I hear you asking Caroj. We have millions of videos on YouTube and you know the real world is already there. Just look at what humans do. Well, not so fast. This data also needs to be labeled. And we can do that by demonstrating a set of movements to a robot to learn from each. Yes, but unfortunately we would have to do this for every task out there. So we would need millions and millions of labeled demonstrations. Labeled. That means we need to know who is doing what exactly for each of these examples. Labeling all this data is tiring and there is just not enough time. So here comes the crazy secret sauce. They use a system called Omniverse to create a video game version of the world so super accurate you can have a whole factory in it. Everything is digital and everything is labeled, but it does not always look realistic enough. So now let's plug it into Cosmos, their system that can take the video game footage as a baseline and create tons and tons of new realistic videos, and hence tons of training data as much as you want, infinite, practically and most importantly, all labeled. And don't forget, this also ensures that all of these videos are grounded in reality, where physics works, how physics should really be working. And note that with human demonstrations, we are limited by our time in our lives. But with a video game world plus the video generator, we can simulate as quickly as our hardware can do it. And some of the Omniverse works can simulate more than 25 years worth of data in just one day. In real life, which is kind of mind blowing. But surprisingly, that is still not enough. That is just secret sauce number one. Now we need secret sauce number two. This is going to be insane. I mean, this is not a 36 page research paper for nothing. So there are still tons and tons of videos out there on the Internet unlabeled. And then researchers say, well, if it is unlabeled, just teach the AI to do the labeling. Yes, really, I couldn't believe it when I saw it. It looks at the videos and extracts all the useful information out of them. For instance, did the camera move? Where did it move? But not just that. Also, which joint does what, what action is happening on the screen and so on with that. Every frame is labeled with annotations like actions, joints, goals and more. Bam. Get it? Now? This is video game data from reality. My mind is blown again. And thus reality can now be used as proper training data too. As if it were a video game. That is insanity. So in short, this is super smart and can learn from everything. Teleoperation, data, simulation data, Internet data, doesn't matter. It can eat all of it and make sense of it too. Loving it. Now, surprisingly, even this is not good enough to make humanoid robotics work. This problem is super challenging, borderline impossible. But don't despair, because here comes secret sauce number three. You see that? This is going to be super yummy. What is that? Well, for robots to understand the world around them, they are building on a paper called Eagle 2. The this is a vision language model. That is incredible. You see here the second law of papers at work. Everything is connected. Scientists are working all around the globe to build on each other's work to create a better world for us. So why is this super cool? Well, it helps a robot to think on two different levels. System 2 slow thinking with reasoning to understand the world and make a plan. And we also need the faster system one thinking that generates motor actions in real time. Real time is the key here. Just one of the two is not enough. Just system two can make a plan. But that is just a plan. It is also too slow. It cannot act in real time. But system one can move in real time, but does not know what the effect of its movements will be. And would you look at that. The fast system 1 neural network is a diffusion model. That's really weird. You see, diffusion models are often used to create images from a bunch of noise. So what does this have to do with motor actions? Well, get this. It really starts out from noise and denoises it until we get not a smooth image, but smooth motor action. It thinks about motion as others think about images. That is absolutely incredible stuff. So combine the two and you get what exactly? Well, hold on to your papers, fellow scholars, because when comparing it to a previous method, we go from a 46% success rate to 76%. That is absolutely amazing. A result that just a few years ago would have taken us up to a decade to achieve. And now here it is. And it is so much better than anything else we've had before. A complete game changer. And this is why I think Groot N1 is going to kick off a robotics revolution. Useful robots that do helpful things for us are now finally within reach. Bravo. And for some reason I barely see anyone talking about this crazy. But this is why two minute papers exist. And now you also know part of the reason how these robots robots can learn to be super cute as well. Subscribe and hit the bell icon if you appreciate this. Now, yes, we are fellow scholars here, so we also look at the limitations of these works and that is one of them. This is not a turnkey solution yet that you can just deploy and have it fold your laundry at home. Perhaps two more papers down the line. But then it might save your marriage by doing all the folding. Also, this is still about short tasks, which are mostly about mingling with things on a table. But the good news is that the model is free and fully open for all of us. And people can fine tune it to be better on their own tasks. In fact, this is not just a promise. You fellow scholars are already using it for smaller projects. Just listen to the joy. Drop, drop, drop. Amazing. And as you see, it also works for different embodiments so you can train it on your particular robot. What an incredible paper. What a time to be alive. And I waited with this video a bit to see how you fellow scholars are using it. It is not the best for views, but it gets a better video for you. So now let the experiments begin. Here you see me running the Full deep seq AI model through Lambda GPU cloud. 670 billion parameters. Running super fast and super reliably. This is insane. I love it and I use it on a regular basis. Lamda provides you with powerful Nvidia GPUs to run your own chatbots and experiments. Seriously. Try it out now at Lambda AI Papers or click the link in the description.