NVIDIA’s Robot AI Finally Enters The Real World! 🤖

Köszönöm, főnök, főnökek! Ez a 2 Minute Papers, hozzávalók Dr. Károly Zsolnai-Fehér. A NVIDIA kutatói kutatói kutatói kutatói kutatói kutatói kutatói kutatói kutatói kutatói kutatói kutatói kutatói kutatói kutatói kutatói kutatói kutatói kutatói kutatói kutatói kutatói k This is not good at all. It should have grabbed the C-clamp here, not the toy school bus. Why is that? Well, it has only seen one demonstration to learn from yet, so these are just the humble beginnings. Now let's look at what it can do after seeing 10 demonstrations. And? Yes! That is so cool! Learning is happening! The C-clamp goes in the box successfully. Wonderful! And now, for the crayon box, that is not a crayon box. And neither is that one. And it's not that one either. Oh my goodness. Now, hold on to your papers, and let's see what it learned after looking at still as few as a hundred demonstrations. C-clamp into the box, crayon box too, toy school bus, get in that box, and yes! Finally! The rest seems to work fine too. I love it! Good job little AI! And if you were wondering, this is a porcelain plate. And that is twist number one. Learning happening from very few image-action pairs. How many exactly? Well, it learned to do this and more from only 179 image-action pairs. That in the era of learning-based techniques is almost nothing at all. And it gets better, one of the most mindblowing results is that from these very few examples, it also learned to deal with previously unseen colors and words. And not only that, but it was also able to learn a wide variety of tasks from these 179 examples. And its capabilities just keep going on and on and on. Absolutely amazing. Oké, now, that is all well and good, but still, this is a robot arm that can play a computer game. Is that it? Why make a video about this? Why is this so important? Well, now I give you the two more twists that breathe new life into this project. Twist number two. We can use natural language to give instructions to these robots. Yes! And twist number three, if you have been holding on to your paper so far, now squeeze that paper, because the knowledge it learned in the simulation transfers to reality. We come out of the video game world into the real world, and the AI still works wonderfully. Let's have a look! It can move chess pieces. De ez egy tisztesség volt. Tudod mit? Próbáljunk valamit, ami egy kicsit több delikusítása van. Nézzük meg! Ó, igen! Felszeletelni és elfelszeletelni egy kis tisztet is egy másik probléma. Ami tökéletes mozgásokat kéne. Nagyon jó! Szeretem! De próbáljunk valamit, ami még több finességet kéne. Igen! Cserébe tesszük! That is really cool. Sweeping also works. Kind of. As long as we are looking to sweep beans. And believe it or not, it can even read. Again, kind of. You see, we can label two boxes that will contain the good and bad stuff. And then instruct the AI as follows. Place the blue screwdriver in the bad box. This worked well, and it will work even if we try to be a bit more tricky and swap the labels. And as you see, it is not falling for it. I will have to give partial credit for the drop, but still, all this from less than 200 demonstrations, that is incredible. So cool! And what you see here is what researchers refer to as seem-to-real. Teaching an AI something in a simulated world, and then, deploying it into the real world. Oké, so what is all this good for? Here is an amazing example, look, OpenAI trained their robot hand in a simulation to be able to rotate these rubik cubes. And then deployed this software onto a real robot hand, and look! It can use this simulation knowledge És most is működik a valódi világban. De a szint a valódi világban is számítható, hogy a szelfi kocsit is. Nézd, a Tesla már működik a virtuális világokat készítéséhez, és a kocsit ott készíteni. Egy olyan lehetősége, hogy valószínűleg nem lehet, and potentially unsafe scenarios, but in these virtual worlds, the self-driving car can train itself safely. And when we deploy them into the real world, they will have all this knowledge. It is fantastic to see that Waymo is also moving in this direction. As you see, seem to real is real. And NVIDIA's new technique also puts us one step closer to a future where this kind of intelligence can be democratized and used by everyone. What a time to be alive! This episode has been supported by Lambda GPU Cloud. If you are looking for inexpensive cloud GPUs for AI, Check out Lambda GPU Cloud! Get this, they've recently launched an NVIDIA RTX A6000 with 48GB of memory. And hold on to your papers, because Lambda GPU Cloud can cost less than half of AWS and Azure. Thanks for watching and for your generous support, and I'll see you next time!