Finally, Robotic Telekinesis is Here! 🤖

Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to perform something that seems like a true miracle. Oh yes, you are seeing it correctly. This is robotic telekinesis, moving objects from afar. So what is going on here? Well, we take a human operator who performs these gestures which are then transferred to a robot arm and then the magic happens. This is unbelievable. So now hold on to your papers and let's see how well it can pull off all this. Level one. Oh my. It comes out guns blazing. Look at that. It is not a brute. Not at all. With delicate movements, it can pick up these plush toys or even rotate a box. That is a fantastic start. But it can do even better. Level two, let's try to pick up those scissors. That is going to require some dexterous hand movements from the operator. And wow. Can you believe that by the time I am saying this, it is already done. And it can also stack these cups and.

Which is a difficult matter as its.

Own finger might get in the way. This was a little more of a close call, but it still managed. Bravo. So with all these amazing tasks, what could possibly be level three? Well, check this out. Yes, we are going to attempt to open this drawer. That is quite a challenge. Now note that it is slightly ajar to make sure the task is not too hard for its fingers. And let's see. Oh yeah. Good job, little robot. But what good is opening a drawer if we aren't doing anything with it? So here comes plus one, the final boss level. Open the drawer and pick up the cup. There is no way that this is possible through telekinesis. And oh my goodness, I love it. And note that this human has done extremely well with the hand motions. But is this person a robot whisperer or can anyone pull this off? That is what I would like to know. And wow, this works with other operators too. Which is good news because this means that it is relatively easy and intuitive to use. So much so, and get this, that these people are completely untrained operators. So cool. So now is the part where we expect the bad news to come. Where is the catch? Maybe we need some sinfully expensive camera gear to look at our hand to make this happen, Right? Well, if you have been holding on to your papers so far, now squeeze those papers because we don't need to buy anything crazy at all. What we need is just one uncalibrated color camera. Today, this is available in almost every single household. How cool is that? So if it can do that, all we need is a Bunch of training footage, right? But wait a second. Are you thinking what I am thinking? If we need just one uncalibrated color camera, can it be that. Yes, that is exactly right. We don't need to create any training data at all. We can just use YouTube videos that already exist out there in the wild. In a previous paper, scientists at DeepMind harnessed the power of YouTube by having their AI watch humans play games. And then they would ask the AI to solve hard exploration games. And it just ripped through these levels in Montezuma's Revenge and other games too. And here comes the best part. What was even more surprising there is that it didn't just perform an imitation of the teacher. No, no. It even outperformed its human teacher. Wow. I wonder if we could somehow do a variant of that in a future paper. Imagine what good we could do with that. Virtual surgeries. A surgeon could perform a life saving operation on anyone from anywhere else in the world. Wow, what a time to be alive. Now for that, the success rate needs to be much closer to 100% here. But still, this is incredible progress in AI research. And of course you are an experienced fellow scholar. So you don't forget to apply the first law of papers here which says that research is a process. Do not look at where we are, look at where we will be. Two more papers down the line and two more papers down the line. I bet this will not only be significantly more accurate, but I would dare to say that even full body retargeting will be possible. Yes, we could move around and have a real robot replicate our movements. Just let the computer graphics people in with their motion capture knowledge and this might really become a real thing soon. So does this get your mind going? What would you use this for? Let me know in the comments below. What you see here is a report of this exact paper we have talked.

About which was made by Weights and Biases. I put a link to it in the description. Make sure to have a look. I think it helps you understand this paper better.

Weights and Biases provides tools to track.

Your experiments in your deep learning projects. Using their system, you can create beautiful reports like this one to explain your findings to your colleagues better. It is used by many prestigious labs including OpenAI, Toyota Research, GitHub and more. And the best part is that Weights and Biases is free for all individuals, academics and open source projects. Make sure to Visit them through wnb.com papers or just click the link in the video description and you can get a free demo today. Our thanks to weights and biases for their long standing support and for helping us make better videos for you.

Thanks for watching and for your generous support and I'll see you next time.