Researchers combined GPT-3 with physical robots so they can understand language and plan multi-step tasks. A robot successfully located a Coke can and threw it away, fetched a sponge, and brought water or an apple, with planning success rates around 70% on tasks up to 16 steps long.
Dear fellow scholars, this is 2 Minute Papers with Dr. Today you are going to see how one area of research can unlock new capabilities on a seemingly completely different area. We are now living the advent of amazing AI based language models. For instance, OpenAI's GPT3 technique is capable of all kinds of wizardry. For instance, finishing our sentences, or creating plots, spreadsheets, mathematical formulae and many other things. While their Dall E2AI is capable of generating incredible quality images from our written descriptions, even if they are too specific. Way too specific. Now note that all this wizardry is possible as long as we are dealing with text and and images. How about endowing a real robot moving around in the real world with this kind of understanding of language? I wonder what that could do. Well, check this out. Scenario 1. This little robot uses GPT3 and other language models. And it not only understands it, but it can also use this knowledge to help us. Don't believe it. Have a look. For instance, we can tell it that we spilled the Coke and ask it how it could help us out. And it recommends finding the Coke can, picking it up, going to the trash can, throwing it out, and bringing us a sponge. Yes, we will have to do the rest of it ourselves, but still. Wow. Good job, little robot. Now the key here is that it not only understands what we are asking, propose how to help, but hold on to your papers because here comes scenario two. Oh my. It also looks around, locates the most important objects required, and now it knows enough to make recommendations as to what to do. And of course, not all of these make sense. Look at that. It can say that it is very sorry about the mess. Well, thank you for the emotional support, little robot, but we also need some physical help here. Oh yes, that's better. And now look, it uses its eyes to look around. Yes, I have my hand. Well, does it work? Yes, it does. Great. Now, Coke can, trash can, sponge. It's time to make a cunning plan. Perfect. And it can also do plenty more. For instance, if we've been reading research papers all day and we feel a little tired, if we tell it, it can bring us a water bottle, hand it to us, and can even bring us an apple. Now, I'd love to see this. Be gentle. And oh my. Thank you so much. The amenities at Google HQ seem to have leveled up to the next level. And believe it or not, these were just really simple things it can do way better. Look, this is a plan that requires planning 16 steps ahead and it does not get stuck. And. And doesn't mess up too badly anywhere. This one is as close to a personal butler as it gets. Absolutely incredible. These are finally real robots that can help us with real tasks in the real world. So cool. What a time to be alive. Now this is truly an amazing paper, but make no mistake, not even this is perfect. For instance, the success rate for the Planning is about 70% and it can properly execute the plans most of the time, but clearly not all the time. The longer term planning results may also need to be a bit cherry picked to get a good one. It doesn't always succeed. Also note that all this is played at 10x speed so it takes a while. Clearly typing the message and waiting for the robot still takes longer than just doing the work. However, this is an excellent opportunity for us to apply the first law of papers, which says that research is a process. Do not look at where we are, look at where we will be. Two more papers down the line and if OpenAI's image generator AI Dall? E looked like this and just a year and the paper later it looks like this, well, just imagine what this will be able to do. Just a couple more papers down the line and what do you think? Does this get your mind going? If you have ideas for cool applications, let me know in the comments below. This episode has been supported by Cohere AI Cohere builds large language models and makes them available through an API so businesses can add advanced language understanding to their system or app quickly. With just one line of code, you can use your own data, whether it's text from customer service requests, legal contracts, or social media posts, to create your own custom models to understand text or even generate it. For instance, it can be used to automatically determine whether your messages are about your business hours, returns, or shipping. Or it can be used to generate a list of possible sentences you can use for your product descriptions. Make sure to go to Cohere AI Papers or click the link in the video description and give it a try. Today. It's super easy to use. Thanks for watching and for your generous support and I'll see you next time.
