OpenAI trained an AI agent to play Minecraft by watching tutorial videos. The agent can complete long-horizon tasks like crafting diamond tools and performing pillar jumps, learning from unlabeled video data without explicit instruction.
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to see what happens if we unleash an AI to watch Minecraft videos and then ask this AI to play it. And the big question is, can it play as well as we humans do? Well, you'll get your answer today. This is OpenAI's project, which is not to be confused with Nvidia's similar project called Mind Dojo. So what was that? That was an AI project that watched a ton of YouTube tutorial videos on Minecraft. Minecraft is an open world sandbox game, one of the most played games in the world. And this is a true sandbox, which means that you can do whatever you wish. You create the story of this game, gather raw materials, build something, explore. Or don't. It's all up to you. This is a sandbox game and your story, after all. So what could Nvidia's AI do after watching these tutorial videos? Well, a great deal. For instance, we could even use natural language and ask it to explore an ocean monument. I really like this one because the word explore is sufficiently vague and open ended. A human understands what exploring means. But does the AI understand it too? Well, as you see, it does. Amazing. It could also encircle these llamas with a fence. Or it could also be dropped into an arena with an ender dragon, the final boss of the game, if you will. However, look, this is a very limited setting for this task as the dragon is not charging at the AI. But at the very least, Nvidia's AI understood what fighting meant in this game. Now, in this video, I set the following quoting. Starting this game from scratch and building up everything to defeat such a dragon takes a long time horizon and will be an excellent benchmark for the next AI. One more paper down the line. I'd love to see it perform this. Start to end. Make sure to subscribe. If such a paper appears, I'll be here to show it to you. End quote. And I thought maybe a year or two later there will be some activity here. And now hold on to your papers because OpenAI scientists are here with their own spin of this project. Not one year later, but just one week later. Wow. And with this paper, I also got what I was looking for, which is an AI that understands longer time horizons. You see, they make a huge claim. They say that this is the first AI that is capable of crafting diamond tools. That is very ambitious. Can that really be? Well, I say that I will believe it when I see it do you see this long sequence of subtasks? We need to do all this correctly and in the correct order to be able to perform that. This takes up to 24,000 actions and 20 minutes even for a human player. So let's see if the AI is also up for the task. It starts chopping wood right away. Then the planks are coming along. Later it starts mining, finds iron ore, crafts a furnace, starts smelting. And at this point I am starting to think that that it might make it. That would be absolutely amazing. However, not so fast. We still need some diamonds. Can it find some diamonds? It takes a while and oh my Gotem. And from then on we have established that this is a smart AI. So from here on out, this is a piece of cake. There we go. I can't believe it. A diamond pickaxe. And here is my other favorite. It can also perform pillar jumps. What is that? This is a cool technique where we jump and while mid air we quickly put a block below us and ta da, we are now standing a little higher. If we do this over and over again, we will be able to reach places that otherwise would be way out of reach. And the AI learned this too. How cool is that? And while we are looking at what else it is capable of, I would like to bring up one more remarkable thing here. Here goes. It learns from unlabeled videos. This means that the AI is not told what these videos are about. It is just allowed to watch them and find out for itself. And to be able to learn this much information from this kind of unlabeled data is just an incredible sign for future projects to come. You see, playing Minecraft well is great, but this is an AI that can learn new things from unstructured data. In the future, these agents will be able to perform more general non gaming related tasks. What a time to be alive. For instance, the amazing DeepMind lab is already walking this path. First they wrote an AI that could learn to play chess, go and Starcraft 2. And then they used a similar system to solve protein folding. Now make no mistake, the protein folding system is not exactly the same system as the chess AI. But they share a few building blocks. And here having two amazing Minecraft AIs appear within a week of each other is a true testament to how quickly projects are coming along in AI research. So let's apply the first law of papers. What do you think this will be capable of? One more paper down the line? Let me know in the comments below. This video has been supported by weights and biases. Look at this. They have a great community forum that aims to make you the best machine learning engineer you can be. You see, I always get messages from you fellow scholars telling me that you have been inspired by the series but don't really know where to start. And here it is. In this forum you can share your projects, ask for advice, look for collaborators, and more. Make sure to visit wnb me paperforum and say hi or just click the link in the video description. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.
