OpenAI Five Beats World Champion DOTA2 Team 2-0! 🤖

Dear fellow scholars, this is two minute papers with this episode has been sponsored by Lambda Labs. Not so long ago we talked about DeepMind's Alpha Star, an AI that was able to defeat top tier human players in StarCraft II, a complex real time strategy game. Of course I love talking about AIs that are developed to challenge pro gamers at a variety of difficult games. So this time around we will have a look at another major OpenAI 5 which is an AI that plays Dota 2, a multiplayer online battle arena game with a huge cult following. As this game requires long term strategic planning, it is a classic nightmare scenario for any AI. But OpenAI is no stranger to Dota 2. In 2017 they showed us an initial version of their AI that was able to play one versus one games with only one hero and was able to reliably beat Dandy, a world champion player. That was quite an achievement. However, of course this was meant to be a stepping stone towards playing the real Dota 2. Then in 2018 they unveiled OpenAI 5, an improved version of this AI that played 5 versus 5 games with a limited hero pool. This team was able to defeat competent players, but was still not quite at the level of a world champion human team. In a one hour interview, the OpenAI research team mentioned that due to the deadline of the international event, they had to make quite a few concessions. And this time several things have changed. First, they didn't just challenge some local team of formidable players. No, no, they flat out challenged og, the reigning world champion team. An ambitious move that exudes confidence from their side. Second, this time around there was no tight deadline as the date of the challenge was chosen by OpenAI. Let's quickly talk about the rules of the competition and then see if OpenAI's confident move was justified. These learning agents don't look at the pixels of the game and as a result they see the world as a big bunch of numbers. And this time around it was able to play a pool of 17 heroes and trained against itself for millions and millions of games. And now let's have a look at what happened in this best of three series. In match one, right after picking the roster of Heroes, the AI estimated its win probability to be 67%. So it was quite a surprise that early on it looked like OpenAI's bots were running around aimlessly. Over time we found out that it was not at all the case. It plays unusually aggressively from the get go and uses buybacks quite liberally at times where human players don't really consider it to be a good choice. These buybacks resurrect a perished hero quickly, but in return cost money. Later it became clearer that these bots are no joke. They know exactly when to engage and when to back out from an engagement with the smallest sliver of health left. I will show quite a few examples of those to you during this video, so stay tuned. A little less than 20 minutes in, we had a very even game won. If anything, OpenAI seemed a tiny bit behind and someone noted that we should perhaps ask the bots what they think about their chances. And then the AI. Yeah, no worries. We have a higher than 95% chance to win the game. This was such a pivotal moment that was very surprising for everyone. Of course, if you call out a win with confidence, you better go all the way and indeed win the game. Right? Right. And sure enough, they wiped out almost the entire world champion team of the human players immediately after Notel's pushed all.

The way back, a Fischer will come out to hold them down. But OpenAI, they've got two more kills. Seb and Jarex fall a third as well as they snipe down n0tail right up to the high ground. The depro has the Yule scepter controlling Anna. Anna sort of left on his and.

Then noted, you know what, Remember that we just said forget that we estimate our chances to win to be above 99% now and shortly after they won match number one. Can you believe this? This is absolutely amazing. Interestingly, one of the developers said that the AI is great at assessing whether a fight is worth it. As an interesting corollary, if you engage with it and it fights you, it probably means that you are going to lose. That must be quite confusing for the players. Some mind games for you love it at the event. It was also such a joy to see such a receptive audience that understood and appreciated high level plays. Onwards to match number two. Right after the draft, which is the process of choosing the heroes for each team, the AI predicted a win percentage that was much closer this time around, around 60% in this game. The AI turned up the heat real fast and said just five minutes into the game, which is nothing, that it has an over 80% chance to win the game. And now watch this. Early in the game you can see a great example of where the AI just gets away with a sliver of health. Look at this guy.

To immediately kill him. They will find the follow up wraparound kill on towards him. At least they're trying. But with that warcry out he's able to run away. Another fissure comes out. Surely this kills gonna be there. But no. A stun from the Sven holds back the shaker and he tps out on 30 hp. OpenAI gets out of there with the Sven. They cannot get that kill.

OG look at that. This is either an accident or some unreal level foresight from the side of this agent. I'd love to hear your opinion on which one you think it is. By the 9 and a half minute mark, which is still really early, OpenAI 5 said yes, we got this one too. Over 95%. Here you see an interesting scenario where the AI loses one hero, but it almost immediately kills two of the human heroes and comes out favorably. At which point we wonder whether this was a deliberate bait it pulled on the humans.

They do at the disable hex comes out from N0tail, they'll kill up the Crystal Maiden but OpenAI Viper diving past the tower on towards top of the Sven coming back in on minimal HP to throw that stun or secure the kill as OpenAI again getting the favorable trade. Another tower taken. They are playing at a ferocious speed here in the second game.

By the 15 minute mark the human players lost a barracks and were heavily underfunded and outplayed with seemingly no way to come back. And sure enough by the 21 minute mark the game was over. There is no other way to say it. This second game was a one sided beatdown. Game one was a strategic back and forth where OpenAI5 waited for the right moment to win the game in a big team fight where here they pressure the human team from the get go and never let them reach the endgame where they might have an advantage with their picks. Also have a look at this.

Looking to play at full power. Yeah, it just looks fantastic. I wonder do they have a Force app if they get this kill? It's worth so much but they're not getting it. They're not getting it. This viper will not die.

Unreal. The final result is 20 for OpenAI. In the postmatch interview n0tail, one of the human players noted that he's confident that from five games they would take at least one and after 15 games they would start winning reliably. Very reminiscent of what we heard from players playing against DeepMind's AI in Starcraft 2 and I hope this will be tested. However, in the end he agreed that it is inevitable that this AI will become unbeatable at some point. It was also noted that in 5 versus 5 fights they seem better in planning than any human team is, and there is quite a lot to learn from the AI for us humans. They were also trying to guess the reasoning for all of these early buybacks. According to the players, initially they flat out seemed like misplays. Perhaps the reason for these instant and not really great buybacks might have been that the AI knows that if the game goes on for much longer, statistically their chances with their given composition to win the game dwindles. So it needs to immediately go and win right now, whatever the cost. And again, an important lesson is that in this project, OpenAI is not spending so much money and resources to just play video games. Dota 2 is a wonderful testbed to see how their AI compares to humans at complex tasks that involve strategy and teamwork. However, the ultimate goal is to reuse parts of this system for other complex problems outside of video games. For instance, the algorithm that you've seen here today can also do this. But wait, there's more. Players after these show matches always tend to get these messages from others on Twitter telling them what they did wrong and what they should have done instead. Well, luckily these people were able to show their prowess as OpenAI gave the chance for anyone in the world to challenge the OpenAI5 competitively and play against them online. This way not only team og, but everyone can get crushed by the AI. How cool is that? This arena event has concluded with over 15,000 games played where OpenAI 5 had a 99.4% win rate. There are still ways to beat it, but given the rate of progress of this project, likely not for long. Insanity as always. If you are interested in more details, I put a link to a Reddit AMA in the video description and I also can't wait to pick the algorithm apart for you. But for now we have to wait for the full paper to appear. And note that what happened here is not to be underestimated. Huge respect to the OpenAI team, to OG for the amazing games, and congratulations to the humans who were able to beat these beastly bots online. So there you go, another long video that's not two minutes and it's not about a paper yet. Welcome to Two Minute Papers. If you're doing deep learning, make sure to look into Lambda GPU systems. Lambda offers workstations, servers, laptops and the GPU cloud. For deep learning you can save up to 90% over AWS, GCP and Azure GPU instances. Every Lambda GPU system is Pre installed with TensorFlow, PyTorch and Keras. Just plug it in and start training. Lamda customers include Apple, Microsoft and stanford. Go to lambdalabs.compapers or click the link in the description to learn more. Big thanks to Lamda for supporting two Minute Papers and helping us make better videos. Thanks for watching and for your generous support and I'll see you next time.