OpenAI trained humanoid agents to compete in a Sumo-like game using self-play, where agents fight progressively harder opponents. The agents developed complex behaviors without explicit instruction, and the researchers tested whether skills learned in simulation transferred to new scenarios.
Dear fellow scholars, this is two minute papers with.
Earlier we had an episode about OpenAI's.
Absolutely amazing algorithm that mastered Dota 2.
A competitive online battle arena game, and.
Managed to beat some of the best.
Players in the world in a relatively.
Limited one versus one game mode.
While the full five versus five version.
Of this learning algorithm is still in.
The works, scientists at OpenAI experimented with some self play in other games and.
Have found some remarkable results.
It was you can see here that.
Most of these amusing experiments take place in a made up 3D game with simulated physics. For instance, performing well with these humanoid creatures means controlling 17 actuated joints properly.
These agents use a reinforcement learning algorithm.
To maximize a reward. For instance, a Sumo Warrior gets 1000 points for pushing their opponent out of the ring.
The first interesting thing is that a learning curriculum was used, which means that.
The algorithm was allowed to explore on.
Their own by relaxing the string strict.
Scores that are given only when winning.
This is combined with the fact that.
These agents play against themselves led to.
Some remarkable emergent behaviors.
Here you can see with the score.
How much of a difference this curriculum makes.
And you also see that whenever a plot is symmetric, that means that they.
Are zero sum games.
So if one agent wins a given number of points, the other loses the same amount.
The self play part is also particularly.
Interesting as many agents are being trained.
In parallel at the same time.
And if we're talking about one versus.
One games, we have to create some.
Useful logic to decide who to pair with whom.
It seems that training against an older.
Version of a previously challenged opponent was the best strategy.
This makes sense because they are running a similar algorithm and for self play this means that the algorithm is asked.
To defeat an older version of itself.
If it can reliably do that, it.
Will lead to a smooth and predictable learning process.
It is kinda incredible to think about.
The fact that we have a virtual world with a bunch of simulated learning creatures and we are omnipotent beings trying to craft the optimal learning experience for them.
The perks of being a researcher in machine learning.
And we are even being paid for this.
Isn't this incredible?
Psst.
Don't tell anyone about this.
There are so many interesting results here.
And so much to talk about.
For instance, we haven't even talked about.
Transfer learning where these creatures learn to generalize their knowledge, learned from previous tasks.
To tackle new challenges more efficiently.
Make sure to have a look at.
The paper and the source code is available for everyone free of charge.
If you're one of those fellow tinkerers.
You'Ll be more than happy to look into the video. Description if you wish to hear more.
About transfer learning, subscribe and turn on.
Notifications because the next episode is going.
To be about some really cool results in this area.
Thanks for watching and for your generous.
Support, and I'll see you next time.
Sam.
