DeepMind proposes seven capabilities to test AI agents beyond raw game scores: generalization, credit assignment, and scalability among them. The paper uses radar plots to measure performance and emphasizes the need for committee governance and public experiments to evaluate AI fairly.
Dear fellow scholars, this is two minute papers with a few years ago, scientists at DeepMind published a learning algorithm that they called Deep Reinforcement Learning which quickly took the world by storm. This technique is a combination of a neural network that processes the visual data that we see on the screen and a reinforcement learner that comes up with the gameplay related decisions which proved to be able to reach superhuman performance on computer games like a TORY breakout. This paper not only sparked quite a bit of mainstream media interest, but also provided fertile grounds for new follow up research works to emerge. For instance, one of these follow up papers infused these agents with a very human like quality curiosity, further improving many aspects of the original learning method. However, had a disadvantage. I kid you not, it it got addicted to the TV and kept staring at it forever. This was perhaps a little too human like. In any case, you may rest assured that this shortcoming has been remedied since and every follow up paper recorded their scores on a set of Atari games. Measuring and comparing is an important part of research and is absolutely necessary so we can compare new learning methods more objectively. It's like recording your time for the olympics at the 100 meter dash. In that case, it is quite easy to decide which athlete is the best. However, this is not so easy in AI research. In this paper, scientists at DeepMind note that just recording the scores doesn't give us enough information anymore. There's so much more to reinforcement learning algorithms than just scores. So they built a behavior suite that also evaluates the seven core capabilities of reinforcement learning learning algorithms. Among these seven core capabilities, they list generalization, which tells us how well the agent is expected to do in previously unseen environments, how good it is at credit assignment, which is a prominent problem in reinforcement learning. Credit assignment is very tricky to solve because for instance, when we play a strategy game we need to make a long sequence of strategic decisions and in the end if we lose an hour later, we have to figure out which one of these many many decisions led to our loss. Measuring this as one of the core capabilities was in my opinion, a great design decision here. How well the algorithm scales to larger problems also gets a spot as one of these core capabilities. I hope this testing suite will see widespread adoption in reinforcement learning research. And what I am really looking forward to is seeing these radar plots for newer algorithms which will quickly reveal whether have a new method that takes a different trade off than previous methods, or in other words, has the same area within the polygon but with a different shape, or in the case of a real breakthrough, the area of these polygons will start to increase. Luckily, a few of these charts are already available in the paper and they give us so much information about these methods. I could stare at them all day long and I cannot wait to see some newer methods appear here. Now note that there is a lot more to this paper. If you have a look at it in the video description, you will also find the experiments that are part of this suite, what makes a good environment to test these agents in, and that they plan to form a committee of prominent researchers to periodically review it. I love that part. If you enjoyed this video, please consider supporting us on Patreon. If you do, we can offer you early access to these videos so you can watch them before anyone else, or you can also get your name immortalized in the video description. Just click the link in the description if you wish to chip in. Thanks for watching and for your generous support and I'll see you next time.
