Dr. Zsolnai explains how AI can catch bugs and glitches that human testers miss during game development. AI automates playtesting to find problems faster and more thoroughly, with Weights and Biases helping organize the experiments.
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Have you ever got stuck in a video game or found a glitch that would prevent you from finishing it? As many of you know, most well known computer games undergo a ton of playtesting, an important step that is supposed to unveil these issues. So how is it possible that all these bugs and glitches still make it to the final product? Why did the creators not find these issues? Well, you see, playtesting is often done by humans. That sounds like a good thing. And it often is. But here comes the problem. Whenever we change something in the game, our changes may also have unintended consequences somewhere else, away from where we applied them. New oversights may appear elsewhere. For instance, moving a platform may make the level more playable. However, also this might happen. The player may now be able to enter a part of the level that shouldn't be accessible, or be more likely to encounter a collision bug and get stuck. Unfortunately, all this means that it's not enough to just test what we have changed, but we have to retest the whole level of, or maybe the whole game itself for every single change, no matter how small. That not only takes a ton of time and effort, but is often flat out impractical. So what is the solution? Apparently a proper solution would require asking tons of curious humans to test the game. But wait a second. We already have curious learning based algorithms. Can we use them for playtesting? That sounds amazing. Well, yes, until we try it. You see, here is an automated agent, but a naive one, trying to explore the level. Unfortunately, it seems to have missed half the map. Well, that's not the rigorous testing we are looking for, is it? Let's see what this new AI offers. Can it do is any better? Oh my. Now we're talking. The new technique was able to explore not less than 50%, but a whopping 95% of the map. Excellent. But we are experienced fellow scholars over here, so of course we have some questions. Apparently this one has great coverage, so it can cruise around. Great. But our question can these AI agents really find game breaking issues? Well, look, it just found a bug where it could climb to the top of the platform without having to use the elevator. It can also build a graph that describes which parts of the level are accessible and through what path. Look, this visualization tells us about the earlier issue where one could climb the wall through an unintentional issue. And after the level designer supposedly fixed it by adjusting the steepness of the wall. Let's see. The new path. Yes. Now it could only get up there by using the elevator. That is the intended way to traverse the level. Excellent. And it gets better. It can even tell us the trajectories that enabled it to leave the map so we know exactly what issues we need to fix without having to look through hours and hours of video footage. And whenever we apply the fixes, we can easily unleash another bunch of these AIs to search every nook and cranny and try these crazy strategies, even ones that don't make any sense but appear to work well. So how long does this take? Well, the new method can explore half the map in approximately an hour or two, can explore 90% of the map in about 28 hours, and if we give it a couple more days it goes up to about 95%. That is quite a bit, so we don't get immediate feedback as soon as we change something. Since this method is geared towards curiosity and not efficiency, note that this is just the first crack at the problem and I would not be surprised if just one more paper down the line. This would take about an hour and two more papers down the line. It might even be done in a matter of minutes. What a time to be alive. This episode has been supported by Weights and Biases. In this post they show you how to use sweeps to automate hyperparameter optimization and explore the space of possible models and find the best one. During my PhD studies I trained a ton of neural networks which were used in our experiments. However, over time there was just too much data in our repositories and what I am looking for is not data but insight. And that's exactly how Weights and Biases helps you by organizing your experiments. It is used by more than 200 companies and research institutions including OpenAI, Toyota Research, GitHub and more. And get this Weights and Biases is free for all individuals, academics and open source projects. Make sure to Visit them through wnba.com papers or just click the link in the video description and you can get a free demo today. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.
