Károly Zsolnai-Fehér explains how AI learns to track people and objects in video without labeled training data. The method looks at nearby frames to correct its own predictions, handles brief occlusions and rotation, and nearly matches supervised systems on some tasks, though gains plateau with more data.
Dear fellow scholars, this is two minute papers with There are many AI techniques that are able to look at a still image and identify objects, textures, human poses and object parts in them really well. However, in the age of the Internet we have videos everywhere, so an important question would be how we could do the same for these animations. One of the key ideas in this paper is that the frames of these videos are not completely independent and they share a lot of information. So after we make our initial predictions on what is where exactly, these predictions from the previous frame can almost always be reused with a little modification. Not only that, but here you can see with these results that it can also deal with momentary occlusions and is ready to track objects that rotate over time. A key part of this method is that one, it looks back and forth in these videos to update these labels, and second, it learns in a self supervised manner, which means that all it is given is just a little more than data, and was never given a nice data set with explicit labels of these regions and object parts that it could learn from. You can see in this comparison table that this is not the only method that works for videos. The paper contains ample comparisons against other methods and comes out ahead of all other unsupervised methods. And on this task it can even get quite close to supervised methods. The supervised methods are the ones that have access to these cushy labeled data sets and therefore should come out way ahead. But they don't, which sounds like witchcraft considering that this technique is learning on its own. However, all this greatness comes with limitations. One of the bigger ones is that even though it does extremely well, it also plateaus, meaning that we don't see a great deal of improvement if we add more training data. Now, whether this is because it is doing nearly as well as it is humanly or computerly possible, or because a more general problem formulation is still possible, remains a question. I hope we find out soon. Thanks for watching and for your generous support and I'll see you next time.
