Károly Zsolnai-Fehér covers a new dataset of 100,000 labeled videos designed to teach AI visual common sense. The dataset captures everyday actions like moving, picking up, and pouring that existing video datasets miss. Early results are promising but not yet superhuman.
Dear fellow scholars, this is two minute papers with Today we are going to talk about a new endeavor to teach some more common sense to learning algorithms. If you remember, in an earlier episode we talked about an excellent work by Andrej Karpathy who built an algorithm that looked at an input image and described in a full, well formed sentence what is depicted there. By the way, he recently became Director of AI at Tesla. Before that he worked at OpenAI freshly after graduating with a PhD. Now that is a scholarly career if I've ever seen one. Reading about this earlier work was one of those moments when I really had to hold on to my papers, not to fall out of the chair. But of course, as it should be with every new breakthrough, the failure cases were thoroughly discussed. One of the motivations for this new work is that we could improve the results by creating a video database that contains a ton of commonly occurring events that would be useful to learn. These events include moving and picking up or holding, poking, throwing, pouring or plugging in different things and much more. The goal is that these neural algorithms would get tons of training data for these and would be able to distinguish whether a human is showing them something or just moving things about. The already existing video databases are surprisingly sparse in this sort of information and in this new freshly published dataset we can learn on a hundred thousand enabled videos to accelerate research in this direction. I love how many of these works are intertwined and how follow up research works try to address the weaknesses of previous techniques. Some initial results with learning on this dataset are also reported to kick things off and they seem quite good if you look at the results here. But since this was not the focus of the paper, we shouldn't expect superhuman performance. However, as almost all papers in research are stepping stones. Two more follow up papers down the line. This will be an entirely different discussion. I'd love to report back to you on the progress later. Super excited for that. Thanks for watching and for your generous support and I'll see you next time.
