Károly Zsolnai-Fehér explains how neural networks trained only on synthetic virtual humans can detect faces and landmarks in real videos nearly as well as models trained on actual people. The synthetic approach avoids privacy issues and annotation costs, though it struggles with eye tracking and works best with varied lighting and head rotations.
Dear fellow scholars, this is two minute.
Papers with Dr. None of these faces are real. And today we are going to find out whether these synthetic humans can in a way, pass for real humans, but not quite in the sense that you might think. Through the power of computer graphics algorithms, we are able to create virtual worlds. And of course, within those virtual worlds, virtual humans too. So here is a wacky idea. If we have all this virtual data, why not use these instead of real photos to train new neural networks? Hmm. Wait a second. Maybe this idea is not so wacky after all. Especially because we can generate as many virtual humans as we wish. And all this data is perfectly annotated. The location and shape of the eyebrows is known even when they are occluded. And we know the depth and the geometry of every single hair strand of the beard. If done well, there will be no issues about the identity of the subjects or the distribution of the data. Also, we are not limited by our wardrobe or the environments we have access to. In this virtual world, we can do anything we wish.
So good.
But of course, here is the ultimate question that decides the fate of this project. And that question, does this work? Now we can use all this data to train a neural network. And the crazy thing about this is that this neural network never saw a real human. So here is the ultimate test. Videos of real humans. Now hold on to your papers and let's see if the little AI can label the image and find the important landmarks. Wow. I can hardly believe my eyes. It not only does it for a still image, but for even a video. And it is so accurate from frame to frame that no flickering artifacts emerge. That is outstanding. And get this, the measurements say that it can stand up to other state of the art detector neural networks that were trained on real human faces. And oh my. Are you seeing what I am seeing? Can this really be? If we use the same neural network to learn on real human faces, it won't do better at all. In fact, it will do worse than the same AI with the virtual data. That is insanity. The advantages of this practically infinitely flexible synthetic dataset should shows really well here. The paper also discusses in detail that this only holds if we use the synthetic dataset well and include different rotations and lighting environments for the same photos. Something that is not always so easy in real environments. Now, this test was called face parsing. And now comes landmark detection. This also works remarkably well. But wait a second. Once again, are you seeing what I am seeing? The landmarks can rotate all they want and it will know where they should be even if they are occluded by headphones, the hair, or even when they are not visible at all. Now of course not even this technique is perfect. Tracking the eyes correctly requires additional considerations and real data, but only only for that which is relatively easy to produce. Also, the simulator at this point can only generate the head and the neck regions and nothing else. But you know the drill. A couple papers down the line and I am sure that this will be able to generate full human bodies.
This episode has been supported by Lambda GPU Cloud. If you're looking for inexpensive cloud GPUs for AI channel, check out Lambda GPU Cloud. They've recently launched Quadro RTX 6000, RTX 8000 and V100 instances and hold on to your papers because Lambda GPU Cloud can cost less than half of AWS and Azure. Plus they are the only cloud service with 48 gigabyte RTX 8000. Join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances with workstations or servers. Make sure to go to lambdalabs. Com papers to sign up for one of their amazing GPU instances today. Our thanks to Lambda for their long standing support and for helping us make.
Better videos for you.
Thanks for watching and for your generous.
Support and I'll see you next time.
