A short YouTube commentary (Two Minute Papers #254) describing a 2018 method that advances facial reenactment beyond Face2Face (2016) by transferring head, torso, mouth interior, and gaze in real time without neural networks. The narrator highlights applications (self-reenactment for meetings, avatar digitization, telepresence, game character creation), notes limitations (illumination changes, artifacts with long hair and extreme motion), and frames the technique as a step toward full character reenactment for media.
Chapters
Dear fellow scholars, this is two minute papers with two years ago in 2016, we talked about a paper that enabled us to sit and transfer our gestures onto a virtual actor. This work went by the name Face2Face and showcased a bunch of mesmerizing results containing reenactments of famous political figureheads. It was quite amazing, but it is nothing compared to this one. And the reason for this is that the original Face to Face paper only transferred expressions, but this new work is capable of transferring head and torso movements as well. Not only that, but mouth interiors also appear more realistic and more gaze directions are also supported. You see in the comparison here that the original method disregarded many of these features and how much more convincing this new one is. This extended technique opens up the door to several really cool new applications. For instance, consider this self reenactment application. This means that you can reenact yourself. Now what would that be useful for, you may ask? Well, of course you can appear to be the most professional person during a virtual meeting, even when sitting at home in your undergarment. Or you can quickly switch teams based on who is winning the game. Avatar digitization is also possible. This basically means that we can create a stylized version of our likeness to be used in a video game, somewhat similar to the memoji presented in Apple's latest Keynote with the iPhone X. And the entire process takes place in real time without using neural networks. This is as good as it gets. What a time to be alive. Of course, like every other technique, this also has its own set of limitations. For instance, illumination changes in the environment are not always taken into account, and long haired subjects with extreme motion may cause artifacts to appear. In short, don't use this for rock concerts. And with this we are also one step closer to full character reenactment for movies, video games, and telepresence applications. This is still a new piece of technology and may offer many more applications that we haven't thought of yet. After all, when the Internet was invented, who thought that it could be used to order pizza or transfer Bitcoin? Or order pizza and pay with bitcoin? Anyway, if you have some more applications in mind, let's let me know in the comment section. Thanks for watching and for your generous support and I'll see you next time.
