KƔroly Zsolnai-FehƩr covers a new method that reconstructs full 3D hair geometry from a single photo using neural networks. One network predicts hair shape per frame, another estimates motion between frames. It works on short and long hair, handles motion and forces, and matches multi-view results with far less data.
Dear fellow scholars, this is two minute papers with. In this series we talk about research on all kinds of physics simulations, including fluids, collision physics, and we have even ventured into hair simulations. If you look here at this beautiful footage, you may be surprised to know how many moving parts a researcher has to get right to get something like this. For instance, some of these simulations have to go down to the level of computing the physics between individual hair strands. If it is done well, like what you see here from our earlier episode, these simulations will properly show us how things should move. But that's not all. There is also an abundance of research works out there on how they should look. And even then we are not done, because before that we have to take a step back and somehow create these digital 3D models that show us the geometry of these flamboyant hairstyles. Approximately 300 episodes ago, we talked about a technique that took a photograph as an input and created a digital 3D model that we can use in our simulations and rendering systems. It had a really cool idea where it initially predicted a coarse result and then this result was matched with the hairstyles found in public data repositories and the closest match was presented to us. Clearly, this often meant that we got something that was similar to the photograph, but often not exactly the hairstyle we were seeking. And now hold on to your papers, because this work introduces a learning based framework that can create a full reconstruction by itself without external help. And now squeeze that paper because it works not only for images, but for videos too. It works for shorter hairstyles, long hair, and even takes into consideration motion and external forces as well. The heart of the architecture behind this technique is this pair of neural networks where the one above creates the predicted hair geometry for each frame, while the other tries to look backwards in the data and try to predict the appropriate motions that should be present. Interestingly, it only needs two consecutive frames to make these predictions, and adding more information does not seem to improve its results. That is very little data. Quite remarkable. Also note that there are a lot of moving parts here in the full paper. For instance, this motion is first predicted in 2D and is then extrapolated to 3D afterwards. Let's have a look at this comparison. Indeed, it seems to produce smoother and more appealing results than this older technique. But if we look here, this other method seems even better. So what about that? Well, this method had access to multiple views of the model, which is significantly more information than what this new technique has that only needs a simple monocular 2D video from our phone or from the Internet. The fact that they are even comparable is absolutely amazing. If you have a look at the paper you will see that it even contains a hair growing component in this architecture. And as you see the progress in computer graphics research is absolutely amazing and we are even being paid for this. Unreal. This episode has been supported by Linode. Linode is the world's largest independent cloud computing provider. They offer affordable GPU instances featuring the Quadro RTX 6000 which is tailor made for AI, scientific computing and computer graphics projects. Exactly the kind of works you see here in this series. If you feel inspired by these works and you wish to run your experiments or deploy your already existing works through a simple and reliable hosting service, make sure to join over 800,000 other happy customers and choose Linode. To spin up your own GPU instance and receive a $20 free credit, visit linode.com papers or click the link in the description and use the promo code Papers20 during signup. Give it a try today. Our thanks to Linode for supporting the series and helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.
