AI-Based Animoji Without The iPhone X | Two Minute Papers #236

Dear fellow scholars, this is two minute papers with.

Many of you have surely heard the word animoji, which refers to these emoji figures that are animated in real time and react to our facial gestures.

This is implemented in the new iPhone X phones.

However, to accomplish this, it uses a.

Dot projector to get a good enough.

Understanding of the geometry of the human face.

So how about a technique that doesn't need any specialized gear, takes not even.

A video of you, but one photograph.

As an input, and creates a digital avatar of us that can be animated in real time. Well, sign me up. Have a look at these incredible results. As you can see, the final result also includes secondary components like eyes, teeth, tongue and gum.

Now, the avatars don't have to be fully photorealistic, but have to capture the appearance and gestures of the user well.

Enough so they can be used in video games or or any telepresence application.

Where a set of users interact in a virtual world.

As opposed to many prior works, the hair is not reconstructed strand by strand because doing this in real time is not feasible.

Also note that the information we are.

Given is highly incomplete because the backside of the head is not captured. But these characters also have a quite.

Appropriate looking hairstyle there.

So how is this even possible?

Well, first the input image is segmented into the face part and the hair part.

Then the hair part is run through a neural network that tries to extract attributes like length, spikiness, are there hair bands? Is there a ponytail where the hairline is?

And more.

This is an extremely deep neural network.

With over 50 layers and it took 40,000 images of different hairstyles to train.

Now, since it is highly unlikely that.

The input photo shows someone with a hairstyle that was never ever worn by anyone else, we can look into a big data set of already existing hairstyles and choose the closest one that fits the attributes extracted by the neural network.

Such a smart idea.

Loving it.

You can see how well this works in practice. And in the next step, the movement.

And appearance of the final hair geometry can be computed in real time through a novel polygonal strip representation. The technique also supports retargeting, which means that our gestures can be transferred to different characters. The framework is also very robust to different lighting conditions, which means that a differently lit photograph will lead to very similar outputs. The same applies for expressions.

This is one of those highly desirable.

Details that makes or breaks the usability of a new technique in production environments. And this one passed with flying colors.

In these comparisons, you can also see.

That the quality of the results also smokes the competition. A variant of the technology can be downloaded through the link in the video description.

Thanks for watching and for your generous.

Support, and I'll see you next time.