NVIDIA’s AI: Amazing DeepFakes And Virtual Avatars!

Dear fellow scholars, this is 2 Minute Papers with Dr. Today we are going to see that the research papers that you see here in this series are real. Here you see Nvidia's game changing video conferencing AI. So what does this do? Why is this so interesting? How does this transfer a video of us over the Internet? Well, here is a crazy idea. It doesn't. What? Transmitting video without transmitting video. How is that even possible? Well now it is possible. What they do in this work is take only the first image from the video and they throw away the entire video afterwards. But before that, it stores a tiny bit of information from it, which is how our head is moving over time and how our expressions change. That is an absolutely outrageous idea. Except the fact that it works. And it not only works, but it works really well. And because this is an amazing paper, it does not stop there, it can do even more. Look at these two previous methods trying to frontalize the input video. This means that we look to the side a little and the algorithm synthesizes a new image of us as if the camera was right in front of us. That sounds like a science fiction movie. Except that it seems absolutely impossible given how much these techniques are struggling with the task. Until we look at the new method. My goodness. There is some jumpiness in the neck movement in the output video here and some warping issues here, but otherwise very impressive results. Now, if you have been holding onto your paper so far, squeeze that paper. Because these previous methods are not some ancient papers that were published a long time ago. Not at all. Both of them were published within the same year as the new paper. How amazing is that? Wow. And it could also perform deep fakes too. Look, we only need one image of the target person and we can transfer all of our gestures to them in a way that is significantly better than most previous methods. Now, of course, not even this technique was perfect. It still struggled a great deal in the presence of occluder objects. But still, just the fact that this is now possible feels like we are living in the future. What a time to be alive. Now I said that today we are going to see that this paper is as real as it gets. So what does that mean? Well, today anyone can try this technique. Yes, just one year after publishing this paper and it is now available as a demo. The link to it is available in in the video description below. And get this. Some people at Nvidia are already using it for their virtual meetings. And by it, I mean both the compression engine where the previous industry standard compression algorithm could do this when given very little data. That is not much. But the new technique with the same amount of data can now do this. That is insanity. Loving it. And they also use their own gestures to create these deepfakes and make virtual characters come alive or to frontalize their videos when talking to each other. It almost feels like we are living in a science fiction movie and all this is out there for us to use. Also, these technologies will soon be part of the Nvidia Video Codec SDK as the AI face codec, which means that it will be soon deployed to an even wider audience to use. These companies are already using it. So this is one more amazing example that shows that the papers that you see here in two minute papers are real. Sometimes so real that we can go from a research paper to a product in just a year. That is absolutely miraculous. So what do you think? What would you use this for? Let me know in the comments below. This video has been supported by weights and biases. Look at this. They have a great community forum that aims to make you the best machine learning engineer you can be. You see, I always get messages from you fellow scholars telling me that you have been inspired by the series but don't really know where to start. And here it is. In this forum you can share your projects, ask for advice, look for collaborators and more. Make sure to visit wnb Me paperforum and say hi or just click the link in the video description. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.