NVIDIA's New AI: Enhance! 🔍

Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to see how Nvidia's new AI can do three seemingly impossible things with just one elegant technique. For reference, here is a previous method that can take a collection of photos like these and magically create a video where we can fly through these photos. This is what we call a nerve based technique and these are truly amazing. Essentially photos go in and reality comes out. So I know you're thinking Karoj, this looks like science fiction. Can even this be topped? And the answer is yes. Yes it can. Now you see this new technique can also look at a small collection of photos and and be it people or cats, it learns to create a continuous video of them. These look fantastic. And remember, most of the information that you see here is synthetic, which means it is created by the AI. So good. But wait, hold on to your papers because there is a twist. It is often the case for some techniques that they think in terms of photos, while other techniques think in terms of volumes. And get this, this is a hybrid technique that thinks in terms of both. Ok, so what does that mean? It means this. Yes, it also learned to not only generate these photos, but also the 3D geometry of these models at the same time. And this quality for the results is truly something else. Look at how previous techniques struggle with the same task.

Wow.

They are completely different than the input model. And you might think that of course they are not so good. They are probably very old methods. Well, not quite. Look, these are not some ancient techniques. For instance in Giraffe is from the end of 2020 to the end of 2021 depending on which variant they used. And now let's see what the new method does on the same data. Wow. My goodness. Now that is something. Such improvement in so little time. The pace of progress in AI research is nothing short of amazing. And not only that, but everything it produces is multi view consistent. This means that we don't see a significant amount of flickering as we rotate these models. There is a tiny bit on the fur of the cats, but other than that, very little. That is a super important usability feature. But wait, it does even more. Two, it can also perform one of our favorites, super resolution. What is super resolution? Simple. A coarse pixelated image goes in and what comes out of course, a beautiful detailed image. How cool is that? And here comes number three. It projects these images into a latent space. What does that mean? A latent space is a made up place where we are trying to organize data in a way that similar things are close to each other. In our earlier work we were looking to generate hundreds of variants of a material model to populate this scene. In this latent space, we can concoct all of these really cool digital material models. A link to this work is available in the video description. Now let's see. Yes, when we take a walk in the internal latent space of this technique, the we can pick a starting point, a human face, and generate these animations as this face morphs into other possible human faces. In short, it can generate a variety of different people, virtual people. We get the 3D geometry for their heads too. With this we are one step closer to democratizing the creation of virtual humans in in our virtual worlds. What a time to be alive. And if you have been holding onto your paper so far, now squeeze that paper because get this, you can do all of this in real time and.

All of The EWS and Azure plus they are the only cloud service with 48 gigabyte RTX 8000. Join researchers at organizations like Apple and MIT and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today. Our thanks to Lamda for their long standing support and for helping us make.

Better videos for you. Thanks for watching and for your generous support and I'll see you next time.