Nvidia's new technique compresses 3D models from billions of triangles down to less than 1% of their original size while keeping them photorealistic. The system uses differentiable rendering to recover geometry and materials, achieving roughly a 200:1 reduction that works on animated characters too.
Dear fellow scholars, this is 2 Minute Papers with Dr. Today we are going to see how crazy good Nvidia's new system is at simplifying virtual objects. These objects are used to create photorealistic footage for feature length movies, virtual worlds and more. But here comes the problem. Sometimes these geometries are so detailed they are prohibitively expensive to store and render efficiently. Here are some examples from one of our papers that were quite challenging to iterate on and render. This took several minutes to render and always ate all the memory in my computer. So what can we do if we would still like to get crisp, high quality geometry but cheaper and quicker? Quicker? I'll show you in a moment. This is part of a super complex scene. Get this. It is so complex that it takes nearly 100 gigabytes of storage space to render just one image of this and is typically used for benchmarking rendering algorithms. This is the Nurburgring of light transport algorithms, if you will. Well, hold on to your papers because I said that. I'll show you in a moment why what we can do to get all this at a more affordable cost. But in fact you are looking at the results of the new method right now. Yes, parts of this image are the original geometry and other parts have already been simplified. So which is which? Do you see the difference? Please stop the video and let me know in the comments below. I'll wait. Thank you. So, let's see together. Yes, this is the original geometry that requires over 5 billion triangles. And this is the simplified one which. What can this really be? This uses less than 1% of the number of triangles. Compared to this, in fact, it's less than half a percent. That is insanity. This really means that about every 200 triangles are replaced with just one triangle and it still looks mostly the same. That sounds flat out impossible to me.
Wow.
So how does this witchcraft even work? Well now you see, this is the power of differentiable rendering. The problem formulation is as we tell the algorithm that. Here are the results that you need to get. Find the geometry and material properties that will result in this. It runs all this by means of optimization, which means that it will have a really crude initial guess that doesn't even seem to resemble the target geometry. But then over time it starts refining it and it gets closer and closer to the reference. This process is truly a sight to behold. Look at how beautifully it is approximating the target geometry. This looks very close and is much cheaper to store and render. I loved this example too. Previously, this differentiable rendering concept has been used to be able to take a photograph and find a photorealistic material model that we can put in our simulation program that matches it. This work did very well with materials, but it did not capture the geometry. This other work did something similar to this new paper, which means that it jointly found the geometry and material properties, but as you see, high frequency details were not as good as with this one you see here. These details are gone and now just two years and one paper later, we can get a piece of geometry that is so detailed that it needs billions of triangles and it can be simplified 200 to 1. Now if even that is not enough, admittedly it is still a little rudimentary, but it even works for animated characters. I wonder where we will be two more papers down the line from here and for now, wow. Scientists at Nvidia knocked it out of the park with this one. Huge congratulations to the team. What a time to be alive. So there you go. This was quite a ride and I hope you enjoyed it at least half as much as I did. And if you enjoyed it at least as much as I did and are thinking that this light transport thing is pretty cool and you would like to learn more about it. I held a master level course on this topic at the Technical University of Vienna. Since I was always teaching it to a handful of motivated students, I thought that the teachings shouldn't only be available for the privileged few who can afford a college education. But no, no. The teachings should be available for everyone. Free education for everyone. That's what I want. So the course is available free of charge for everyone. No strings attached. So make sure to click the link in the video description to get started. We write a full light simulation program from scratch there and learn about physics, the world around us, and more.
This episode has been supported by Lambda GPU Cloud. If you're looking for inexpensive cloud GPUs for AI CH, check out Lambda GPU Cloud. They've recently launched Quadro RTX 6000, RTX 8000 and V100 instances and hold on to your papers because Lambda GPU Cloud can cost less than half of AWS and Azure. Plus they are the only cloud service with 48 gigabyte RTX 8000. Join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances with workstations or servers. Make sure to go to lambdalabs.com papers to sign up for one of their amazing GPU instances today. Our thanks to Lambda for their longstanding support and for helping us make better videos for you.
Thanks for watching and for your generous support, and I'll see you next time.
