A short YouTube monologue (2 Minute Papers) by Karoly Zsolnai-Feher describing rapid recent progress in neural graphics. The narrator contrasts earlier NeRF methods that required hours of training with successive papers (planoxels, then an NVIDIA paper) that reduce training time from minutes to seconds while improving quality, and highlights additional capabilities such as gigapixel image approximation, neural signed distance fields (geometry via distance-to-surface mapping), and neural radiance caching (learning light transport). The video also contains a sponsor message for Lambda GPU Cloud.
Chapters
Dear fellow scholars, this is 2 Minute Papers with Dr. Today we are going to do this and this and this. One of these applications is called a nerf. What is that? Nerfs mean that we have a collection of photos like these and magically create a video where we can fly through these photos. Yes. Typically scientists now use some sort of learning based AI method to fill in all this information between these photos. This is something that sounded like science fiction just a couple years ago. And now here we are. Now these are mostly learning based methods. Therefore these techniques need some training time. Wanna see how their results evolve over time? I surely do. So let's have a look together. This NERF paper was published about a year and a half or two years ago. We typically have to wait for at least a few hours for something to happen. Then came the planoxels paper with something that looks like black magic. Yes, that's right. This trains in a matter of minutes and it was published just two months ago. Such improvement in just two years. But here is Nvidia's new paper from about a month ago. And yes, I hear you asking Karoj, are you telling me that a two month old paper of this caliber is going to be outperformed by a 1 month old paper? Yes, that is exactly what I'm saying. Now hold on to your papers and look here. With the new method, the training takes what? Less time than I have to utter this sentence because it is already done. So first we wait from hours to days, then two years later it trains in minutes and a month later, just a month later it trains in a couple seconds, basically nearly instantly. And if we let it run for a bit longer, but still less than two minutes, it will not only outperform a naive technique, but will even provide better quality results than a previous method while training for about 10 times quicker. That is absolutely incredible. I would say that this is swift progress in machine learning research, but that word will not cut it here. This is truly something else. But if that wasn't enough, Nerfing is not the only thing this one can do. It can also approximate a gigapixel image. What is that? That is an image with tons of data in it. And the AI is asked to create a cheaper neural representation of this image. And we can just keep zooming in and, and zooming in and we still find new details there. And if you have been holding on to your paper so far, now squeeze that paper because what you see here is not the result but the whole training process itself. Really? Yes, really. Did you see it? Well, did you blink? Because if you did, you almost certainly missed it. This was also trained from scratch, like right in front of our eyes. But it's so quick that if you take just a moment to hold onto your papers a bit more tightly and you already missed it. Once again, a couple papers before this took several hours at the very least. That is outstanding and if we were done here, I would already be very happy. But we are not done yet. Not even close. It can still do two more amazing things. 1. This is a neural signed distance field it has produced that is a mapping from 3D coordinates in a virtual world to distance to a surface. Essentially, it learns the geometry of the object better because it knows what parts are inside and outside and it is blazing fast, surprisingly, even for objects with detailed geometry. And my favorite it can also do neural radiance caching. What is that? At the risk of simplifying the problem, essentially it is learning to perform a light transport simulation. It took me several years of research to be able to produce such a light simulation. So let's see how long it takes for the AI to learn to do this. Well, let's see. Holy mother of papers. Nvidia, what are you doing? I give up. As you see, the pace of progress in AI and computer graphics research is absolutely incredible. And even better, it is accelerating over time. Things that were wishful thinking ten years ago became not only possible, but are now easy over the span of just a couple of papers. I am stunned. What a time to be alive.
This episode has been supported by Lambda GPU Cloud. If you're looking for inexpensive cloud GPUs for AI, check out Lambda GPU Cloud. They've recently launched Quadro RTX 6000, RTX 8000 and V100 instances. And hold on to your papers because Lambda GPU Cloud can cost less than half of AWS and Azure plus they are the only cloud service with 48 gigabyte RTX 8000. Join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today. Our thanks to Lamda for their long standing support and for helping us make better videos for you.
Thanks for watching and for your generous support and I'll see you next time.
