NVIDIA and University of Utah developed a ray-tracing technique that extracts 25 times more information per light path, reducing noise flicker and improving anti-aliasing and bokeh. The method works well in most cases but struggles with motion blur and certain pixel filters, and the code is publicly available.
Questions this video answers
The University of Utah and NVIDIA.
Approximately 25× more rays ("25 times more").
Yes — the video states the source code is publicly available.
2024-05-26
Chapters
Feast your eyes upon these beautiful images. Oh my goodness. And get this. None of them are real. None. Yes, that's right. These images were created through a technique called ray tracing. Simulating the path of light rays to create beautiful images and animations. And goodness, it would be an absolute dream to be able to create video games that look like this. So wait, why is not everything ray traced today? Well, there is a problem, and not a small one. And this problem is noise. Look, what is going on here? Well, during this process, we have to shoot millions and millions of light rays into the scene to estimate how much light is bouncing around. And before we have simulated enough rays, the. The inaccuracies in our estimations show up as noise in these images. Don't worry, this clears up over time as we simulate more rays. But it may take from minutes to days for this to happen, even for a smaller scene. For instance, in an earlier paper, this one took us three full weeks to finish. Three weeks? Yes. Really? I am not kidding. Ouch. So can we do this in real time? Well, here is a classical technique, path tracing, trying to create images in quick succession for real time applications. Whoa. This is a terrible disappointment. As you see, there is unfortunately plenty of noise here. But wait, noise filtering techniques exist that are specifically designed for light transport. And. And these do an absolutely amazing work. Look at how much more we know about this scene. But still, come on, if you look at this, who would really be able to guess what the real image should be? No one. Now wait a second. We have decades of research works in the area of ray tracing. So if we jump in time about 30 years, we get this. This is so much better. However, it is unfortunately still not that usable. So is that it is all hope last for getting these beautiful ray traced images in real time. Well, don't despair. Despite the fact that unfortunately the problem gets even worse. Dear fellow scholars, this is Two Minute Papers with Dr. Caro Zsolnai. How does it get even worse? Well, as soon as you start moving the camera, we get this thing that we call high frequency noise that makes almost all of this footage completely unusable. Even after noise filtering, there is just too much flickering. And now hold on to your papers, fellow scholars, because here is an absolutely miraculous new paper that can do this instead. This is a collaboration between the University of Utah and Nvidia and it not only looks so much better immediately. Wow. However, it has four more huge advantages that only reveal themselves to experienced fellow scholars with a keen eye. One, try to look at the frequency of the noise. It is of much lower frequency. In other words, it is a great deal less jumpy. So when we hand it over to the Denoiser it can do so much better with it. 2 It also has much better anti aliasing. What does that mean? Well, look, the edges are more visible, better resolved. Once again that is a lot more helpful for the Denoiser. 3 It is able to get more information out of each light path that we simulate. How much more? Well, 25 times more? Yes, these results are equivalent to 25 times more rays with previous techniques. That is a huge leap in capabilities. And four large glowing light sources that are not in focus called bokeh seem to be often completely gone with the previous methods for quite a while but are resolved better with this technique. I am absolutely stunned by these results. So good. Now despite all this, not even this technique is perfect. Three things here. One we don't yet know how well it generalizes. Two Motion blur things getting a little blurry when we move objects or the camera around. 2 if we need a super sharp still image and we are willing to wait for longer, there are some moving parts called pull pixel filters that this does not do well with. However, this is not super relevant for real time applications. And three it takes a tiny bit more time per frame than previous techniques, but it is totally worth it. And there are some more good news. The source code of this project is also available. They just give this out to all of us free of charge. Absolutely amazing. When I was a PhD student I was thinking whether real time ray tracing will be a possibility in my lifetime. And here we are approximately 10 years later and we get this. I can't believe it. What a time to be alive. Make sure to have a look at the paper in the video description. It is beautifully written. Also, I have a master level course on light transportation that is completely free for everyone, no strings attached. We write a full light simulation program from scratch there. Check it out. If you're looking for inexpensive cloud GPUs for AI, Lambda now offers the best prices in the world for GPU cloud compute. No commitments or negotiation required. Just sign up and launch an instance and hold on to your papers. Because with the Lambda GPU cloud you can now get on demand H1 instances and they are one of the first cloud providers to offer publicly available on demand H100 access. Did I mention they also offer persistent storage? So join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances. Workstations or servers make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today.
