Transcript of a Two Minute Papers episode (Dr. Zsolnai) summarizing recent Nvidia research that advances light transport / ray-tracing toward real-time performance. The video explains the core challenge of noise from Monte Carlo sampling, techniques for reusing samples across frames and denoising, reports results achieving roughly 12 ms per frame on complex scenes (Paris Opera House, large amusement-park scene) with some remaining artifacts (slightly blurry glossy reflections), and notes practical demos and denoising tools from Nvidia.
Chapters
Dear fellow scholars, this is two minute papers with Dr. Zsolnai. I can't believe that I am saying this, but today we are going to see how Nvidia is getting closer and closer to solving an almost impossible problem. And that is writing real time light transport simulations. What is that and why is it nearly impossible? Well, if we wish to create a truly photorealistic scene in computer graphics, we usually reach out to a light transport simulation algorithm and then this happens. Oh yes, concept number one, noise. This is not photorealistic at all. Not yet anyway. Why is that? Well, during this process we have to shoot millions and millions of light rays in into the scene to estimate how much light is bouncing around. And before we have simulated enough rays, the inaccuracies in our estimations show up as noise in these images. This clears up over time, but it may take from minutes to days for this to happen, even for a smaller scene. For instance, this one took us three full weeks to finish. Three weeks. Ouch. Now earlier we talked about this technique which could take complex geometry and 3.4 million light sources and it could really render not just an image, but an animation of it interactively. But how? Well, the magic behind all this is a smarter allocation of these ray samples that we have to shoot into the scene. For instance, this technique does not forget what we did just a moment ago when we move the camera a little and advance to the next image. Thus lots of information that is otherwise thrown away can now be reused as we advance the animation. And note that even then, these smooth, beautiful images are not what we get directly. If we look under the hood, we see that the raw result we get something like this. Oh yes, still a noisy image. But wait, don't despair. We don't have to live with these noisy images. We have denoising algorithms tailored for light simulations. This one does some serious legwork with this noisy input. And in a follow up paper, the they also went on to tackle these notoriously difficult photorealistic smoke plumes, volumetric bunnies and even explosions. Interactively. These results were once again noise filtered to nearly perfection. Not to perfection, but a step closer to it than before. Now note that I use the word interactively twice here. I did not say real time. And that is not by mistake. These techniques are absolutely fantastic, one of the bigger leaps in light transport research. But they still cost a touch more than what production systems can shoulder. They are not quite real time yet. So what did they do? Did they stop there? Well, of course not. They rolled up the sleeves and continued. And now I hope you know what's coming. Oh yes, have a look at this newer paper they have in this area. This is the result on the Paris Opera House scene which is quite detailed. There is a ton going on here and you are all experienced fellow scholars now. So when you see them flicking between the raw noisy and the denoised results, you know exactly what is going on. And hold on to your papers because all this takes about 12 milliseconds per frame. Yes, yes, yes. My goodness, that is finally in the real time domain. And then some. What a time to be alive. Ok, so where's the catch? Our keen eyes see that this is a static scene. It probably can deal with dynamic movements and rapid changes in lighting. Can it? Well, let's have a look. Wow. I cannot believe my eyes. Dynamic movement check mark. And here. This is as much changing in the lighting as anyone would ever want. And it can do this too. I absolutely love it. And remember the amusement park scene from the previous paper? The one with 23 million triangles for the geometry and over 3 million light sources. Well, here are the raw results. And after denoising, this looks super clean. Wow. So how long do we have to wait for this? This can't be real time, right? Well all this takes is about 12 milliseconds per frame. Again and this is where I fell off the chair when reading this paper. Of course not even this technique is perfect. The glossy reflections are a little blurry at places and artifacts in the lighting can still appear. But if this is not a quantum leap in light transport research, I don't know what is. Plus, if you wish to see some properly detailed comparisons against previous techniques, make.
Sure to have a look at the paper.
And if you have been holding on to your papers so far, now squeeze that paper because everything that you see in this paper was done by two people. Huge congratulations Chris and Alexey. And if you are wondering if we ever get to use these techniques, don't forget Nvidia's marbles demo is already out.
There for all of us to use.
And it gets better. For instance, not many know that they already have a denoising technique that is online and ready to use for all of us. This one is a professional grade tool right there. This is really incredible. They have so many tools out there.
For us to use.
I check what Nvidia is up to daily and I still quite often get surprised about how much they have going on. And now finally, real time light transport in our lifetimes. Oh yes, this paper is history in the making.
This episode has been supported by Cohere AI. Cohere builds large language models and makes them available through an API so businesses can add advanced language understanding to their system or app quickly. With just one line of code, you can use your own data, whether it's text from customer service requests, legal contracts, or social media posts, to create your own custom models to understand text or even generate it. For instance, it can be used to automatically determine whether your messages are about your business hours, returns, or shipping. Or it can be used to generate a list of possible sentences you can use for your product descriptions. Make sure to go to Cohere AI papers or click the link in the video description and give it a try Today. It's super easy to use.
Thanks for watching and for your generous support, and I'll see you next time.
