A Two Minute Papers video by Dr. Károly Zsolnai-Fehér reviewing an NVIDIA paper that introduces an interactive light transport algorithm capable of rendering complex volumetric effects (smoke plumes, explosions, volumetric bunnies) with dramatically fewer samples per pixel. The video demonstrates comparisons to previous techniques (e.g., 12 spp baseline), shows how reuse of temporal and spatial information plus denoising produces near-photorealistic results from as little as one sample per pixel, and notes practical tech transfer and availability of a free master-level course on light transport. The video includes a sponsorship message from Weights & Biases.
Chapters
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to see how Nvidia's incredible light transport algorithm can render these notoriously difficult photorealistic smoke plumes, volumetric bunnies, and even explosions interactively. We are going to talk about this amazing paper in a moment. And given that Nvidia can already render the this absolutely beautiful marbles demo in real time, not only with real time light transport, but also with our other favorite around here, real time physics, this is just to say that tech transfer is happening here. The papers are real. The papers that you see here really make it to real applications that everyone can use at home. Please note that they won't say exactly what tech is under the hood here, but looking at their best light transport papers might be a good starting point of what is to come in the future. Now back to the intro. What you are seeing here should not be possible at all. Why is that? Well, when we use a light transport simulation algorithm to create such an image, we get a photorealistic image. But as you see, not immediately, not even close. It is a miracle of science that with this we can shoot millions and millions of light rays into the scene to estimate how much light is bouncing around. And initially the inaccuracies in our estimations show up as noise in these images. As we shoot more and more rays, this clears up over time. However, there are two. One, this can take hours to finish. And two, in the case of volumetric light transport, these light rays can bounce, scatter and get absorbed inside a dense medium. In that case, it takes much, much longer. Not only hours, sometimes even days. Oh my goodness. So how did Nvidia pull this off? Well, they built on a previous technique of theirs that is capable of rendering 3.4 million light sources and not just for one image, but it can even handle animation. Just look at all this beautiful footage. Now it would be so great to have a variant of this that works on volumetric effects such as explosions and smoke plumes. But of course, anyone who has tried it before says that there is no chance that those could run interactively. No chance at all. Well, scientists at Nvidia beg to differ. Check this out. This is a previous technique and it says 12 spp. That means 12 samples per pixel, which is more or less simulating 12 light rays for each pixel of these images. That is not a lot of information. So let's have a closer look at what this previous method can do with that. Well, there is still tons of noise and it flickers a lot when we rotate the camera around. Well, I would say that there is not much hope here. We would need hundreds or maybe even thousands of samples per pixel to get a clean image, not 12. And look, it gets even worse. What? The new technique runs not thousands, not hundreds, and not even 12 samples per pixel in the same amount of time. But really, can this really be one sample per pixel? Can you do anything meaningful with that? Well, hold on to your papers and check this out. Wow, look at that. It can do so much better with just one ray per pixel than the previous technique can do with 12. That is absolutely incredible. I really don't know what to say. And it works not just on this example, but on many others as well. This is so far ahead of the previous technique that it seems like science fiction. I absolutely love it. However, yes, I can hear you asking, Karoj, but these are still noisy. Why get so excited for all these incomplete results? Well, do you remember what we did with the previous Nvidia light transport paper? We took these noisy inputs and plugged them into a denoising technique that is specifically designed for light transport algorithms. And it tries to guess what is behind the noise. And as you see, these can help a ton. But you know, can they help so much that a still noisy input, a meager one sample per pixel, can become usable? Well, let's have a look. Oh my goodness. Look at that. The result is clearly not perfect. One light ray per pixel can hardly give us a perfect image. But after denoising, this is unreal. We are almost there right away. It has so much less flickering than the previous technique with much more samples. And we are experienced fellow scholars over here. So let's also check the amount of detail in the image and. Whoa. There is no contest here. This technique also pulls off all this wizardry by trying to reuse information that is otherwise typically thrown away. For instance, with no reuse, we get this baseline result. And if we reuse information from a previous frame in the animation, we get this. That is significantly better than the baseline. If we reuse previous rays spatially, that is also an improvement. So our question, are these different kinds of improvements? Well, let's add them together and. Oh yes, now this is what we are here for. Look at how much more information there is in this image. So now even these amazing volumetric scenes can be rendered interactively. I am out of words. What a time to be alive. And if you feel inspired by these results, I have a free master level course on light transport where we write a full light simulation program from scratch and learn about physics, the world around us and more. If you watch it you will see the world differently. That is free education for everyone. That's what I want. So the course is available free of charge for everyone. No strings attached. Check it out in the video description.
This video has been supported by weights and biases. Being a machine learning researcher means doing tons of experiments and of course creating tons of data. But I am not looking for data, I am looking for insights and weights and biases helps with exactly that. They have tools for experiment tracking, dataset and model versioning and even hyperparameter optimization. No wonder this is the experiment tracking tool choice of OpenAI, Toyota Research, Samsung and many more prestigious labs. Make sure to use the link WNB me paperintro or just click the link in the video description and try this 10 minute example of weights and biases today to experience the wonderful feeling of training a neural network and being in control of your experiments. After you try it, you won't want to go back.
Thanks for watching and for your generous support and I'll see you next time.
