NVIDIA Renders Millions of Light Sources! 🔅

Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to see how Nvidia's light transport algorithm can render this virtual amusement park that contains no less than 3.4 million light sources. Interactively. Yes. Millions of light sources. How is this even possible? Well, to understand what is going on here, we. We need to talk about two concepts if we wish to create a fully photorealistic scene in computer graphics. We reach out to a light transport simulation algorithm and then this happens. Oh yes, concept number one, noise. This is not photorealistic at all. Not yet anyway. Why is that? Well, during this process, we have to shoot millions and millions of light rays into the scene and to estimate how much light is bouncing around. And before we have simulated enough rays, the inaccuracies in our estimations show up as noise in these images. This clears up over time, but it may take from minutes to days for this to happen, even for a smaller scene. For instance, this one took us three full weeks to finish. Weeks. Ouch. And in a moment you will see that Nvidia's technique can render even bigger scenes than this. Not in weeks, but in milliseconds. Yes, really, I am not kidding. And for now, concept number two, biased and unbiased. There are different classes of light transport algorithms. Here you see an unbiased technique that doesn't cut any corners. And now look. Oh yes, this is a biased technique. And as you see, this one can give us a much cleaner image in the same amount of time. But the price to be paid for this is that some corners have been cut in the process. And with that. Now hold on to your papers and have a look at Nvidia's technique. Oh yes, this was the scene with the 3.4 million light sources. And this method can really render not just an image, but an animation of it interactively. That is absolutely incredible. And it has even more to offer. Look, the light sources can even be dynamic. Or in other words, they can move around as they please and the algorithm still doesn't break a sweat. Oh my goodness. That is absolutely amazing. Now let's see how it compares to a previous technique. Oh boy. See the noise levels here. This is so noisy we don't even know what the scene should exactly look like. But now when we have a look at this technique. Oh yes. Now we know for sure that, that there are some glossy reflections going on here. Even the unbiased version of the new technique is significantly better in this. However, look, if we are willing to let the simulator cut some Corners. This is the biased version. And oh my goodness, those difficult glossy reflections are so much cleaner. I love it. This amusement park scene contains a total of over 20 million triangles. And once again, the biased version of the new technique makes it truly a sight to behold. And this does not take from minutes to days to compute. Each of these images were produced in a matter of milliseconds. Wow. But it gets better. How? Well, the more detailed comparisons in the paper reveal that this method is 10 to 100 times faster than than previous techniques. And it also maps really well onto our graphics cards. Ok, what is behind all this wizardry? How is this even possible? Well, the magic behind all this is a smarter allocation of these ray samples that we have to shoot into the scene. For instance, this technique does not forget what we did just a moment ago when we moved the camera a little and advance to the next image. Thus, lots of information that is otherwise thrown away can now be reused as we advance the animation. Now note that there are so many papers out there on how to allocate these rays properly. This field is so mature. It truly is a challenge to create something that is just a few percentage points better than previous techniques. It is very hard to make even the tiniest difference. And to be able to create something that is 10 to 100 times better in this environment, that is insanity. And this proper ray allocation has one more advantage. What is that? Well, have a look at this. Imagine that you are a good painter and you are given this image. Now your task is to finish it. Do you know what this depicts? Hmm, maybe. But knowing all the details of this image is out of question. Now look, we don't have to live with these noisy images. We have denoising algorithms tailored for light simulations. This one does some serious legwork with this noisy input. But even this one cannot possibly know exactly what is going on because there is so much information missing from the noisy input. And now if you have been holding onto your paper so far, squeeze that paper because look, this technique can produce this image in the same amount of time. Now we're talking. Now let's give it to the denoising algorithm. And yes, we get a much sharper, much more detailed output. Actually, let's compare it to the clean reference image. Oh yes, this is so much closer. And the fact that just one more paper down the line, we could go from this to this absolutely blows my mind. So cool. But wait, we noted that the biased version of the technique cuts some corners. What is the price to be paid for this? What are we losing here, well, some energy. What does that mean? It means that the image it produces is typically a little darker. There are other differences, but usually they are relatively minor, making this an excellent trade off. Adding it all together, this will be an excellent tool in democratizing light transport algorithms and putting it into the hands of everyone. And to think that years ago we had this and now we can get this running on a commodity graphics card. I am truly out of words. The pace of progress in computer graphics research is nothing short of amazing. What a time to be alive. Now, if you also got excited by this, look here. This is not an implementation of this paper, but the first author also has a nice little web app where you can create interesting 2D light transport situations with lots of caustics and the results are absolutely beautiful. The link is available in the video Description Check it out. This episode has been supported by Lambda GPU Cloud. If you're looking for inexpensive cloud GPUs for AI, check out Lambda GPU Cloud. Get this They've recently launched an Nvidia RTX A6000 with 48 gigabytes of memory and hold on to your papers because Lambda GPU Cloud can cost less than half of AWS and Azure. Join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today. Our thanks to Lambda for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.