A short YouTube monologue by Dr. Károly Zsolnai-Fehér surveying ~30 years of light-transport research. The speaker compares three core rendering approaches—path tracing (baseline), Metropolis Light Transport (improved), and manifold exploration (state-of-the-art)—and highlights a recent paper that reimagines manifold exploration for easier implementation. The video also promotes a free master-level course on light simulation and includes a job posting and sponsor mention.
Chapters
Dear fellow scholars, this is two minute papers with Dr. Zsolnai. I have been yearning for a light transport paper and goodness, was I ecstatic when reading this one. And by the end of the video, I hope you will be too. And now we only have to go through just about 30 years of light transport research. As some of you know, if we immerse ourselves into the art of light transport simulations, we can use our computer to simulate millions and millions of light rays and and calculate how they get absorbed or scattered off of our objects in a virtual scene. Initially, we start out with a really noisy image, and as we add more rays, the image gets clearer and clearer over time. The time it takes for these images to clean up depends on the complexity of the geometry and our material models, but it typically takes a while. This microplanet scene mostly contains vegetation, which are matte objects. These we also refer to as diffuse materials. These typically converge very quickly. As you see, we get meaningful progress on the entirety of the image within the first two minutes of the rendering process. And remember, in light transport simulations, noise is public enemy number one. This used a technique called path tracing. Let's refer to it as the OK technique. And now let's try to use path tracing this OK technique to render this torus in a glass enclosure. This is the first two minutes of the rendering process and it does not look anything like the previous scene. The previous one was looking pretty smooth after just two minutes, whereas here, you see, this is indeed looking very grim. We have lots of these fireflies which will take us up to a few days of computation time to clean up, even if we have a modern powerful machine. So why did this happen? The reason for this is that there are tricky cases for specular light transportation that take many, many millions, if not billions of light rays to compute properly. Specular here means mirror. Like materials, those can get tricky. And this torus that has been enclosed in there is also not doing too well. So this was path tracing, the OK technique. Now let's try a better technique called Metropolis Light Transport. This method is the result of a decade of research and is much better in dealing with difficult scenes. This particular variant is a proud Hungarian algorithm by a scientist called Czobok Kellermann and his colleagues at the Technical University of Budapest. For instance, here is a snippet of our earlier paper on a similarly challenging scene. This is how the OK path tracer did and in the same amount of time. This is what Metropolis Light Transport, the better technique could do. This was a lot more efficient. So let's See how it does with the torus. Now that's unexpected. This is indeed a notoriously difficult scene to render, even for Metropolis light transport. The better technique. As you see, the reflected light patterns that we also refer to as caustics on the floor are much cleaner. But the torus is still not giving up. Let's jump another 15 years of light transport research and use a technique that goes by the name manifold exploration. Lets call this the best technique. Wow, look at how beautifully it improves the image. It is not only much cleaner but also converges much more gracefully. It doesn't go from a noisy image to a slightly less noisy image, but almost immediately gives us a solid baseline and new cleaned up paths also appear over time. This technique is from 2012 and it truly is mind boggling how good it is. This technique is so difficult to understand and implement that to the best of my knowledge, the number of people who can and have implemented it properly is exactly one. And that one person is Vencel Jakob, one of the best minds in the game. And believe it or not, he wrote this method as a PhD student in light transport research and today as a professor at EPFL Switzerland. He and his colleagues set out to create a technique that is as good as manifold exploration. The best technique, but is much simpler. Well, good luck with that I thought when skimming through the paper, let's see how it did. For instance, we have some caustics at the bottom of a pool of water. As expected, lots of firefly noise with the OK path tracer. And now hold on to your papers and here is the new technique. Just look at that. It can do so much better in the same amount of time. Let's also have a look at this scene with lots and lots of specular microgeometry or in other words glints. This is also a nightmare to render. With the OK path tracer we have lots of flickering from one frame to the next. And here you you see the result with the new technique. Perfect. So it is indeed possible to take the best technique, manifold exploration, and reimagine it in a way that ordinary humans can also implement. Huge congratulations to the authors of this work that I think is a crown achievement in light transport research. And that's why I was ecstatic when I first read through this incredible paper. Make sure to have a look at the paper and you will see how they borrowed a nice little trick from a recent work in nuclear physics physics to tackle this problem. The presentation of the paper and the talk video with the details is also brilliant and I urge you to have a look at it in the video description. This whole thing got me so excited I was barely able to fall asleep for several days now. What a time to be alive. Now, while we look through some more results from the paper. If you feel a little stranded at home and are thinking that this light transport thing is pretty cool, I held a master level course on this topic at the Technical University of Vienna. Since I was always teaching it to a handful of motivated students, I thought that the teachings shouldn't only be available for a privileged few who can afford a college education, but that teachings should be available for everyone. Free education for everyone. That's what I want. So the course is available free of charge for everyone. No strings attached. So make sure to click in the video description to get started. We write a full light simulation program from scratch and learn about physics, the world around us and more. Also note that my former PhD advisor Michael Wimmer is looking to hire a postdoctoral researcher in this area, which is an amazing opportunity to push this field forward. The link is available in the video Description this episode has been supported by Lambda GPU Cloud. If you're looking for inexpensive cloud GPUs for AI, check out Lamda GPU Cloud. They've recently launched Quadro RTX 6000, RTX 8000 and V100 instances and hold on to your papers because Lambda GPU Cloud can cost less than half of AWS and Azure plus they are the only cloud service with 48gb RTX 8000. Join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances with workstations or servers. Make sure to go to lambdalabs.com papers to sign up for one of their amazing GPU instances today. Our thanks to Lambda for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.
