The Story of Light! ☀️

Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Whenever we look at these amazing research papers on physical simulations, it is always a joy seeing people discussing them in the comments section. However, one thing that caught my attention is that some people comment about how things look and not on how things move in these papers, which is fair enough. And to this end, I will devote this episode to talk about a few amazing techniques in light transport simulations. But first things first. When talking about physics simulations, we are talking about a technique that computes how things move. Then we typically run a light simulation program that computes how things look. The two are completely independent, which means that it is possible that the physical behavior of bread breaking here is correct, but the bread itself does not look perfectly realistic. The second part depends on the quality of the light simulation and and the materials used there. We can create such an image by simulating the path of millions and millions of light rays. And initially this image will look noisy, and as we add more and more rays, this image will slowly clean up over time. If we don't have a well optimized program, this can take from hours to days to compute. We can speed up this process by carefully choosing where to shoot these rays. And this is a technique that is called importance sampling. But then, around 1993, an amazing paper appeared by the name bidirectional path tracing that proposed that we don't just start building light paths from one direction, but two instead. One from the camera and one from the light source, and then connect them. This significantly improved the efficiency of these light simulations. However, it opened up a new can of worms. There are many different ways of connecting these paths, which leads to mathematical difficulties. For instance, we have to specify the probability of a light path forming. But what do we do if there are multiple ways of producing this light path? There will be multiple probabilities. What do we do with all this stuff? To address this, Eric Veitch described a magical algorithm in his thesis, and thus multiple important sampling was born. I can say without exaggeration that this is one of the most powerful techniques in all photorealistic rendering research. What multiple important sampling, or from now on MIS in short does, is combine these multiple sampling techniques in a way that accentuates the strength of each of them. For instance, you can see the image created by one sampling technique here, and the image from a different one here. Both of them are quite noisy, but if we combine them with mis, we get this instead in the same amount of time, a much smoother, less noisy image. In many cases, this can truly bring down the computation times from several hours to several minutes. Absolute witchcraft. Later, even more advanced techniques appeared to accelerate the speed of these light simulation programs. For instance, it is now not only possible to compute light transport between points in space, but between a point and a beam instead. You see the evolution of an image using this photon beam based technique. This way we can get rid of the point based noise and get a much, much more appealing rendering process. The lead author of this beam paper is Wojta Kjaros, who three years later ended up being the head of the rendering group and at the amazing Disney Research Lab. Around that time he also hired me to work with him at Disney on a project I can't talk about, which was an incredible and life changing experience. And I will be forever grateful for his kindness. By the way, he is now a professor at the Dartmouth University and just keeps pumping out one killer paper after another. So, as you might have guessed, if it is possible to compute light transport between two points, a point and a beam, later it became possible to do this between two beams. None of these are for the faint of the heart, but it works really well. But there is a huge problem. These techniques work with different dimensionalities, or in other words, they estimate the final result so differently that they cannot be combined with multiple importance sampling. That is indeed a problem because all of these have completely different strengths and weaknesses. And now hold on to your papers because we have finally arrived to the main paper of this episode. It bears the name upbp, which stands for Unifying Points, beams and paths. And it formulates multiple important sampling between all of these different kinds of light transport simulations. Basically what we can do with this is throw every advanced simulation program we can think of together and out comes a super powerful version of them that combines all their strengths and nullifies nearly all of their weaknesses. It is absolutely unreal. Here you see four completely different algorithms running and as you can see, they are noisy and smooth at very different places. They are good at computing different kinds of light transport. And now hold on to your papers because this final result with the UPBP technique is this. Wow. Light transport on steroids. While we look at some more results, I will note that in my opinion, this is one of the best papers ever written in light transport research. The crazy thing is that I hardly ever hear anybody talk about it, if any paper would deserve a bit more attention. So I hope this video will help with that. And I would like to dedicate this video to Yaroslav Krvanek, the first author of this absolutely amazing paper who has tragically passed away a few months ago. In my memories I think of him as the true king of multiple important sampling and I hope that now you do too. Note that MIS is not limited to light transport algorithms. It is a general concept that can be used together with a mathematical technique called Monte Carlo Integration which is used pretty much everywhere from finding out what an electromagnetic field looks like to to financial modeling and much, much more. If you have anything to do with Monte Carlo integration, please read Eric Vici's thesis and this paper and if you feel that it is a good fit, try to incorporate multiple important sampling into your system. You'll be glad you did. Also, we have recorded my lectures of a master level course on light transport simulations at the Technical University of Vienna. In this course we write such a light simulation program from scratch and it is available free of charge for everyone, no strings attached, so make sure to click the link in the video description to get started. Additionally, I have implemented a small one dimensional example of mis. If you wish to pick it up and try it, that's also available in the video description. While talking about the Technical University of Vienna, we are hiring for a PhD and a postdoc position. The call here about lighting simulation for architectural design is advised by my PhD advisor Michael Wimmer, who I highly recommend apply now if you feel qualified. The link is in the video description. Thanks for watching and for your generous support and I'll see you next time.