TU Wien Rendering #34 - SDS Transport, Photon Mapping

Welcome to the last lecture where we are going to be talking about the state of the art in global illumination. Now if you remember this scene, you hopefully also remembered that we were able to render caustics insanely fast with Metropolis light transport. And the rest of the scene is also cleaning up pretty quickly. But what about this? This was a not so difficult looking scene and you can see that the Kalaman style Metropolis is doing pretty awful here. I'm not even sure if it would converge if we would wait for a very long time. What the problem is here is called SDS or specular diffuse specular transport. Let's talk about this for a bit. So imagine that I have a light path that starts out from a light source, hits this glass object, that is a specular bounce, and and now it has another specular bounce after the refraction. Then we hit a diffuse object, then the mirror, and then we hit the eye. Let's put up there heck Bert's notation and let's rip out the middle part of the light path. Now this says sds. What is the intuition of this? It is reflected caustics because 1s and 1d gives you caustics like we discussed before. And then if you have another specular bounce, then this says that I am seeing the caustics through the mirror. The intuition for SDS light paths is reflected caustics. So what is exactly the problem here? Imagine that we start out from a diffuse surface, we sample the BRDF and therefore we arrive to this other specular surface. And off of this specular surface we are supposed to hit the pinhole camera. Also note that this diffuse point on the surface of the was chosen by a specular interaction before and it was chosen explicitly. Now you can hopefully see that this means that depending on the material models, if we have perfect specular interreflections, then sampling such a light path is impossible. And this is a problem that you can encounter very often because imagine that if you have a light source that is covered by a specular surface, so for instance a glass light bulb, then even if you have a regular DS path, so one diffuse and one specular bounce, then you add one more S, because all the light that is exiting the light source is going to hit the COVID the glass part of the light bulb. And therefore every DS is going to be sds another image for the intuition and better understanding of what is exactly going on. You can also imagine that you are starting out a light path from the light source and from the eye and you have the SD from the light Source and you have the SS from the eye. Now what you would like to do is you would like to connect this diffuse to the specular vertex. Now this is impossible. The specular vertex would have to be chosen by the diffuse. And the diffuse BRDF would be choosing one outgoing direction on the hemisphere uniformly sampled. And it is only one possible direction that we would be happy with. The probability of sampling this one possible direction is exactly the same as the probability of sampling one point. And that is zero. So this is the SDS problem. And we are going to look at biased algorithms that try to address this problem. So this looks like SDS to me because we hit the glass cube, that's a specular bounce. Then we hit the donut inside and then we hit the glass cube again. So this is sds. This is why it is so difficult to sample with Metropolis light transport photo mapping. The key idea is that we don't want to evaluate all possible light paths. What we would like to do is sending photons out of light sources. And we are going to store all of these photons in the map. And when we are computing actual light paths, we are going to rely on interpolation. We are going to use this knowledge that we have in the photon map some visualization images to get an idea how it exactly looks. Let's take a look at the first bounce. This is an image with only the very first bounces in the path tracer. This is the direct lightmap. Now let's take a look at the indirect lightmap. This is the second and higher order bounces. This is basically indirect illumination color bleeding. And you can see that this is actually low frequency information. You can see that the colors don't really seem to change so quickly if we have indirect illumination. This is a mostly low frequency signal which lends itself to the idea of interpolation. This is an example on how to use all this information in the photo map. So I would be interested in the incident radiance at the red dot. And what I can do is use the information from the nearby photo. And I would average all this information to get an estimation for the red dot. And you can see that the brighter regions of the image seem to have more photons in the photon map. Why is that? Well, it's simple. It's because we are shooting photons out of the light sources. And these are the regions that are very visible from the light source. Let's take a look at some results. You can see a difficult scene rendered with path tracing. Bi directional path tracing. Is much, much better, but you can still see the Firefly noise. We also have some results with Metropolis Light. Transport also doesn't help a lot with SDS transport and photo mapping. You can see that all this high frequency noise is gone, but the result is slightly more blurry than it should be. Because of the interpolation we are averaging samples and therefore this smoothing behavior is inherent to that interpolation. What are the upsides of photon mapping? Well, caustics, indirect illumination, they converge really quickly. Caustics, why? Because you have a lot of samples. Because you see it from the light source. Indirect illumination. Why? Because it's mostly a low frequency signal that you can interpolate very easily. Note that it also helps with the SDS problem. Because of the interpolation you don't really get high frequency noise for most cases. However, don't forget that you may need to shoot and store a lot of photons depending on how complex your scene is. And this can be very computationally intensive and also memory intensive. And interpolation can cause artifacts to appear. And this actually happens quite often for more complex scenes because you are looking up photons in the photo map nearby. If these nearby photons are on the same object that you would like to query, then these are usually depending on textures and many other things. This is usually usable information. But you are looking up nearby photons and you may see many examples in the room you're sitting in where you have discontinuities nearby. So there may be a wall that is one color and there may be a wall nearby at an intersection that is a different color. It may be that during the interpolation you use samples from the other wall because it is nearby and it doesn't really take this property into consideration. So so therefore artifacts may appear. What about this algorithm? Well, we are cutting corners. We are using interpolation. We are not computing all the possible light paths there are. Therefore this algorithm has to be biased. What about the consistency? Well, it is consistent provided that you have an infinite number of photons in the photon map and therefore you always get perfect information. However, this is only of theoretical value because obviously having an infinite number of photons may make sense in a mathematical way, but in a practical implementation you cannot even shoot and you cannot even store an infinite number of photons. Some Literature this is the place where you can look up the original photon mapping paper from Henrik van Jensen. Some delightful news. This is an image I shot at the Urographic symposium of rendering EGSR last year. If you take a look at these people. You can see, for instance, Wojta Kjaros, lead of the rendering group at Disney Research. And he and the EGSR organizer crew gave out the Test of Time award to Henrik Van Jensen because of the photo mapping algorithm. It's been around for a while and it had seen a lot of use and it's a fantastic piece of work, and he got recognized for that.