TU Wien Rendering #31 - Unbiased, Consistent Algorithm Classes

Let's talk about, just briefly, about the PBRT architecture. PBRT is not exactly the renderer that we are going to use. We are going to use Lux Render. But Lux Render was built upon pbrt and therefore the basic structure remained completely intact. And this is a really good architecture that you would see that many of the rendering engines out there, global illumination rendering engines out there, use, most of them use the very same architecture. So we have a main sampler render task that asks the sampler to provide random samples. So the sampler, you can imagine as a random number generator, we need a lot of different random numbers, because the pixel that we are sampling, some techniques choose it deterministically, going from pixel to pixel. Some techniques take pixels randomly. I mean, which pixel we choose to be sampled is usually deterministic, but the displacement, because we would be sampling the pixels not only in the midpoint, like recursive ray tracing, but you would take completely random samples from nearby and use filtering to sum them up in a meaningful way. Now, this requires random numbers. They come from the sampler. You would also send outgoing rays in the hemisphere of different objects. You also need random numbers for this. So in this sample, these random numbers arrive. And this sample you would send to the camera, and the camera would give back to you a ray. So you tell the camera, please give me a ray that points to this pixel. And this camera would give you back a ray which starts from the camera starting point and points exactly there. Now all you need to do is give this ray to the integrator, and the integrator will tell you how much radiance is coming along this ray. And what you can do after that is write it to a film. And this is not necessarily trivial because, for instance, you could just simply write it to a PPM or a PNG file and be done with it. In contrast, what Lux Render does is it has a film class. And what you can do is that you can save different, for instance, different contributions in different buffers. So what you could do is, for instance, separate direct and indirect illumination into different films, different images, and then you can, in the end, sum them up. But maybe you could say that I don't need caustics on this image, and then you would just cut that image. So you can do tricky things if you have a correctly implemented film class. Okay, so Lux Render, just what I have been talking about. It's built upon PBRT and uses the very same architecture. This is how it looks. So it has graphical user interface, and you can also Manipulate different tone mapping algorithms in there, different denoising image denoising algorithms in there, and you can even manipulate light groups. This is another tricky thing with the film class. Basically what this means is that you save the contributions of different light sources into different films by films. You can imagine image files so every single light source has a different PNG file, if you will, and they are saved into there. And the final image would come up as a sum, a sum of these individual films. But you could say that one of the light sources is a bit too bright. I would like to tone it down. But if you want to tone it down, then you would have to re render your image because you change the physical properties of what's going on. Now you can do this if you have this light groups option because they are stored into individual buffers. So you can just dim one of these images and just add them up together and then you would have the effect of that light source a bit dimmer. You can for instance, completely turn off sunlight or television that you don't want to use in the scene. It sounded like a good idea, but it wasn't. You can just turn it off without re rendering the scene. You can operate all of these things through the Lux render gui. Now, before we go into algorithms, lets talk about algorithm classes. What kinds of algorithms are we interested in? First? What we are interested in is consistent algorithms. Consistent means that if I use an infinite number of Monte Carlo samples, then I would converge exactly to the right answer. I would get back the exact integral of the function. Intuitively, it says if I run this algorithm, sooner or later it will converge. It also is important to note that no one said anything about when this sooner or later happens. So if an algorithm is consistent, it doesn't mean that it is fast, it doesn't mean that it's slow. It can be anything, absolutely anything. It may be that there is an algorithm that's theoretically consistent. So after an infinite amount of samples you would get the right answer. But it really feels like infinity. So it may be that after two weeks you still don't get the correct image. There are algorithms like that, and theoretically that's consistent. That's fine, because you can prove that it's going to converge sooner or later. The more difficult class that many people seem to mess up is unbiased algorithms. Now what does it mean? If you just read the formula, then you can see that the expected error of the estimation is zero. And we have to note that this is completely independent of N. N is the number of samples that we have taken now, the expected error of the estimation is zero. It doesn't mean that the error is zero because it's independent of the number of samples. It doesn't mean that after one sample per pixel I get the right result. It says that the expected error is zero. I will give you many intuitions for this because it is very easy to misunderstand and misinterpret. Because in statistics there is a difference between expected value and variance. And this doesn't say anything about the variance. This only tells you about the expected values. So for instance, if you are a mathematician and think a bit about this, then you could say that if I have an unbiased algorithm and I have two noisy images, you render something on your machine, I render something on my machine, that's two noisy images. I could merge them together, I could average them because they are unbiased samples. It doesn't matter where they come from. I would add these samples together, average them, and I would get a better solution. We will see an example for that. My favorite intuition is that the algorithm has the very same chance of over and underestimating the integrand. So it means that if I would try to estimate the outcome of a dice roll, the expected value you can roll from 1 to 6 with equal probabilities, the expected value is 3.5. So this means that I would have the very same probability of saying 4 as I would have the probability for saying 3. So, so it's the very same chance to under and overestimate the integral. And I'll give you my other favorite intuition. This is what journalists tend to like the best. It means that there's no systematic error in the algorithm. The algorithm doesn't cut corners. And if there are errors in the image, then this can be only noise. And this noise comes because you don't have enough samples. And if you add more, you are guaranteed to get better. Now let's take another look at this really good intuition. So I can combine together two noisy images. So this means that I should be able to do network rendering without actually using a network, which sounds a bit mind boggling. I really like the parallel to this, which is a famous saying of Einstein from long ago where they talked about sending electromagnetic waves out and they talked about the telephone and people could not grasp the idea of a telephone. And he said that we would have a super, super long cat, the tail of the cat would be in Manhattan, and if you would just pull the tail of the cat in Manhattan, then the front of the cat would Be in New York. And if you pull the tail in Manhattan, then she would say, meow in New York. And he asked the people, is this understandable? Yes, this is understandable. Okay, perfect. We're almost there. Now imagine that there's no cat. And this is the exact same thing. So this is network rendering without an actual network. Well, okay, mathematical theories, okay? But let's actually. Let's give it a try. So what I did here is I rendered this interior scene, and this is how it looks like after two minutes, it's really noisy right now. What I did is I ran 10 of these rendering processes and saved the images 10 times. So I didn't run one rendering process for long. I ran many completely independent rendering processes for two, two minutes. And what I did is I merged the images together. What it means is that I averaged the images. I added them together and averaged them. Now, basically, this means that you could do this on completely independent computers that have never heard of each other. And now let's take a look. This is the noisy image that we had. And now let's merge 10 of these together. This is what we will get. Look closely. Look at that. Now one more time. This is the noisy after two minutes. And this is merging some of these noisy images together. So this is unbelievable that this actually works. So if you have unbiased algorithms, you can expect this kind of behavior. And you don't need to write sophisticated networking to use your path tracer, for instance, in a network, because you don't need the network at all. And this is really awesome. No, because if you don't add any kind of seed to your computations, then you are computing completely independent samples. And it doesn't matter if the sample is computed on the same machine or in a different machine. If you have some kind of determinism, then it may be possible that the same paths are computed by multiple machine, and that's indeed wasted time. But otherwise it works just fine. Now let's practice a bit instead. There is a question. Yes. Just how big is the difference between one picture rendered 20 minutes and 10 pictures rendered two minutes each and then combined? Nothing in terms of samples? Nothing. The only difference is that you actually need to fire up that scene or on multiple machines. So if there is like 10 gigabytes of textures, then it takes longer to load it up on multiple machines and maybe transfer the data together. But if you think only in terms of sample, it doesn't matter where it comes from. Okay, let's practice a bit. We have different techniques, and this is how the error is evolving in time. Now, the intuition of consistent means that the error tends to zero over time. So if I render for long enough, then the error is going to be zero. Is this black one a consistent algorithm? No, because it converges here to the dashed line and not to zero. Now, what about the other two guys? Are they consistent or not? Yes. Okay, they are consistent. Why? Because it converged to the baseline. Okay, so the error seems to converge to zero. Okay, now what about these techniques? Are they biased or unbiased? Which is which? What about this one? This is the darker gray. Is this biased or unbiased? Now, if we have this intuition that if we render for more, the image is guaranteed to get better, or at least not worse, then this dark gray is definitely not unbiased. Because it is possible that I'm rendering for 10 minutes, that's this point, for instance. And I say, okay, I almost have a good enough image and I render for another five image and expect it to be better. And then I get this, maybe a completely garbled up image full of artifacts and errors. And that is entirely possible with biased algorithms. No one said that it's likely, but it is possible. So you cannot really predict how the error would evolve in time. And if you take a look at the other two lines, you can see that they are unbiased algorithms. So as you render for longer, you are guaranteed to get a better image.