Disney trained a neural network to render clouds in seconds instead of hours. The hybrid approach uses AI to predict light scattering while traditional rendering handles the rest, trained on 75 procedural and artist-made clouds. Results stay visually close to ground truth while letting artists tweak parameters in real time.
75
Up to 30 hours
November 9, 2017
Károly Zsolnai-Fehér
Dear fellow scholars, this is Two Minute Papers Withehir. This is a fully in house Disney paper on how to teach a neural network to capture the appearance of clouds. This topic is one of my absolute favorites because it is in the intersection of the two topics I love most, computer graphics and machine learning. Hell yeah. Generally we use light simulation programs to render these clouds, and the difficult part of this is that we have to perform something that is called volumetric path tracing. This is a technique where we have to simulate rays of light that do not necessarily bounce off of the surface of objects, but may penetrate their surfaces and undergo many scattering events. Understandably, in the case of clouds, capturing volumetric scattering properly is a key element in modeling their physical appearance. However, we have to simulate millions and millions of light paths with potentially hundreds of scattering events, which is a computationally demanding task even in the age of rapidly improving hardware. As you can see here, the more we bump up the number of possible simulated scattering events, the closer we get to reality, but the longer it takes to render an image. In the case of bright clouds here, rendering an image like this can take up to 30 hours. In this work, a nice hybrid approach is proposed where a neural network learns the concept of in scattered radiance and predicts it rapidly. So this part we don't have to compute ourselves. It is a hybrid because some parts of the renderer are still using the traditional algorithms. The dataset used for training the neural network contains 75 different clouds, some of which are procedurally generated by a computer, and some are drawn by artists to expose the learning algorithm to a large variety of cases. As a result, these images can be rendered in a matter of seconds to minutes. Normally, this would take many, many hours on a powerful computer. Here's another result with traditional path tracing, And now the same with deep scattering. Yep, that's how long it takes. The scattering parameters can also be interactively edited without us having to wait for hours to see if the new settings are better than the previous ones. Dialing in the perfect results typically takes an extremely lengthy trial and error phase, which now can be done almost instantaneously. The technique also supports a variety of different scattering models. As with all results, they have to be compared to the ground truth renderings, and as you can see here, they seem mostly indistinguishable from reality. It is also temporally stable, so animation rendering can take place flicker free, as is demonstrated here in the video. I think this work is also a great testament to show how these incredible learning algorithms can accelerate progress in practically all fields. Of science. And given that this work was done by Disney, I am pretty sure we can expect tons of photorealistic clouds in their upcoming movies in the near future. There are tons of more details discussed in the paper, which is remarkably well produced. Make sure to have a look. The link is in the video description. This is a proper, proper paper. You don't want to miss out on this one. Also, if you enjoy this episode and you feel that the series provides you value in the form of enjoyment or learning, please consider supporting us on Patreon. You can pick up cool perks there, like deciding the order of the next few episodes, and you also help us make better videos in the future. Details are available in the description. Thanks for watching and for your generous support and I'll see you next time. Sam.
