A neural network learns to relight photographs by studying images taken with a light source in different positions. The system predicts what a photo looks like when the light moves to places it never saw during training, handling tricky effects like shadows and reflections surprisingly well.
Dear fellow scholars, this is two minute papers with. In this work, we place a small light source to a chosen point in a scene and record a photograph of how things look like with the given placement. Then we place the light source to a new position and record an image again. We repeat this process several times. Then after we have done that, we have the question, what would the photograph look like if I put the light source to places I haven't seen yet? This process we call image relighting. This work uses neural networks to do relighting by learning how different light source placements behave. If you haven't heard about neural networks before, make sure to check out our previous episodes on the topic. I have put links for you in the description box after the training. This technique guesses how completely unknown light source setups would look like in reality. We give the algorithm a light source position we haven't seen yet and it will generate us a photograph of how it would look like in reality. And the first question is, it's okay, but how well does it do the job? I am not sure if you are going to believe this one as you will be witnessing some magnificent results. On the left you will see real photographs and on the right, reconstructions that are basically the guesses of the algorithm. Note that it doesn't know how the photograph would look like. It has to generate new photographs based on the knowledge that it has from seeing other photos and it's completely indistinguishable from reality. This is especially difficult in the presence of the so called high frequency lighting effects. The high frequency part means that if we change the light source just a bit, there may be very large changes in the output image. Such a thing can happen when a light source is moved very slightly but is suddenly hidden behind an object. Therefore, our photograph changes drastically. The proposed technique uses ensembles. It means that multiple neural networks are trained and their guesses are averaged to get better results. What do you do if you go to the doctor and he says you have a very severe and very unlikely condition? Well, you go and ask multiple doctors and see if they say the same thing. It is reasonable to expect that the more doctors you ask, the clearer you will see. And this is exactly what the algorithm does. Now look at this. On the left side there's a real photo. And on the right, the guess of the algorithm after training. Can you believe it? You can barely see the difference. And this is a failure case. The success story scenarios for many techniques are not as good as the failure cases here. These results are absolutely stunning. The algorithm can also deal with multiple light sources of different colors. As you can see, machine learning techniques such as deep neural networks have opened so many doors in research lately. We are starting to solve problems that everyone agreed were absolutely impossible before. We are currently over 2,000 subscribers. Our club of scholars is growing at a really rapid pace. Please share the series so we can reach people that don't know about us yet. Let's draw them in and show them how cool research really is. Thanks for watching and I'll see you next time.
