A new non-neural method applies artistic style to entire videos using just a few hand-stylized keyframes. Guide images enforce temporal coherence to prevent flicker, and the technique outperforms neural approaches. A free standalone tool is available at ebsynth.com.
Dear fellow scholars, this is 2 minute papers with Zsolnai neural style transfer just appeared 4 years ago in 2015. Style transfer is an interesting problem in machine learning research where we have two input images, one for content and one for style, and the output is our content image reimagined with this new style. The cool part is that the content can be a photo straight from our camera and the style can be a painting, which leads to the super fun results that you see here. However, most of these works are about photos, so what about video? Well, hold on to your papers because this new work does this for video and the results are marvelous. The process goes as we take a few keyframes from the video and the algorithm propagates our style to the remaining frames of the video. And wow, those are some silky smooth results. In specific, what I would like you to take a look at is the temporal coherence of the results. Proper temporal coherence means that the individual images within this video are not made independently from each other, which would introduce a disturbing flickering effect. I see none of that here, which makes me very, very happy. And now hold on to your papers again, because this technique does not use any kind of AI. No neural networks and other learning algorithms were used here. Ok, great, no AI, but is it any better than its AI based competitors? Well, look at this. Hell yeah. This method does this magic through building a set of guide images. For instance, a mass guide highlights the stylized objects and sure enough, we also have a temporal guide that penalizes the algorithm for making too much of a change from one frame to the next one, ensuring that the results will be smooth. Make sure to have a look at the paper for a more exhaustive description of these guides. Now, if we make a carefully crafted mixture from these guide images and plug them in to a previous algorithm by the name Stylet. We talked about this algorithm before in the series. The link is in the video description. Then we get these results that made me fall out of my chair. I hope you will be more prepared and held on to your papers. Let me know in the comments. And you know what is even better? You can try this yourself because the authors made a standalone tool available free of charge. Just go to ebsynth.com or just click the link in the video description. Let the experiments begin. Thanks for watching and for your generous support and I'll see you next time.
