Károly Zsolnai-Fehér explains a new AI method that transfers style between images by matching semantic parts like noses and eyes, not just pixels. It works on photos, sketches, videos, and timelapses, and the code is available online.
August 6, 2017.
#177.
LARRUZ.
Károly Zsolnai-Fehér.
Yes — the video states the source code of the project is available (link in the description).
Dear fellow scholars, this is 2 minute papers with Zsolnai style transfer is an amazing area in machine learning and AI research where we take two images. Image number one is an input photograph and image number two is the desired style. And the output of this process is the content of image number one with the style of image number two. This first paper opened up an incredible new area of research a As a result, a ton of different variants have emerged in the last two feedforward style transfer for close to real time results, temporally coherent style transfer for videos, and much much more. And this one not only outperforms previously existing techniques, but also broadens the horizon of possible style transfer applications. And obviously a human would be best at doing this because a human has an understanding of the objects seen in these images. And now hold on to your papers because the main objective of this method is to create semantically meaningful results for style transfer. It is meant to do well with input image pairs that may look completely different visually, but have some semantic components that are similar. For instance, a photograph of a human face and the drawing of a virtual character is an excellent example of that. In this case, this learning algorithm recognizes that they both have noses and uses this valuable information in the style transfer process. As a result, it has three super cool applications. First, the regular photo to style transfer that we all know and love. Second, it is also capable of swapping the style of two input images. Third, and hold on to your papers because this is going to be even more insane style or sketch to photo and we have a one here as well. So. Fourth, it also supports color transfer between photographs which will allow creating amazing timelapse videos. I always try to lure you fellow scholars into looking at these papers, so make sure to have a look at the paper for some more results on this. And you can see here that this method was compared to several other techniques. For instance, you can see the cycle consistency paper and patch match. And this is one of those moments when I get super happy because more than 170 episodes into the series we can not only appreciate the quality of these new results, but we also had previous episodes about both of these algorithms. As always, the links are available in the video description. Make sure to have a look. It's going to be a lot of fun. The source code of this project is also available. We also have a ton of episodes on computer graphics in the series. Make sure to have a look at those as well. Every now and then I get emails from viewers who say that they came for the AI videos and just in case watched a recent episode on computer graphics and were completely hooked. Give it a try. Thanks for watching and for your generous support, and I'll see you next time.
