Károly Zsolnai-Fehér explains how a 19-layer neural network combined with Markov random fields can paint in different artistic styles while preserving image structure better than previous methods. The algorithm captures features at different levels, from edges to full objects, and the team documented where it fails.
Dear fellow scholars, this is two minute papers withehir. In a previous episode we discussed how a machine learning technique called a convolutional neural network could paint in the style of famous artists. The key thought is that we are not interested in individual details. We want to teach the neural network the high level concept of artistic style. A convolutional neural network is a fantastic tool for this since it does not only recognize images well, but the deeper we go in the layers, the more high level concepts neurons will encode. Therefore the better idea the algorithm will have of the artistic style. In an earlier example we've shown that the neurons in the first hidden layer will create edges as a combination of the input pixels of the image. The next layer is a combination of edges that create object parts. One layer deeper, a combination of object parts create object models and this is what makes convolutional neural networks so useful in recognizing them. In this follow up paper, the authors use a very deep 19 layer convolutional network that they mix together with Markov random fields, a popular technique in image and texture synthesis. The resulting algorithm retains the important structures of the input image significantly better than the previous work, which is also awesome. By the way. Failure cases are also reported in the paper, which was a joy to read. Make sure to take a look if you're interested. We also have a ton of video resources in the description box that you can voraciously consume for more information. There is already a really cool website where you either wait quite a bit and get results for free, or you pay someone to compute it and get results almost immediately. If any of you are in the mood of doing some neural art of something two minute papers related, make sure to show it to me. I'd love to see that as a criticism. I've heard people saying that the technique takes forever on an HD image, which is absolutely true. But please bear in mind that the most exciting research is not speeding up something that runs slowly. The most exciting thing about research is making something possible that was previously impossible. If the work is worthy of attention, it doesn't matter if it's slow. 3 follow up papers later, it will be done in a matter of seconds. In summary, the results are nothing short of amazing. I was full of ecstatic glee when I first seen them. This is insanity and it's only been a few months since the initial algorithm was published. I always say this, but we are living amazing times indeed. Thanks for watching and for your generous support and I'll see you next time.
