AI Learns to Synthesize Pictures of Animals | Two Minute Papers #152

Dear fellow scholars, this is two minute papers with. I just finished reading this paper and I fell out of the chair. And I can almost guarantee you that the results in this work are so insane, you will have to double or even triple check to believe what you are going to see here. This one is about image translation, which means that the input is an image and the output is a different version of this input image that is changed according to our guidelines. Imagine that we have a Monet painting and we'd like to create a photograph of this beautiful view. There we go. What if we'd like to change this winter landscape to an image created during the summer? There we go. If we are one of those people on the Internet forums who just love to compare apples to oranges, this is now also a possibility. And have a look at this. Imagine that we like the background of this image, but instead of the zebras, we would like to have a couple of horses. No problem. Coming right up. This algorithm synthesizes them from scratch. The first important thing we should know about this technique is that it uses generative adversarial networks. This means that we have two neural networks battling each other in an arms race. The generator network tries to create more and more realistic images and these are passed to the discriminator network which tries to learn the difference between between real photographs and fake forged images. During this process, the two neural networks learn and improve together until they become experts at their own craft. However, this piece of work introduces two novel additions to this process. One, in earlier works, the training samples were typically paired. This means that the photograph of a shoe would be paired to a drawing that depicts it. This additional information helps the training process a great deal and the algorithm would be able to map drawings to photograph. However, a key difference here is that without such pairings, we don't need these labels. We can use significantly more training samples in our datasets, which also helps the learning process. If this is executed well, the technique is able to pair anything to anything else, which results in a remarkably powerful algorithm. Key difference number two. A cycle consistency loss function is introduced to the optimization problem. This means that if we convert a summer image to a winter image and then back to a summer image, we should get the very same input image back. If our learning system obeys to this principle, the output quality of the translation is going to be significantly better. This cycle consistency loss is introduced as a regularization term. Our seasoned fellow scholars already know what it means, but in case you don't, I've put a link to our explanation in the video description. The paper contains a ton more results and fortunately the source code for this project is also available. Multiple implementations in fact. Just as a side note, which is jaw dropping by the way, there's some rudimentary support for video. Amazing piece of work. Bravo. Now you can also see that the rate of progress in machine learning research is completely out of this world. No doubt that it is the best time to be a research scientist. It's easy. Ever been. If you've liked this episode, make sure to subscribe to the series and have a look at our Patreon page where you can pick up cool perks like watching every single one of these episodes in early access. Thanks for watching and for your generous support, and I'll see you next time.