Google researchers used evolution strategies to reconstruct images with transparent triangles, achieving results 50 times faster than previous methods. The algorithm reorganizes shapes to match target images like the Mona Lisa, and can generate images from text descriptions by combining it with CLIP.
Questions this video answers
June 25, 2022.
10,000 steps of the new method is close to equivalent to about 500,000 steps of the previous algorithm.
As few as 200 or even just 25 triangles.
Weights & Biases; stated as free for individuals, academics, and open source projects.
Chapters
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today is going to be all about simulating evolution on our computers. This can build virtual cars, virtual creatures, and even paint the Mona Lisa. These all sound amazing, but evolution on a computer how? A few years ago, a really fun online app surfaced that used a genetic algorithm to evolve the morphology of a simple 2D car with the goal of having it roll as far away from a starting point as possible. A genetic algorithm? What is that? Well, it is a super simple technique where we start out from a set of random solutions and likely find out that none of them really work too well. However, we start combining and mutating, or in other words, changing small parts of this solution until, oh yes, something starts to move. Now, as soon as at least one wheel is placed correctly, the algorithm will recognize that this one rolled so much further and keep the genes of this solution in the pool to breed new similar solutions from it. A similar concept can also be used to design the optimal morphology of a virtual creature to make it skip forward faster. And look. That is so cool. Over time, the technique learns to more efficiently navigate a flat terrain by redesigning its legs that are now reminiscent of small springs and uses them to skip its way forward. And here comes something even cooler. If we change the terrain, the design of an effective agent also changes accordingly. And the super interesting part here is that it came up with an asymmetric design that is able to climb stairs and travel uphill efficiently. Loving it. And in this new paper, scientists at Google Brain tried to turbocharge an evolutionary algorithm to be able to take a bunch of transparent triangles and get it to reorganize them so they will paint the Mona Lisa for us or any image in particular. The technique they are using is called evolution strategies and they say that it is much faster at this than previous techniques. Well, I will believe it when I see it. So now hold on to your papers and let's see together. Here is a basic evolutionary algorithm after 10,000 steps. Well, it is getting there, but it's not that great. And let's see how well their new method does with the same amount of steps. Wow. My goodness. That is so much better. In fact, their 10,000 steps is close to equivalent to half a million steps with a previous algorithm. And we are not done yet. Not even close. The paper has two more amazing surprises. Surprise number one, we can even combine it with OpenAI's technique called Clip. This learns about pairs of images and their captions describing these images, which is an important part of Dall E2, their amazing image generator AI. This could take completely outlandish concepts and create beautiful photorealistic images out of it that are often as good as we would expect from a good human artist. Scholars holding onto their papers, a cyber frog, you name it. So get this. Similarly to that, we can even write a piece of text and the evolutionary triangle builder will try to create an image that fits this text description. With that, we can request a triangle version of a self portrait a human and look at how similar they are. Food for thought. But it can also try to draw Walt Disney World. That is remarkable. Look at how beautifully it boils it down to its essence. With as few as 200 or even just 25 triangles. Loving it. Also drawing the Google headquarters. No problem at all. And here comes surprise number two. The authors claim that it is even faster than a differentiable renderer. These are really powerful optimization techniques that can even grow this beautiful statue out of nothing and do all that in just a few steps. So claiming that it is even faster than that, well, that is very ambitious. Let's have a look. And oh yes, as expected, the differentiable technique creates a beautiful image very quickly. Now try to beat that. Wow. The new method converges to the final image even quicker. That speed is simply stunning. Interestingly, they also have different styles. The differentiable renderer introduces textures that are not present in the original image, while the new technique uses large triangles to create a smooth approximation of the background and the hair, and uses the smaller ones to get a better approximation of the intricate details of the face. Loving it. Now let's ramp up the challenge and test these a little more and add some prompts. And oh yes, this is why the differentiable technique flounders on the prompt look. It almost seems to try to do everything all at once. While the new technique starts out from a fair approximation and converges to a great solution super quickly. I hope that this new take on evolution strategies will be plugged in to applications where differentiable techniques do well and perhaps do even better. That would be absolutely amazing because these are a treasure trove of science fiction like applications. For instance, we would be able to almost instantly solve this billiard game with just the right amount of force and from the right direction, such that the blue ball ends up close to the black spot. Or simulate ink with a checkerboard pattern and exert just the appropriate forces so that it forms exactly the yin yang symbol shortly after. Or here comes a previous science fiction like example. This previous differentiable technique adds carefully crafted ripples to the water to make sure that it ends up in a state that distorts the image of the squirrel in a way that a powerful and well known neural network sees it not as a squirrel, but as a goldfish. Wow. And if this new evolutionary technique could do all of these tasks, but better and faster, sign me up right now. What a time to be alive. What you see here is a report of this exact paper we have talked about which was made by Weights and Biases. I put a link to it in the description. Make sure to have a look. I think it helps you understand this paper better. Weights and Biases provides tools to track your experiments in your deep learning projects. Using their system you can create beautiful reports like this one to explain your findings to your colleagues better. It is used by many prestigious labs including OpenAI, Toyota Research, GitHub and more. And the best part is that Weights and Biases is free for all individuals, academics and open source projects. Make sure to Visit them through wnb.com papers or just click the link in the video description and you can get a free demo today. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.
