Dr. Zsolnai explains neural cellular automata, which combine cellular automata with neural networks to grow shapes and regenerate when damaged. The technique creates seamless textures and animations that can self-heal, and you can run the interactive demos in your browser.
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to play with a cellular automaton and fuse it together with the neural network. It will be quite an experience. But of course, first, what are these things anyway? You can imagine a cellular automaton as a small game where we have a bunch of cells and a set of simple rules that describe when a cell should be full and when it should be empty. What you see here is a popular example called Game of Life, which simulates a tiny world where each cell represents a life form. So why is this so interesting? Well, this cellular automaton shows us that even a small set of simple rules can give rise to remarkably complex life forms, such as gliders, spaceships, and even John von Neumann's universal constructor, or in other words, self replicating machines. Now this gets more interesting. A later paper fused the cellular automaton with a neural network. It was tasked to grow and even better, maintain a prescribed shape. Remember these two words, grow and maintain shape. And the question was, if it can recover from undesirable states, can it perhaps regenerate when damaged? Well, here you will see all kinds of damage and then this happens. Nice. The best part is that this thing wasn't even trained to be able to perform this kind of regeneration. The objective for training was that it should be able to perform its task of growing and, and maintaining shape. And it turns out some sort of regeneration is included in that. This sounds very promising. And I wonder if we can apply this concept to something where healing is instrumental. Are there such applications in computer science? If so, what could those be? Oh, yes, yes, there are. For instance, think about texture synthesis. This is a topic that is subject to a great deal of research in computer graphics. And those folks have this down to a science. So what are we doing here? Texture synthesis typically means that we need lots of concrete or gravel road skin, marble, create unique stripes for zebras, for instance, for a computer game or the post production of a movie. And we really don't want to draw miles and miles of these textures by hand. Instead, we give it to an algorithm to continue this small sample where the output should be a bigger version of this pattern with the same characteristics. So how do we know if we have a good technique at hand? Well, first it must not be repetitive, check mark, and it has to be free of seams. This part means that we should not be able to see any lines or artifacts that would quickly give the trick away. Now get this. This new paper attempts to do the same with Neural cellular automatons. What an insane idea. We like those around here. So let's give it a try. How? Well, first, by trying to expand this simple checkerboard pattern, the algorithm is starting out from random noise. And as it evolves. Well, this is a disaster. We are looking for squares, but we have quadrilaterals. They are also misaligned and they are also inconsistent. But luckily we are not done yet. And now hold on to your papers and observe how the grid cells communicate with each other to improve the result. First the misalignment is taken care of, then the quadrilaterals become squares and. And then the consistency of displacement is improved and the end result is lock. In this other example, we can not only see these beautiful bubbles grow out of nowhere, but the density of the bubbles remains roughly the same over the process. Look, as two of them get too close to each other, they coalesce or pop. Damaged bubbles can also regrow. Very cool. Ok, it can do proper texture synthesis, but so can a ton of other handcrafted computer graphics algorithms. So why is this interesting? Why bother with this? Well, first you may think that the result of this technique is the same as other techniques, but it isn't. The output is not necessarily an image, but can be an animation too. Excellent. Here it was also able to animate the checkerboard pattern. And even better, it can not only reproduce the weave pattern, but the animation part extends to this too. And now comes the even more interesting part. Let's ask, why does this output an animation and not an image? The answer lies within these weaving patterns. We just need to carefully observe them. Lets see. Yes, again we start out from the noise where some woven patterns emerge. But then it almost looks like a person who started weaving them until it resembles the initial sample. Yes, that is the key. The neural network learned to create not an image, not an animation, but no less than a computer program to accomplish this kind of texture synthesis. How cool is that? So, armed with all that knowledge, do you remember the Regenerating Iguana project? Let's try to destroy these textures too and see if it can use these computer programs to recover and get us a seamless texture. First we delete parts of the texture, then it fills in the gap with noise. And now lets run that program. Wow. Resilient self healing texture synthesis. How cool is that? And in every case it starts out from a solution that is completely wrong, improves it to be just kind of wrong, and after further improvement. There you go. Fantastic. What a time to be alive. And note that this is a paper in the wonderful distilled journal, which not only means that it is excellent, but also interactive. So you can run many of these experiments yourself right in your web browser. The link is available in the video description this episode has been supported by Weights and Biases. In this post they show you how to debug and compare models by tracking predictions and hyper parameters, GPU usage and more. If you work with learning algorithms on a regular basis, make sure to check out weights and biases. Their system is designed to help you organize your experiments and it is so good it could shave off weeks or even months of work from your projects and is completely free for all individuals, academics and open source projects. This really is as good as it gets and it is hardly a surprise that they are now used by over 200 companies and research institutions. Make sure to visit them through wnb.com papers or just click the link in the video description and you can get a free demo today. Our thanks to WeightsAndBiases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.
