These Neural Networks Have Superpowers! 💪

Dear fellow scholars, this is two minute papers with Dr. Zsolnai. I got so excited by the amazing results of this paper. I will try my best to explain why. And by the end of this video there will be a comparison that blew me away and I hope you will appreciate it too. With the rise of neural network based learning algorithms, we are leaving the advent of image generation techniques. What you see here is a set of breathtaking results created with a technique called Stylegan too. This can generate images of humans, cars, cats and more. As you see, the progress in machine learning based image generation is just stunning. And don't worry for a second about the progress in text processing, because that is also similarly amazing these days. A few months ago, OpenAI published their GPT3 model that that they unleashed to read the Internet and learn not just our language, but much, much more. For instance, the Internet also contains a lot of computer code, so it learned to generate website layouts from a written description. But that's not all. Not even close to the joy of technical PhD students around the world. It can properly typeset mathematical equations from a plain English description as well. And get this, it can also translate a complex legal text into plain language or the other way around. And it does many of these things nearly as well as humans. So what was the key to this work? One of the keys of GPT3 was that it uses a neural network architecture that is called the Transformer network. These really took the world by storm in the last few years. So our first question, why transformers1? Transformer networks can typically learn on stupendously large data sets like the whole Internet and extract a lot of information from it. That is a very good thing. And two, Transformers are attention based neural networks, which means that they are good at learning and generating long sequences of data. Okay, but how do we benefit from this? Well, when we ask Openais GPT3 to continue our sentences, it is able to look back at what we have written previously. And it looks at not just a couple of characters. No, no. It looks at up to several pages of writing backwards to make sure that it continues what we write the best way it can. That sounds amazing, but what is the lesson here? Just use transformers for everything and off we go. Well, not quite. They are indeed good at a lot of things when it comes to text processing tasks, but they don't excel at generating high resolution images at all. Can this be improved somehow? Well, this is what this new technique does and much, much more. So let's dive in and see what it can do first. We can give it an incomplete image and ask it to finish it. Not bad. But OpenAI's ImageGPT could do that too. So what else can it do? Oh boy, a lot more. And by the way, we will compare the results of this technique against ImageGPT at the end of this video. Make sure not to miss that. I. I almost fell off the chair. You will see in a moment. Why? Two, it can do one of my favorites, depth to image generation. We give it a depth map, which is very easy to produce, and it creates a photorealistic image that corresponds to it, which is very hard. We do the easy part. The AI does the hard part. Great. And with this, we not only get a selection of these images, but. But since we have their depth maps, we can also rotate them around as if they were 3D objects. Nice. Three, we can also give it a map of labels, which is again, very easy to do. We just say, here goes the sea, put some mountains there and the sky here, and it will create a beautiful landscape image that corresponds to that. I can't wait to see what amazing artists all over the world will be able to get out of these techniques. And these results are already breathtaking. But research is a process, and just imagine how good they will become. Two more papers down the line. My goodness. Four, it can also perform super resolution. This is the CSI thing, where in goes a blurry image and out comes a finer, more detailed version of it. Witchcraft. And finally, five, we can give it a pose, and it generates humans that take these poses. Now, the important thing here is that it can supercharge transformer networks to do these things at the same time with just one technique. So how does it compare to OpenAI's image completion technique? Well, remember, that technique was beyond amazing and set a really high bar. So let's have a look together. They were both given the upper half of this image and had to fill in the lower half. Remember, as we just learned, transformers are not great at high resolution image synthesis. So here, for OpenAI's ImageGPT, we expect heavily pixelated images and. Oh yes, that's right. So now hold on to your papers and let's see how much more detailed the new technique is. Holy mother of papers. Do you see what I see here? ImageGPT came out just a few months ago, and there's already this kind of progress. So there we go. Just imagine what we will be able to do with these supercharged transformers that just two more papers down the line. Wow. And that's where I almost fell off the chair when reading this paper. Hope you held onto yours. It truly feels like we are living in a science fiction world. What a time to be alive. What you see here is a report of this exact paper we have talked about which was made by Weights and Biases. I put a link to it in the description. Make sure to have a look. I think it helps you understand this paper better. Weights and Biases provides tools to track your experiments in your deep learning projects. Their system is designed to save you a ton of time and money and it is actively used in projects at prestigious labs such as OpenAI, Toyota Research, GitHub and more. And the best part is that Weights and Biases is free for all individuals, academics and open source projects. It really is as good as it gets. Make sure to Visit them through WNB.com papers or or just click the link in the video description and you can get a free demo today. Our thanks to WeightsAndBiases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.