Google’s New AI: These Are More Than Images!

Dear fellow scholars, this is two minute papers with Dr. Today we will see that modern AIs can not only generate images, but they can also make them come alive. Oh yes. Now as all of you know, we already have a bunch of text to image AIs around OpenAI's Dall E2 and a free variant, stable diffusion and more. Here we enter a text prompt and it paints a beautiful image for us that satisfies our prompt. And whenever I create such an image, I am always wondering, what if we could also use this AI to actually edit these images? Or you know what, maybe even videos to our liking. Today we will find out whether that is possible. And wait, today images are not the only thing we can generate with these techniques. As of October 2022, Google's Imagen video AI is also out there. And with this we have not just text to image, but text to video. And the results are truly insane. And this work from scientists at Google, this is a little different. It builds on stylegan instead. This is not a text to image AI, but a generative model. The visual quality it can create is simply astounding. Just look at that. And get this. This new paper just added a bunch of amazing text to video like features to it. Now, it can take a generative model like this and create a video for us. But we can choose what should happen in this video. How? Well, first we give it a bunch of images, for instance, about horses. And it now generates new high quality images within this domain. But here's the key. These images are generated by using a latent space. What is that? You see an example of a latent space here from our previous paper, which helps an artist generate similar virtual material models by exploring with this dot on a 2D plane. Now, most of these techniques guarantee that when exploring nearby, we will get similar material models. However, it seems almost impossible to say, for instance, that we wish to see more metallic materials. We get similar materials, but we can't describe exactly what we are looking for. However, hold on to your papers, because this new technique claims that it can do exactly that. Well, I will believe it when I see it. Let's see. Whoa. We can make this lion roar. And many others too. And what I really love about these results is that almost all of the intermediate images that were generated fit together so well that it seems like a believable continuous video. But that's not all. It can do a ton more. For instance, it can rotate the heads of these parrots. And temporal consistency is remarkably good here too, which. Which means that we don't have a ton of flickering now. Make no mistake, there is some. But these are getting closer and closer to being believable as actual video footage. What a time to be alive. And if we have just the perfect parrot, but it takes up just a bit too much of the frame. Not a problem. Look, zooming in and out of images is also possible, and it fills in the remaining parts of the image with sensible information. More of the parrot or more background. How cool is that? And if we have created the perfect horse image with this AI, but we are yearning for a little more action. Not a problem. Look, it can even make them run. Some of the intermediate images are not perfect. So this might not pass as a video and yet, but as images. Incredible. The paper is available in the video description and it is chock full of these incredible results. And you know what? Let's pop the hood and look inside together. For instance, they propose a new filtering step that supposedly has magical powers. Does it? Let's see. A somewhat unorganized dataset comes in, for instance. These can be made by another AI. We bunch them up, throw it at the new neural network, and after filtering them down to a smaller subset, the new technique can generate significantly better images than other methods could before. How much better? Let's see. Whoa, look at that. Other AIs prefer the results generated with the new filtering step. That's good news. But come on, we are not computers. We are humans. Does this really say anything about what humans prefer? You bet your papers it does. Look, humans are also loving the results with the new technique, and the difference is remarkable.

So good.

So, welcome to Two Minute Papers, where we look at tables and flip out together. Hope you're enjoying it as much as I do. So what do you think? Does this get your mind going? Let me know in the comments below.

This episode has been supported by Cohere AI. Cohere builds large language models and makes them available through an API so businesses can add advanced language understanding to their system or app quickly. With just one line of code, you can use your own data, whether it's text from customer service requests, legal contracts, or social media posts, to create your own custom models to understand text or even generate it. For instance, it can be used to automatically determine whether your messages are about your business hours, returns, or shipping. Or it can be used to generate a list of possible sentences you can use for your product descriptions. Make sure to go to Cohere AI papers or click the link in the video description and give it a try.

Today, it's super easy to use. Thanks for watching and for your generous support, and I'll see you next time.