YouTube commentary by Dr. Zsolnai (channel: LARRUZ) presenting and reacting to Google’s new text-to-video/film system (Phenaki). The video (uploaded Feb 12, 2023) explains how Phenaki chains multiple text prompts to generate coherent, longer videos (examples: teddy-to-panda transformation, astronaut on Mars, campfire zoom-out, motorcycle ride, penguin story). The presenter situates Phenaki relative to earlier models (DALL·E 2 — Apr 2022; Oct 2022 short-video models), highlights current limits (128×128 resolution, examples up to ~2.5 minutes), and argues this marks an early step toward high-quality AI-made movies.
Questions this video answers
February 12, 2023.
Phenaki.
DALL·E 2, April 2022.
October 2022.
Two and a half minutes (~2.5 minutes).
Chapters
Dear fellow scholars, this is two minute papers with Dr. Zsolnai. I have to pinch myself that I am saying this, but here goes. Today we are going to use Google's amazing new AI to create not an image and not even a short image sequence, but we are going to make an entire film, an AI made film. And all this within seconds. Yes, that's right. Less than a year ago, in April 2022, an amazing text to Image AI called Dall E2 was published by scientists at OpenAI. This took a piece of text and painted an image for us. And this paper ushered in a new era of AI assisted tools. And then something amazing happened. And not six years later, but only about six months later in October 2022, this follow up paper could generate not just images, but short videos of our choosing. But even if it did that really well, this was still made to create only that short sequences. A true miracle of AI research. However, here is another one. And hold on to your papers because now this new technique called Phenaki promises no less than entire films made by an AI. Entire films. Now, I am super excited for this. So let's have a look at five of my favorite examples together. 1. This new work's superpower is that we can write several text prompts and the AI finds a way to chain them together in a way that makes sense. That sounds exactly like a script for a movie. Amazing. If it works, of course. So first we can say a photorealistic teddy bear is swimming in the ocean in San Francisco. Then we say the teddy bear goes underwater. Then the teddy bear keeps swimming under the water with colorful fishes. And finally, get this, a panda bear is swimming underwater. And the AI finds out by itself how this magical transformation from teddy bear to a panda should take place. I love it. Now this was really cool. So how about another one? 2. We start out with a side view of an astronaut who's walking through a paddle on Mars, who then starts dancing. I am particularly interested in these transitionary moments. And this one checks out really cool. Then he walks his dog on Mars. And look at that. The dog does not seem to just grow out of nothing, but it gradually enters the frame. Once again, a graceful transition. And then the astronaut and this good boy who watch fireworks together. Well, kind of.
3.
I also loved that. We can also give the AI a little more explicit instructions. For instance, we can ask it to zoom out at the end of the video when the campfire appears. And it did exactly that.
Good job, little AI.
4. The paper also showcases a two and a half minute long video that starts out with a scenic motorcycle ride, running through the woods and finding beautiful things and of course robots and more. And five, while we look through the beautiful story of this penguin, let's talk about pixels. These are 128 by 128 videos. That is not a lot of pixels. Now of course these are not super high quality videos yet they do not compare to the super high quality images that text to image AIs are capable of. So why be so excited for this? Well, we are experienced fellow scholars here. So what do we do? Of course we apply the first Law of Papers. The first Law of Papers says that research is a process. Do not look at where we are, look at where we will be to two more papers down the line. So where will we be a couple more papers down the line? Well, for reference, this is what Dall? E1 was capable of. And just one more paper down the line. This was possible with Dall? E 2. And looking at the previous results in image generation, we are likely having a Dall? E. One moment for AI video generation. Yes, this means that we are likely on a cusp of a revolution. Just look at how much progress can be made in just one paper. So high quality AI movies are likely not impossible anymore and not even decades away. They might very well be just months away from now. What a time to be alive. And if you wish to be a part of this amazing progress in AI research, consider subscribing and becoming a fellow scholar. And even better, hitting the bell icon to not miss it. Now one this has become one of my favorite parts of the paper. Get this. It can even animate an already existing image of ours given a text prompt. So we can not only create entire movies, but we can even tell the AI what kind of movie we wish to create. Wow. My goodness. So what do you think? What would you use this for? Let me know in the comments below.
Weights and Biases provides tools to track your experiments in your deep learning projects. Using their system, you can create beautiful reports like this one to explain your findings to your colleagues better. It is used by many prestigious labs including OpenAI, Toyota Research, GitHub and more. And the best part is that Weights and Biases is free for all individuals, academics and open source projects. Make sure to Visit them through wnb.com papers or just click the link in the video description and you can get a free demo today. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you.
Thanks for watching and for your generous support, and I'll see you next time.
