NVIDIA’s New AI: Nature Videos Will Never Be The Same!

Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to use Nvidia's new AI to create absolutely magical time lapse videos that were otherwise nearly impossible to make until now. You see, watching a video of a landscape in real time is a smooth experience, but not as interesting as seemingly nothing is happening now. Speeding them up creates us a time lapse video. This is finally interesting. However, then this happens. Oh yes, we get too much flickering. This is not for human consumption, that's for sure. If only an AI based algorithm existed that could somehow take the essence of the changes in this video, but make it smooth as well. Now of course, that sounds flat out impossible. Why? Well, in these time lapse videos we experience drastic weather changes within a day. Photos for some days are missing, and even worse occlusions might happen. Now let's have a look at the previous method trying to do that. Hmm, something is happening here. It is able to keep the weather conditions stable, so we get no more flickering. However, do you see? The problem growing is clearly happening, but the details have been washed out so much that the most interesting part of the video is now gone. I am not that hopeful that this is possible to solve. Well, but just in case, let's have a look at Nvidia's new AI and see what it is made of. This is the input. And wow. The new method does an absolutely outstanding job at keeping the weather conditions stable, but in the meantime, also showing us sharp footage of this amazing growth. Now let's compare it to the previous method and my goodness, there is no contest here. So much better. Loving it. And let's have a look at another example of this valley. And then I will show you that this technique also has some more superpowers. As expected, the input video has way too much going on. So now let's ask the new method to boil it down to its essence. That is absolutely incredible. Look smooth and detailed at the same time. And yes, I promised some superpowers. So what are those? Well, this technique can not only stabilize the weather conditions and create smooth and detailed videos, but one depending on our artistic goals. It even gives us tools to control the time of day. And if we wish to think bigger, we can also control the time of year. But all that is nothing compared to my favorite. So hold on to your papers for trends. What is that? If there are structural changes in a landscape or a city, for instance, at some point this building has been repainted and now we can decide to forget about it and and keep it as if it never even happened. Or we can even decide to use its new appearance for the entirety of the year even before it was painted. Wow. And we can also account for growth. This growth can take place as the video progresses. Or look, it can be fixed in time. This is so good. I love it. And now time for some interesting tidbits. This technique is based on the legendary Stargan system which is able to synthesize these photorealistic humans and other things in a somewhat controllable manner. Also, I bet the authors had a few conversations asking each other how on earth do we put all this video information into one stationary image in the paper? This concept is a little harder to convey there, so I am so happy to be able to show you these videos instead. And what I also loved is this look. Are you seeing what I am seeing? My goodness. Some of these data sets have not only a few days missing here and there. There are often a series of tens of days missing or even worse. And the new AI can still create a smooth video of the changes as if nothing happened. It synthesizes what happened in between. And there is something that is even worse. Look, the reality of taking these photos is that the location of the camera can change over the year, making this barn appear to be bending upwards. And the new AI can fix that too. Not a problem at all. It is absolutely incredible that humans are able to build AI based algorithms that are this capable today. What a time to be alive. Now, not even this technique is perfect. You see here that for instance the chairs are still moving around. But have a look at how much progress has been made. Just one more paper down the line. My mind is officially blown. And good news. The source code and pre trained models are also available so everyone who has the available resources and expertise can start using it right now. And given Nvidia's excellent track record in tech transfer, this might make an introduction in one of their products soon. So what do you think? What would you use this for? Let me know in the comments below.

If you're looking for inexpensive cloud GPUs for AI, Lambda now offers the best prices in the world for GPU Cloud compute. No commitments or negotiation required. Just sign up and launch an instance. And hold on to your papers because with Lambda GPU Cloud you can get on demand a 100 instances for $1.10 per hour versus $4.10 per hour with AWS. That's 73% savings. Did I mention they also offer persistent storage? So join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs.compapers to sign up for for one of their amazing GPU instances today.

Thanks for watching and for your generous support and I'll see you next time.