NVIDIA's AI Makes Amazing Slow-Mo Videos! 🚘

Dear fellow scholars, this is two minute papers Withehir, how about some slow motion videos? If we would like to create a slow motion video and we don't own an expensive slo mo camera, we can try to shoot a normal video and simply slow it down. This sounds good on paper. However, the more we slow it down, the more space we have between our individual frames and at some point our video will feel more like a slideshow. To get around this problem, in a previous video we discussed two basic techniques to fill in these missing frames. One was a naive technique called frame blending that basically computes the average of two images. In most cases, this doesn't help all that much because it doesn't have an understanding of the motion that takes place in the video. The other one was optical flow. Now this one is much smarter as it tries to estimate the kind of translation and rotational motions that take place in the video and and they typically do much better. However, the disadvantage of this is that it usually takes forever to compute and it often introduces visual artifacts. So now we are going to have a look at Nvidia's results and the main points of interest are always around the silhouettes of moving objects, especially around regions where the foreground and the background meet. Keep an eye out for these regions throughout this video. For instance, here is one example I found. Let me know in the comments section if you have found more. This technique builds on U net, a super fast convolutional neural network architecture that was originally used to segment biomedical images from limited training data. This neural network was trained on a bit over a thousand videos and computes multiple approximate optical flows and combines them in a way that tries to minimize artifacts. As you see in these side by side comparisons it works amazingly well. Some artifacts still remain but are often hard to catch. And this architecture is blazing fast. Not real time yet, but creating a few tens of these additional frames takes only a few seconds. The quality of the results is also evaluated and compared to other works in the paper. Make sure to have a look. As the current commercially available tools are super slow and take forever. I cannot wait to be able to use this technique to make some more amazing slow motion footage for you fellow scholars. Thanks for watching and for your generous support and I'll see you next time.