Dr. Zsolnai demos Adobe's new AI for editing photorealistic video and images at scale. It can add synthetic data to video and edit people, pets, and cars, though it struggles with odd angles and occlusions. Try it yourself in a Colab notebook.
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. As you see here, I love Two Minute Papers. But fortunately I am not the only one. Elon Musk loves two Minute papers too. What's more, even cats love it. Everybody loves two Minute Papers. So what was that? Well, these are all synthetic videos that were made by this new magical AI. So here we are adding synthetic data to an already existing photorealistic video. And the AI is meant to be able to understand the movement, facial changes, geometry changes, and track them correctly. And move the mustache, the tattoos or anything else properly. That is a huge challenge. How is that even possible? Many AI based techniques from just a year or two ago were not even close to being able to pull this off. However, now hold on to your papers and have a look at this new technique. Now that is a great difference. Look at that. So cool. This is going to be really useful for at the very least two one, augmented reality applications. For instance, have a look at this cat. Well, the bar on the Internet for cat videos is quite high and this will not cut it. But wait. Yes, now we're talking. The other examples also showcase that the progress in machine learning and AI research these days is. Is absolutely incredible. I would like to send a big thank you to the authors for taking the time off their busy day to create these results only for us. That is a huge honor. Thank you so much. So far this is fantastic news. Especially given that many works are limited to one domain. For instance, Microsoft's human generator technique is limited to people. However, this works on people, dogs, cats and even Teslas. So good. I love it. What a time to be alive. But wait, I promised two applications, not one. So what is the other one? Well, image editing. Image editing? Really? What about that? That is not that new, but aha. Not just image editing, but mass scale image editing. Everybody can get antlers, tattoos, stickers, you name it. We just chuck in the data set of images, choose the change that we wish to make, and out comes the entirety of the edited dataset automatically. Now that is a fantastic value proposition. But of course, the seasoned fellow scholar immediately knows that surely not even this technique is perfect. So where are the weak points? Look. Oh yes, that. I see this issue for nearly every technique that takes on this task. It still has troubles with weird angles and occlusions. But we also have good news. If you can navigate your way around an online collab notebook, you can try it too. You know what to do. Yes. Let the experiments begin. So what would you use this for? Who would rock the best mustache. And what do you expect to happen a couple more papers down the line? Please let me know in the comments below. This episode has been supported by Cohere AI Cohere builds large language models and makes them available through an API so businesses can add advanced language understanding to their system or app quickly. With just one line of code, you can use your own data, whether it's text from customer service requests, legal contracts or social media posts, to create your own custom models to understand text or even generate it. For instance, it can be used to automatically determine whether your messages are about your business hours, returns or shipping. Or it can be used to generate a list of possible sentences you can use for your product descriptions. Make sure to go to Cohere AI Papers or click the link in the video description and give it a try today. It's super easy to use. Thanks for watching and for your generous support and I'll see you next time.
