A LARRUZ YouTube video (Oct 30, 2024) reviewing RunwayML's Act 1—an AI system that maps real actor facial performances onto images or 3D characters. The narrator uses humor (a running "lemon" gag) while describing Act 1's strengths—faithful facial motion transfer, professional-looking results especially on 3D characters—and limits, such as difficulty with camera movement and typically stable backgrounds. The video references Runway's blog demos and contrasts Act 1's AI-driven pipeline with traditional multi-step facial-animation workflows.
Chapters
Oh, hey, it's you. Yeah, you can see I'm in a little bit of a situation right now. But you were looking for a video on Act 1 today. I'll do my best, but as you can see, I've been converted into a lemon. It's a little bit difficult for me to even move right now. I asked Chat GPT make me a lemon, and I wanted it to use Dolly3, but you can see how that turned out. What the hell was that?
Oh, gosh.
Oh, someone making stop. Oh, man. You guys don't even want to know how I got out of that situation. All I gotta say is, thank God that cat had some good digestive health. But anyways, we're back. Yes, we're talking about RunwayML's Act 1 today. And if you don't know what Act 1 is, let me give you the lowdown. So this is the blog post for Act 1. This is made by RunwayML, and it's a new way to generate very expressive character performances using real video inputs made by a real actor. And you could record it with anything. It could be your cell phone, and you could act out pretty much every character in an entire story. Right. It's basically taking whatever facial movements that you act out in your video and then transplanting them on any image of any character, all using the power of AI. And the quality is really quite good as well. In a lot of cases, it looks professional, especially with 3D animation, for example. And there's a lot of different scenarios. Use cases and tests that we're going to get into to show you what Act 1 is truly capable of today. But I want to start off here on the actual Runway blog, showing you their demo videos.
Am I looking here?
Oh, there we go. Sorry, guys.
What, you're going to fire me because there's hot milk everywhere on the floor of the coffee shop? I mean, it's one latte. It's not that big of a deal. Everything's fine. Everything's totally great. This is going to be the best day ever. Smashed my phone the other night, and you know how much I love that phone. Okay, all right. Just breathe. Okay, Maybe don't breathe that hard. Who knew there were so many kinds of beans to choose from, Ava? Black kidney butter, white beans.
Oh.
Now you're surprised. Oh, wow. I'm beginning to think that you don't understand the gravity of this situation. You came all the way down to the Department of Motor Vehicles and didn't bring your driver's license. Settle down, everyone. Settle down, please. I Don't know how to be sad about closing my eyes. Perfect. We got it. Thank you so much. Thank you so much.
So, yeah, that's basically act one. I know in some of my other videos I have shown off this little demo that RunwayML put together, but I think it's a really, really good little advertisement for their product. And there's actually a lot to point out. So first of all, we know it can be transplanted on pretty much any character, whether it be 3D animated, a photo of a real person, etc, etc. The requirements seem to be that they have eyebrows, eyeballs, a nose and a mouth. Pretty much it. And it actually tries to detect the face when you're generating with it. We can see that it's also possible to do animated backgrounds, for example, but I have noticed that the camera can't really move. So we can have like a burning building or maybe even some swaying trees or something in the background, maybe. Possibly. I think it's a little bit difficult to get out of the model, but it is possible. It's just camera movement, isn't there yet. All kinds of different, varying styles of characters, whether they're animated and all kinds of different settings. And you can see that like in this example, we have the character's head totally turned to the side and it still does it very faithfully, very accurately, showing off the 3D detail as well in the character's face as he turns his head so his head can move. It's just a background can be a little bit difficult, I think. I think this one was a pretty impressive example of how much the characters can move in the scene and. And how far the act one can take that. But you'll notice like stable background in pretty much all of these clips, with very few limited exceptions. But the characters look good. They look very convincing. And I think if you use this well, you can produce something that is very much watchable sometimes maybe even indistinguishable from non 3D animated movies, I. I would say. Or something that wasn't created with AI. I think this one, for example, I gu this very realistic aspect of it, and it's not really all that creepy, but you can tell that it's not a real person. It almost looks like a super photorealistic render or something like that from Blender. It's so strange. But like, like I said, it's. It's very watchable. That's the whole point in the blog. They go on to say that they capture the essence of a performance. And I think that is very much true. Traditional pipelines for facial animation often involve complex multi step workflows. These can include motion capt equipment, multiple footage references, manual face rigging, among other techniques. That's absolutely true. The goal is to transpose an actor's performance onto a 3D model suitable for an animation pipeline. The key challenge with traditional approaches lies in preserving emotion and nuance from the reference footage into the digital character. Their approach obviously uses a completely different pipeline. It's all AI driven and all.
