Ross Zeiger tests Descript's new AI avatar feature that turns still photos into talking video. The system struggles with unusual lighting and unnatural head movement, but handles lip sync reasonably well. Cartoon avatars work cleaner than realistic photos.
If you're new to video, maybe you avoid hitting record because you hate being on camera or setting up equipment. Feels overwhelming. I totally get it. But what if an AI could be you on screen? In this video, I'm going to show off Descript's mind blowing new AI avatar feature and demo. Exactly how it lets you create videos that look like you're on camera without ever actually having to be.
I'm just going to show you a couple demos of how the AI avatar feature performs under different camera lighting conditions, etc. And how it performs with Descript's own AI avatar. So my first test was with unusual lighting. So over here on the right side is the reference photo. This was the photo that I gave the system. And here is the video it came up with.
The past doesn't exist, it's just memories in the present. The future doesn't exist, it's just thoughts in the present, all that exists is now.
So it changed my face quite a bit, it changed the eyes, and there's a lot of unnatural head movement. But with everything I'm showing you today, keep in mind, this is like v0, this is brand new. So a year from now this is going to look laughable. But that's why I'm showing you this. This is kind of a timepiece of what we have right now. Moving on, test number two. For this one I used this photo and what I was testing in this one is how it would affect my hand and body. So is the AI going to animate my hand and my body in this photo since my hand is up near my face? Let's take a look.
The past doesn't exist, it's just memories in the present. The future doesn't exist, it's just thoughts in the present. All that exists is now.
So yes it does, it does change my hand. Look at that right at the end. It's like turns into a super blurry, blobby looking thing. By the way, the voices are descripts, built in stock voices. So that was a female voice obviously. But once again with all of these, the head movements are a little unnatural, the eyes are unnatural, but overall I think the better lighting gave us a slightly better result on the face. And my third experiment is with a blurry photo. So this was just kind of an out of focus photo. And we're going to see how that does with the AI avatar. And by the way, if you want to see how to make this yourself, then click on the video up here where I go into the steps of actually generating your face into one of these AI avatars. But here we go with test number three, the blurry photo.
The past doesn't exist, it's just memories in the present. The future doesn't exist, it's just thoughts in the present, all that exists is now.
In some ways that's almost a better result because the, the out of focus of my face kind of distracts from the weird eye movements, the weird distortions on the cheeks and the jaw, but it definitely didn't ruin it or give us a bad result. So that's an interesting observation there. Test number four. This was just a webcam photo. So it's super excessively bright, it's a little bit blown out and it's a lower resolution photo. And let's see how it does with just our webcam photo. Here we go.
The past doesn't exist, it's just memories in the present. The future doesn't exist, it's just thoughts in the present. All that exists is now.
That was actually the best one so far in terms of the face looking natural. It still changed my face. Like to me, that doesn't look like me anymore. It looks like it's a different person, but it's at least not like totally weird. Like there's not like weird things happening to the cheeks or to the jaw. And it looks like it retains more of that humanness. Even if it's a different human than me, it still maintains that. Okay. And the fifth photo for myself was a higher resolution photo. This was taken with a DSLR camera. And so this is going to be the highest, best lighting, all that kind of stuff. Let's see how it does with a really good photo.
The past doesn't exist, it's just memories in the present. The future doesn't exist, it's just thoughts in the present. All that exists is now.
I think it retained my face a little bit better. Definitely better than the previous one. But it's got all the same issues. You know, the weird head nods, the eyes get a little bit distorted. But I think it retained my, it looked like me throughout. So I think in that sense this was a better result. Now let's go to Descript's actual built in avatars, which they're all these cartoon looking ones and because they're cartoon, it's harder to see, you know, the wrinkles in the face and the little minute facial movements. And so you can get a lot more a cleaner result by using these cartoons. And that's why they've provided them for now. So what you could do is create your own cartoon avatar using, you know, something like Apple's AI Playground or ChatGPT. Make a cartoon version of yourself and then use that as your avatar model. But let's see how this does. The past doesn't exist, it's just memories in the present. The future doesn't exist, it's just thoughts in the present.
All that exists is now.
So, yeah, like I said, you don't see all the little minute facial movements, the forehead, you know, things like that. So it seems a lot more natural. And we're used to seeing, like, there is no such thing as natural when you're looking at a cartoon. Right. So our brains have kind of gotten used to seeing like a Pixar film or a Disney film. We're used to seeing animated anamorphic creatures and we can pass it off as lifelike. And so these tend to work pretty well. And I think if you make your own cartoon version of yourself and then use that as your AI avatar, that could be a pretty interesting approach. Here's our second cartoon one.
The past doesn't exist, it's just memories in the present. The future doesn't exist, it's just thoughts in the present. All that exists is now.
Same thing there. We just got a female and different voice, slightly different movement, but still better overall than our real human photos. And let's do one last cartoon character here and let's see how she does.
The past doesn't exist, it's just memories in the present. The future doesn't exist, it's just thoughts in the present. All that exists is now.
I should say with all of these, the lips are pretty decent. Like the lips actually look like they're forming. The words exaggerated. Yes. Like the mouth is opening too wide at parts or not wide enough at other parts. But the mouth part of it seems pretty good. It's the rest of the body, like the way that the shoulders move. It's. It seems a little bit unnatural in that sense. And then the very last one was the intro to the video that you're watching right now, which was another one.
You're new to video. Maybe you avoid hitting record because you hate being on camera.
So another thing to point out here is it looks like it gave me a little mole next to my. My mouth right there. So that's interesting. And I have brown eyes. It changed my eyes to like blue or green. So again, we got some face change going on with the webcam photo. This was just a webcam photo that I took for the intro. If you want to go deeper into making AI avatars and the rest of Descript's features. Check out the Descript Mastery School, my program for teaching you how to become a video creator.
