How to Auto Lip Sync Mouths in Adobe Animate I 2023

Working, working, cool. Today I'm going to be designing a character in Adobe Animate. I already have the sketch drawn, doing an old-school tutorial here. No fancy edits, no fancy jump cuts, just a couple of cameras. And now I gotta focus for about an hour and design this character. So let's jump in and get started, and hopefully you learn something about Adobe Animate. Welcome to the weird world of Adobe Animate, formerly known as Adobe Flash. If you're as old as I am, aka a dinosaur, you will remember Flash from back in the day in the early 2000s. Uh, Homestar Runner, classic. Uh, some other cartoons I can't remember the names of, but tons of cartoons have been designed and animated in Flash. Adobe Animate is pretty cool. There's been some updates since Flash, but I won't bore you with those. I'm gonna give you a little overview here of how we're gonna design this character, and this is gonna be— we're gonna be designing Bear, and the character on the right here, and this is Heiner on the left, who I already have designed and rigged. So let's pick apart this character first. So you'll notice I'm on the main timeline. I don't have any of my layers named. Let's delete some of these so you can see them. Let's name this Heiner. And this main character I can shift around, and then when you double-click inside of it, you get all of these pieces of the character, and you'll notice they're all— all the— what's that— the anchor points are all set up in place so these objects could be moved around and animated. You will also notice that this character— and I try to make all the pieces graphics instead of movie clips. Graphics will allow you to stream the inside of a timeline along with the audio inside of it. And if that's confusing at this point, don't worry about it. We'll talk about it more. Just know that I'm going to set up everything as a graphic here. And for the majority of this video, I'm going to focus on the mouth and the eyes of the character. So let's pick apart the anatomy of a mouth here. So I'm just gonna keep double-clicking in until I run into Heiner's mouth, also a graphic. And then within— so I can move this around. And if you pick a frame, you can go to this frame picker here, and I have this all set up so I can change his mouth around by just selecting these frames here. And if I go over here and I select a mouth and— sorry, if I select a mouth, it will automatically create keyframes. And we'll get to that in a little bit. I'm gonna undo that for now, but I just want to give you the sample and the principle of what we're setting up so it makes sense why we're doing what we're doing. So when you click into the mouth, I'm gonna have this chart that I downloaded. I Googled Adobe Animate lip syncing chart. You can do the same thing and you'll find this exact same graphic. If you need it, I can send it to you. And these are the preset mouth shapes that Adobe Animate wants you to create for your characters. So you have neutral, A, D, E, F, and so on. And these are the designs of each mouth that we're gonna set up. And you're going to set each one up on a different frame here. So this is my neutral position, frame 1. Frame 2 is the "ah." Frame 3 is the "d." Frame 4 is the "e." So pretty sure that's straightforward unless I'm confusing you. You're going to build the shapes using the shape tools right where the mouth goes. And then I create a layer on top of that. And within this frame label over here in the properties panel, I like to type neutral. Actually, I believe you have to type neutral, then ah, and type the shape of each mouth in these frame labels because I believe that's how the frame picker knows what frame to pick when you're on the outside of this graphic. So on one or two layers, design the shape of the mouth, and then on one layer above it, we're going to drop the labels. And then on this top layer, I set this layer to a guide, and then I have this mouth chart here, which I'm going to copy because we're going to bring it over to the new character we're going to design for in a second. So that's the anatomy of a mouth. I'm clicking outside of the mouth now, so you're going to see here's one level. And here's the pieces of the head. We have the shapes of the eye, the blinks, which we'll get to later on. And now let's double-click back out. Now we have the head that we're shifting around with the anchor point down here, so we can sort of tilt the head. And that's it for the anatomy of a character. Um, it's really just the concept of nesting and nesting and nesting. So you can animate on every single layer. So if you have like movements of a mouth with inside a layer, that mouth could be stretched and shifted around on the outside of that, on the outside of that nest and so on and so forth. So hopefully that makes sense. And let's dive into designing Bear. Here we have our main sketch. I'm gonna lock this layer so nothing happens on that layer. I can't shift that around. And I'm just gonna create a basic shape to turn Bear's head into a symbol. So I'm going to drag a circle out here. I'm going to create a basic shape of a head. I am going to focus on the mouth at first, but let's get the shape of the head in here. I'm just going to triple-click it. I hit Command+R, and this is going to— I'm going to type in bear's head, make it a graphic, and I'm going to put the registration anchor point down below. Don't worry about anything in the advanced and hit OK. Now I'm going to double-click into here. On top of this, I'm going to make a new layer. This is going to be, uh, Bear's Mouth. You don't even need to name the layers. Uh, what I like to do is name the graphics that you create, and then later on when we distribute those to layers, it will automatically set the layer name to the same as the graphic name. So I'm going to lock Bear's Head layer, and I'm going to make it invisible. Now we can just create the mark for the neutral mouth. I'm gonna use a combination of the line tool and the pencil tool, which is the up arrow and N is the shortcut. But for now, let's get the shape of this neutral mouth in here. I like to move quick, get some straight lines in there, and then shape them later. So now we're going to select all these, make sure there's no artifacts floating around. Bear's mouth interior. Now we are going to double-click in there and we're going to create those layers that we talked about One with the— let's see if we can grab— I got to go back and grab the chart from here. There's a lot of double-clicking around, double-clicking on the character, off the character, and on the character body parts to get into position. So I'm just going to paste in my mouth chart here so I know what it looks like. I'm going to set— I'm going to right-click on this guide or on the layer and set it to guide. I'm going to lock that layer and then I'm going to select a bunch of layers. Shift— I wish I could talk. Shift, select, click, insert blank keyframes. Sorry, undo that. Insert, convert to blank keyframes. Now we can have all separate keyframes here. This is going to be mouth shapes. Top, and then— okay, cool. Bottom. So these are going to be my design layers here. This is going to be my labels here. And you don't need to label your layers. It's a good habit to get into just so you don't get lost. I'm actually going to convert these to blank keyframes. And now let's start building out the mouth. We're going to need to clear these keyframes because I got to be able to see this chart for all the shapes that we're building. So we're going to do an "ah," which is the flat top of the mouth. So let's draw a flat top of the mouth. And usually the top of the mouth, that doesn't change. That position of that top of the mouth is really going to hold its place. It'll wiggle around, but it's not going to shift positions like the bottom of the mouth. If you watch yourself talk in the mirror, your top lip pretty much stays in the same spot, but your jaw is moving the bottom layer. So, and try to make these shapes as simple as possible. People would prefer that things are simple and read well. And have the shape of the sound more so than, you know, the most beautifully detailed mouth. So like, look at South Park designs. They're not perfect, but they get the job done, especially if you're drawing a comedy or like an adult comedy cartoon. Or even like an older kid cartoon or NFT line, whatever. I don't know what you're turning your characters into. These things just need simple mouth shapes. Let's build, let's, let's build all the lines of the mouth because I know when I get into the details, I start to get tripped up and slow down. So I'm going to empty out the interior of this, and I actually haven't used onion skin. I'm just going to select it right there. It looks pretty straightforward, so I can see my mouth. And if you hold the, if you hold the Shift key when you select the shape, it's going to create somewhat of a point that that you can drag around and reshape the shape easier, if that makes sense. Very hard to talk when I'm in creative mode, bear with me. The teeth shape, the shape of the teeth, never going to move. Your teeth don't change shape on the top row. Your bottom row will change shape, or not change shape, but will move around. And actually, by the angle of them, they do change shape. But again, simple is best. There's my D. I need an E. And you can kind of make these mouth shapes goofy, like fit them to what the character is like. You know, do like the side of their— side of the mouth, you know, does it go all the way to the side when they say their E's or their R's? It doesn't have to be like a perfect, super basic mouth shape like you see in this chart here. But you might need to go back and correct it. If you do do that, you might need to go back and correct it. If you push that too far, it won't read as the letter. So don't go overboard with that. That just might be me saying that I hate things that are all boring. In the same This is the F shape. Next up is the L. Yeah, it's sort of the mouth, the top of the mouth is a little bit straighter. So I'm going to drag this shape over here, and then I'm going to hold down Alt right here. I'm going to get this extra point, and now I can reshape a little bit like clay. Maybe we'll drag that up. I'm going to go on this bottom layer and draw in the tongue real quick. Just so I don't forget. I'm gonna have an M. Let's get the R going. Rs are always tough for me. I can never get the perfect shape of the R mouth. See, in this chart it shows that the tongue shows R. Yeah, I guess the tongue does sort of show when you make the R sound, but to me it looks so much like other shapes. I'm just going to throw the tongue in there like that. That sounded terrible. Let's make the S. These are relatively quick and easy. The "uh." Okay, let's go circle and Alt-drag these out. It doesn't matter what color these are when you first set them up. You can always— let's see. Uh. I feel like the bottom teeth showing up more than the top teeth. I'm going to throw the top teeth in here, but let me just quickly color this so I know what I'm doing. I will leave it for now. And then we have a woo. And you can actually create as many mouth shapes as you want after this, but this chart is only going to pick up these 12. There we go. Now we have our mouth, and when we double-click into it, I'll turn off this chart, and I will turn off onion skinning. We have our mouth moving. Now, within this layer, within this, within this symbol, within this— what am I trying to say here? When you're nested this deep in the symbol, you're not going to have your audio on this timeline here. You're going to double-click out here. You're going to extend this timeline by inserting frames. Then inserting a layer, and then you'll have audio on this track right here. And then you can auto lip sync your mouth here, or you can keyframe these in here by selecting your keyframe shapes. Let's go in and color the mouth. You might need to have two layers set up for this. You want the fill tool And you want to select close medium gaps. Let's do all the white colors first, which are the teeth. And not liking that. Don't like that, uh, close medium gaps. Maybe don't close gaps at all. Then that won't even work. Let's go to close small gaps. And that doesn't work. All right. So we're just going to fill in the whole mouth, grab the eraser, and I'm messing up. All right, let's, let's go to the brush. And I'm going to change the color of the stage too, so I can sort of see through. Let's turn off our sketch here. Create one more additional layer just in case we need it. Convert to, convert to blank keyframes. There we go. Grab that brush. Let's see what size that is. That's pretty big. So She— let's scale this down a little bit. I can't believe I'm not using the fill tool for this. Go through and do all the teeth and I'll meet you on the other side. Now that we have Bear's mouth designed, let's just break it down a little bit because I sped through it, didn't show you all the details. In the frame picker, we have all of his mouth shapes. Interestingly enough, not labeled with the actual names here. Let me double-click in here. I did not add the frame labels. Let me just double-check that if I add that in, it will show up. 1 neutral, 2 Yeah, so it still works, I guess, without even naming, and you can add extra layers here or extra frames here. I'd stick with these 12 just to keep things simple at first. Let's double-click inside the mouth here, and you can see how it's all set up. Sort of sloppy. I mean, the exterior, the top layer is, are the black lines and the shadows. I actually added this shadow This is a brush stroke at about 20% opacity, and I put it over the teeth. Just adds a little bit of depth. And you can see all of our mouth shapes here. And I'm going to double-click out. Now we're inside Bear's head. I'm going to double-click out here, and now we're on the main timeline where we can move Bear's head around and and the mouth and everything else moves with it. Before we build Bear's eyes, let's dive into how Heiner's eyes are built so you can see finalized product. What am I talking about, product? Uh, all right, so we have the pupil here, we have the eye white here, and then on top of that We have a layer that you cannot see, which is called an eye blink. I hear an alarm going off. I have to go stop that. Be right back. Let's start designing this eye so we can, uh, not waste too much time here. All right, I'm going to double-click out of Heiner. I'm going to go into Bear. I'm going to create a new layer here, grab my circle tool, make sure there's a black stroke, the white fill, and then drag it over the eye shape here. And then fit this as best as you can to the eye shape. I'm going to go to Modify Shape, Smooth, and you can hit this a few times. You can even create a keyboard shortcut. See, I'm not liking the shape of the eye at the top here. Sometimes it's hard to get it perfectly right. That'll be good for now. So that's our eye white. We're going to Command+R or Control+R, name it bear eye white right. Could duplicate it for the left, but I'm going to, I'm going to do each eye separately. Just, it gives a little, it makes the character a little bit more distinct if you don't, you know, computerize everything. So I'm going to create a new layer, make this a pupil by Command+R, Control+R. Pupils you can duplicate. So, and for the pupils, I like to offset them in, find the midpoint of the eye and just offset them in a little bit. It just makes your character seem a little bit goofier. So if you Alt-drag, that will duplicate your Shape, and I'm just going to size these down a little bit. I have a tendency to make the, the character shapes a little bit too big, so I'm just going to size that down. And now we need to set up the eye blink. So what I'm going to do is just turn off this layer for now, create a new layer on top of it. Make sure it has the stroke. For some reason it won't let me select it. Very interesting. There we go. Grab my circle tool. And I'm gonna pause this here because I can hear my kids about to walk in the house like wild animals. Be right back. Where was I? We're doing the blanks, right? All right, so I have my line drawn. Create this separate line here. Bend it to the eyelids. And this is going to be like the eye at rest. And I actually like the shape of this eye a lot better than what I had before. So I'm actually going to copy this, go here, turn it on, just Now we have a nice smooth eye shape here. I'm liking this. I'm going to do the eye color a little bit of an off-white. Pupil, then we have our eye blinks. Unlock that. I'm going to make his eyelids like a grayish purple. That may not look great. Let's give it a shot. That's okay. So that's our resting pose. We're going to triple-click the eye, make sure the eyelids are all selected. I'm doing a Shift-click to select it all so you can drag it around. You can see the interior of the eye is not filled. And then I'm going to Command+R, Control+R, and call this bear eye blink right. And now it's a symbol. It's not going to break apart here. I'm going to double-click in here and I'm going to create a new layer on top, convert to blank keyframes. I have 3 frames in here of my resting pose. I'm gonna have a pose where the eyes just go super wide like that. And then I'm gonna have a blink pose. So copy this and paste it on the third frame. And then just select this line, select this line. If there are any artifacts like this, just select them and delete them. I'm gonna eye sample. You type the letter I and then sample the color. Fill it with the fill tool, and then the blink. Let's just double-check that's a good spot. That's a good spot for the blink. So we have— and I'm going to move my blink over one frame. So we have a resting, blink. Do we need one more? We have a wide open. Let's do one that's a little bit like a squinted eye. I don't know if— I guess there will be times when he's like this, maybe when he's like dreaming about foods he likes and stuff. So there we go, this eye's— now I'm gonna, as I scrub, you're gonna see this eye goes crazy on the timeline. That's because it is set to loop. You don't want to, you could play it once. It's still gonna do that at the beginning of the frames, then it's gonna stop. What I do is set it to this little symbol here, play single keyframe for graphic. Then we have our eye here. And I'm gonna go through and I'm going to, actually, you know what? I might break my rule here. Let's see if we can speed things up. We'll just copy and paste this, and then we're going to flip it because I do think I made the shape of the eyes pretty nice here. And if this matches, yeah, I don't think there's any reason to change the size of it. You can squeeze it horizontally because it's a little bit, it's a little bit of a 3/4 pose here. And then I'm just going to grab that eye white underneath and flip it and send it over here, scale it. Yeah, it looks pretty good. So now we have blinking eyes. One thing you can do, like I said before, I didn't, I didn't label any of these in the layers because you can Select them all, right-click on it, and go to Distribute to Layers. And you'll see it distributes all the symbols to layers and it names them for you. Except for, I don't know what this is. That's the eye white. Maybe I didn't have that named. Always save your file as you go. So there we have bear's face. And this could be animated like this, or you could double-click in here and you can animate the blinks and you can animate the mouth movement. I'm going to create a separate layer for an audio track in here, and I'll show you a little bit of lip syncing. First, I'm going to go through and I'm going to design the rest of this character exactly the way I did the mouth and the eyes, and then We'll meet back up here and I'll show you a little bit of auto lip syncing. Bear's all designed. We're on the outside, the outermost level of these nested character pieces, and you'll see that I have Bear over to the right on a different layer. I have Heiner here, and then I'm gonna have an audio track on top of that, and I've stretched out the timeline by inserting frames. Now this timeline is going to match the length of my audio track that I'm about to bring in here in a second. On the outermost level of these nested characters, you want the character— or you want the graphic to play once. So in your properties, select this Play Graphic in Loop or Play Once. It's going to be the same thing because it's only going to loop when you get to the very end of the track, and that's not going to matter because you're going to be compositing these characters in After Effects or exporting these directly from Animate. So let's animate Bear's mouth with some auto animation. I'm going to double-click into his character and I'm going to set up his interior to look just like the outside. I'm going to insert frames. Keep doing that. Just make sure his eyebrows are in different layers here. They're not. Then I'm going to add an audio track here. And I'm going to double-click one more time into the mouth. Actually, this is where the frames are, so we don't need to go in that deep. I like to make the frame or the layer look a little bit different. Than everything else. So I'm just gonna change the color of the layer to black and then the layer height to 200%. So I can see that I'm animating this. And you can even drop the audio track— two audio tracks here— you can even drop the audio track right over whatever you're animating. I like to do that just because it lets me see it quickly. And if you select Alt and then the lock, it is going to lock every other layer other than your animated pieces or the layers that you want to affect. So let's bring in the audio track. I'm going to import to library.

5 years ago, 2 Bills tailgaters, Heiner and Bear—

So you can either import WAV files or MP3 files. I find that back in the day, it was all— I think it could only work with one or the other. I might be making that up, but you're just going to drag the audio out onto the file, select the audio here, and then make sure that it is on stream, set to stream. So when you scrub, you can hear it. So I'm going to animate Bear's mouth not to his actual mouth, but actually maybe we can find a section where he talks. Ben Johnson.

Bear, Bills are 13.

Who do you think is better than Johnson?

Bear, Bills are 13-3.

They're in the second round of the playoffs.

We can go to the Super Bowl.

Oh, is Jim Kelly our quarterback? All right, so I'll select right here and I'm going to open up this frame picker here. So I could either listen to the audio and keyframe each one of these and usually do it on twos or threes. So I'm not listening to it for this video. I'm just going to do this really quickly. This is more of a manual process. So I'll zoom in here so you can see it. So that's the beginning. That'd be a manual way. I'm going to clear these keyframes and I'm going to go to Window. Where is it? Auto Keyframe. We got to do Auto Keyframe All right, so select your object, select lip syncing, and now you're gonna— it's, it's gonna show all these shapes need to be set up. You're gonna need to click on these and match them to what you have in here. And then for number 2, sync with audio and layer. Make sure that the audio is selected, and this is named audio because I named my layer audio here. I'm just gonna hit done. I think it's gonna auto lip sync this whole track even though different characters are talking, but it'll show you the speed at which it does this and how accurate it is. So let's run this. Taking a while. Taking a little bit longer than I expected. I'd say the accuracy that this auto-lip syncs is 60%. I think you need to go in and you need to fix a lot of the mouth shapes to get them perfect, but this is still a huge time saver. And this is taking a lot longer than other characters I've done. So I figured out the reason why it was taking so long to animate that section. The audio track was like 11 minutes long. It only needed to be a minute long. So I'm reimporting now. I'm going to select my shape, hit lip syncing. I have all my shapes set up. I have selected on audio and hit done, and now it's moving much faster. So if you accidentally bring in a long audio track, just trim it down to a shorter section or whatever you need. So now the moment we reveal how accurate or inaccurate this actually is.

5 years ago, 2 Bills tailgaters, Heiner and Bear, froze themselves so they wouldn't have to deal with the pain of Bills football seasons anymore. Then someone accidentally left a chicken fryer in the Bills parking lot, and those 2 unsuspecting blocks of ice began to melt. Oh my What year is it?

Rex Ryan still our football coach? I actually think that's really accurate. You can go in and tweak things. It's probably 80% there. If you're making web cartoons or animated NFT characters or animated any like very simple adult cartoons, this mouth shape is going to be ideal. And the auto syncing in Adobe Animate is super quick. And you can just go in and you can stretch things out. I can add a keyframe here and maybe, you know, for a second it stretches out. So it's just not a regular L shape, it's L, you know, you can exaggerate the mouth shapes there. And let's zoom out a little bit, see how this looks. I'm gonna, I'm actually gonna jump out to the outermost skin of the nest and drag in my audio track. So you're going to want an audio track on the outermost layer and then one on this interior layer. And you usually won't have to go deeper than this, just two audio tracks, because if you click in here, you're going to have all your mouth shapes. So go here. One thing I can do is just quickly show you how I would blink the eyes. So what I do is I let the audio track run and I feel whenever I get the sensation of needing to blink, I stop there and I hit Command+E, a shortcut I set up for adding keyframes. So we're going to hear a lot of audio here. I'm just going to hit run. So I feel a blink right here. So I'm going to set up these keyframes for 3 keyframes. Not on the eyebrows, of course. I would never do that. Select the eye blink shapes. Select these, go to properties. I'm going to set this to 2. So now we have our blink for 3 frames. It's going to hold that and then it's going to open up. And then anytime you feel a blink, you can just copy these frames and paste them in. So now you have You have your blinks. So let's jump out and I will show you what this looks like.

5 years ago, two Bills tailgaters, Heiner and Bear, froze themselves.

It's looking pretty good. I'm just going to position my characters here. I actually think Adobe Animate improved the accuracy of their auto lip-syncing. This is Much better. And I'm just gonna make some perfectionist adjustments here. There we go. Let it rip.

So they wouldn't have to deal with the pain of Bills football seasons anymore. Then someone accidentally left a chicken fryer in the Bills parking lot and those two unsuspecting blocks of ice began to melt. Oh my back, what year is it?

Wow, I'm actually impressed with how accurate that these are. Much better than I expected. In the years past, it was pretty off and I had to go in and really correct it. But yeah, that's how you animate and auto lip sync mouths and blink eyes in Adobe Animate and build your characters. And that's going to wrap up this tutorial. If you have questions, just