Create Amazing Videos With AI (Deforum Deep-Dive)

Hey.

Hey, we're live. This is my first live Stream as an AI YouTuber so this is going to be real interesting. Hopefully it's not total amateur hour, but thanks everybody for hanging out. We've got about 55 people watching and I think a handful more are going to be jumping in. And I've got some awesome guests joining us on this live stream today. I've got Revolved from Run to Fusion, which we'll be talking about what Run Diffusion is. It's basically a cloud version of Stable diffusion where you could run stable diffusion in the cloud. We got Human joining us, one of the original creators of Deforum joining us on this call today. We're going to try to create a full on masterclass in this all around Deforum. If you don't know what Deforum is, or you do know what it is, you don't know how to use it yet, or you have any questions lingering about Deform after you've spent some time playing around with it. Ideally, we're going to cover all of that in this call today. So it should be really fun, really interesting, really informative. Have no idea how long we're going to end up going. We're just going to go until.

We'Ve answered questions and everybody's got a good idea of what you can do with the forum here. I'm not going to ramble for too long. I just want to get straight into the content, get into what everybody came here for. And I'm going to introduce you to our first and main guest of the show today. I'm going to bring on Revolved here. He is.

One of the guys over at Run Diffusion. He is the guy who, he's like their expert in Deforum. Him and I have had a handful of conversations. He knows what he's talking about. He's got some stuff lined up that he's going to share with you today. So I'm excited to bring him on. So here is Revolved. Thanks so much for joining us. Excited to get into this. Before we get in, do you want to just real quickly let us know a little bit about Run diffusion and what your role there is there and go ahead and roll into a little bit of what we're going to talk about today.

Yeah, awesome. I've been with rundifusion for a little bit now. Run Diffusion came out. Wow, like October. So the AI space moves quick, right? The solution for rundifusion is basically we took the idea of making it as easy as possible for people to get into stable diffusion. You know, automatic 1111 is like a fantastic tool, but it's very complicated. There's a lot of depth to it. And what we really wanted to do is just take all that part about like installing it and configuring it, make it easy to use, pop it up instantly, as fast as possible, so you can just get right to creating art.

Yeah, and my role there. So I was actually just started in the discord, helping people out. And they brought me on as I have a background in software.

Like in particular around revenue and sales. But I'm also a video artist and an artist myself. So you can see in the background over. Oh, wait here.

I've got like a video synth running and this up here, this poster. It's a video art conference that I used to hold in the States. So, yeah, video art is in my blood. Whether that's analog, pointing a camcorder at a TV and watching that infinite hall of Mirrors feedback thing happen, or more complicated stuff with the hardware synths, I just love it. I love vjing. I love art and cinematic stuff too. So as soon as I saw that AI video was coming in, I was very, very pumped. Especially when I started seeing the Deforum videos. I was blown away what people were able to make with it and knew I had to be doing that. So, yeah, when I saw Rendezvous, I don't have a very good gpu. I have like a amd. Like, man, like, it must be like four gig or something like that.

Yeah, well, it seems like stable diffusion. They mostly want Nvidia cards anyway. Right. You kind. It seems like Even the good AMDs are having problems with it if you don't have Nvidia.

Yeah, there's ways around, but it's like a lot of extra steps to get AMD cards working and they're just a little slower to the draw on drivers and stuff like that. Nvidia's definitely been cutting edge, even in streaming tech for a long time. So they've got the edge for now, for sure.

Yeah, for sure. Yeah. Well, very exciting. So I know what we. You and I have had a few discussions about where we wanted to kind of take this, and we're going to get into the details and all of, you know, showing some stuff off. One thing I wanted to do real quick, just, you know, before we get too deep into it, is for anybody that doesn't know exactly what Deforum is. I did actually upload a couple videos in here to kind of show off some of the stuff that I made into Forum. One of them I made on camera, on one of my tutorials. Tutorial videos. One of them I made off camera. So this was the first video I made right here that you can see on the screen. And I put it up on Twitter and it kind of went viral on Twitter. And this was just me saying, you know, turn a monkey into an astronaut.

Awesome.

That was the video that came out of it. So that's the type of stuff deformed is. This is another one that I made that starts with a wolf, because, you know, my last name's a wolf. And then it kind of morphs into a lion here and then it morphs into a tiger, and then it's supposed to morph into a.

Shark that needs a dentist.

Yeah, Anybody who doesn't know it deform is it makes those kinds of videos. If you're in the AI space right now and you're on Twitter, you've probably seen that kind of stuff circulating a lot of those types of videos. And so that's the forum that's doing a lot of that. And a lot of the tools that are out there that you see that make those kinds of videos, they're using Deforum underneath the hood as well. So, you know, you're seeing a lot of deform floating around out there in the AI video space. But, you know, what we wanted to get into was a little bit of the mindset around this and how to. How to approach the forum, kind of set your expectations of what it can do and what it can't do and what you should expect from it. So maybe that's a good place to start and let's just jump into the deep end here.

Yeah, I think this is a really important piece of just AI art in general is the space is so new. There's just like new stuff coming out every day. There's advances, new services.

And just people who are contributing to this open source community, which is just incredible to watch this stuff kind of come together in real time and it's beautiful, but there has to be like a certain way of approaching things because it's all so new. So an example of this is like, you know, if you're going into things and wanting to achieve a certain goal, you know, you see a video online, you're like, okay, I want to do that exact thing. It's really good to just like kind of try to dive into the nuts and bolts. I mean, I know like when prompting and images came along, you know, someone would post a cool picture and then everyone would be in the comments below. It like, what's the prompt? What's the prompt? What's the prompt? I want to try the prompt, you know, and it's like, kind of funny. I don't think there's much hand holding in the air space. And we're lucky that it's just mostly because it's moving fast. And the kind of stuff that we're doing, I've seen in the communities, like, Matt, what you're doing with these videos, YouTube, like, people who are doing their own documentation, sharing on Discord, like, those are all amazing resources to help figure things out. But a lot of the stuff is so new that there's just not much out there for it. And so the attitude that I've taken as an artist going into it is like, okay, I need to explore this, be willing to make mistakes and try as much stuff as possible. Which is a really, really important attitude to have, especially when you're working with deforum. We'll go into, like, some ways that will help you from wasting time. But I think that it's just so important to have that, like, experimental mindset and that curiosity, because the curiosity is really key for doing that.

So that's the attitude I've tried to take, is to just not, like, you know, look, to copy things one for one, but to learn as much as I can. Now, that being said, there's some incredible settings that people have found out there, and a lot of times that will be what gets you started. That's the training wheels. And once you have the training wheels, tweaking things a little bit is going to help you get that last step. But there's. There's little breadcrumbs everywhere on, like, you know, Twitter, Discord, Reddit, YouTube, they're all. They're all kind of out there. And then you kind of piece it together into this picture anyways.

Absolutely. No, it's funny because that's always my first thought whenever I'm coming on, whenever I get on Twitter and I see one of these videos is I want to know what the prompts are. Not necessarily because I want to make the exact same video, but one of the tricks when it comes to prompting is all of the additional keywords that you can add to the end of a prompt to get the images to look a certain way. It's never really necessary. I want to recreate the same exact image image. But I want to know, you know, how did you get that look to it? How did you get that level of contrast to it? How did you get, you know, that sort of Colorization or style to it. And I feel like that's one of the big reasons everybody's always asking for those prompts. And so that might be another sort of rabbit hole we can, we can go down a little bit once we actually start digging into Deform a little bit is some of the additional keywords and little tricks and things that you can add to the prompts to, to get those, like there's, you know, those really good looking images out of your videos.

Yeah, I do feel like there's a couple metaphors I would use. Like one is like when you're making AI art, you are exploring and you can really find things that people have not seen before while exploring this space. Which I, I think is incredible because as an artist, like, you know, there's so much art that's been done, then there's so much like that has already taken place. But this AI art is a totally new field and there's so many new ways that you can explore things and using it in different contexts and mediums as well. Which is, which is a really interesting piece, but going back to the prompt piece of it because, you know, we're all kind of looking for that gold, right? And I think that there's sort of a, almost like an alchemy. It's not like a science. It's like, it's like the old, like Middle ages, like pouring sulfur and like strange liquids together and not quite knowing why the result you get is the result. Like it's, it's a fun sort of thing to, to be a part of this sort of alchemy. And on the one hand you might like feel sometimes while doing this like, oh, I'm just going to, I'm just like banging rocks together, hoping to make fire. Right. Like we're, we sometimes we just don't know what we're doing with the AI tools. But by slowly trying to like understand the fundamentals of it, you can really accelerate your progress in the field and make some incredible stuff.

Yeah, absolutely. So, you know, again, this whole thing, this is, this is new. I don't know how the whole flow of this thing is going to go, but do we want to, do we want to explain what run diffusion is? So the videos that I put on YouTube, it started off with me showing you how to locally install automatic 1111 and stable diffusion. And then deform is sort of an add on that you can add into automatic 1111 and it sort of starts to get convoluted once you have all of these layers of what you need to do, right, you need to have Python installed, you need to have like your, you know, your git bash installed and then you need to get into automatic 1111 and then you got to add this, you know, the forum plugin on top of it and there's all of these sort of layers of complexity to get to where you can finally start generating with the forum, right? And I feel like that's where run diffusion comes in. And this isn't meant to be like a webinar to pitch run to fusion or anything, but quite honestly, with something like run diffusion you can just sort of jump on the cloud, right? If, if you have an AMD processor that doesn't really like to process stuff with stable diffusion or if you're on a Mac, Macs don't seem to really like working with stable diffusion very well. If you're on an older computer, you can't really do this stuff super easily. Or even if you have a decent graphics card, sometimes it just takes forever to process it because.

Maybe it's an older graphics card, maybe it's an older Nvidia card. And that's where I feel like something like run diffusion where you can just go do this in the cloud and, and depending on, you know, time versus money, you can, you can spend 50 cents an hour and go and run this stuff in the cloud. Or if, you know, you just want it really, really quick, you can pay a little bit more, use a little bit higher ed GPU and get the stuff done a little bit faster. And that's where I feel like run diffusion really comes in. So maybe let's talk really quickly about what run diffusion is and some of the benefits of it. I feel like I gave some of the benefits, but you might have even more context because you know, you're working at run a fusion every day.

Yeah, yeah, maybe we could, maybe I can share my screen here. Let's see. And then we can like kind of get that screen share started.

Here we go. I love, there's a comment I just loved here. Like we are like Admech tech priests from Warhammer 40,000 as a 40K player. I appreciate that back in the day. Yeah, I really feel like, like that's, that's kind of what's going on. If you don't know, they like worship the machine. You know, they're like their religion is the machine. The AI stuff. We just like put in our incantations of these like random prompts and out comes something incredible. You know, it is, it is kind of the way things are going.

Absolutely.

Yeah. Okay, so we got the screen up, so I'm logged in already to run diffusion and I'll just kind of talk things through. We have like a few different things here, so automatic and invoke invokes, like an inpainting out painting thing. Automatic is kind of what everyone's using, especially for Deforum. That's the only place you'll get Deforum on here. And then you can choose the model. And I'm on our creators club, which gives you private storage, so you can load in custom models, embeddings, all of that, and then they'll persist from one session to another, which can be really helpful in getting things going. You know, I think that like what you were saying, Matt, about like the GP having a gpu, maybe you want to do things faster. You know, this is something where you can use these while you're using your local gpu. So people will do things like train on their local and then take the models that they're working on, pop it into here and start seeing what the previews are like, messing with it. You can do the same thing with deformed videos.

So the idea that we have here is let's get one going here. I'll just launch one. So we have a timer that lets you know, we'll basically shut it off after a certain amount of time, so you're not wasting money.

There's an API as well. So we'll just launch this. It takes a little bit to get this set up, but it's quite fast, especially if you've tried doing this in the cloud before. I think that.

In regards to where we sit, we're really trying to just provide the fastest and easiest to use service. That's our goal and as well to provide an extra level of support for things. So although we don't know the answer to everything in automatic 1111, because it's changing every day and it's complicated, we're doing things like trying to make sure that it's the most stable version of automatic 1111, so we don't have people running into errors and we support them with like any issues that they're having. So, you know that support and training and discord help is included in that. And.

You know, we want to make sure that we're providing kind of the best and most stable diffusion you can get. Stable, stable diffusion, yeah.

Yeah. So I. I see Dave in the comments saying that, you know, he noticed he jumped on the $2.50 one. They do have a 50 cent version. But because we're live, we want to keep the pace of the generations going. So we're definitely going to use the fastest version we can, just for the sake of being live. But, yeah, they do definitely have a 50 cents an hour version. And when I was demonstrating it in some of my videos on YouTube, that was the version I was using.

So, yeah, when I was doing Deforum, like on my own, before I even joined. There we go. We're already up here.

So when I was. What happened here?

Diffuse Master. This is a good one. It's always live, you know, I was just.

I was just using this. I've never had that issue before. All right, so we'll try it again. We don't charge for starting it up or spinning it down, so no big issue there.

Yeah. And that is one thing I noticed too, is when you're first signing up to and you're logging in, it says, how much time do you want to use? And you can set it as an hour. That's not meant to be like you're going to pay for an hour. That's more of like a safety net. If you walk away from your computer and forget about it, it'll turn it off after an hour. So it doesn't keep spending. But if you're only in there for 10 minutes and then log out, it only actually spends 10 minutes worth of spend.

Yeah. And this is in the chat, you see, Ren Diffusion is responding. That's not me, by the way. It's the. One of the founders. So he said there's a ton of people. Yeah. Spinning it up right now. I should have maybe prepped a few of them and kept them running in the background, warming up, you know. But when I was doing Deform, I was doing it on the medium setting. That's before I actually joined Run Diffusion, because I felt that that hit the best balance of speed, which I need for deforum. You can't use the slowest one.

And we'll show you. I'll show some of those as well. I'll spin up another instance. So you can see that there is the ability to spin up multiple instances at once. But I felt that that had the best balance. I think what I use Large for is like, I've got a set idea in mind. I've kind of played around with it. I understand what prompts I want to use. I've, like, fully dived into the settings that I want to use. And then when I hit large, I can just run and just like hit it to Render, render, render, render. So I'm not like having a large up spending the larger amount of money and you know, wasting time diddling around with settings.

Absolutely. So just in terms of like the speed difference, you know, do you have an idea of like, so you've kind of got a small, medium and large like server that you can use. What is the, do you have any sort of like benchmarks or sort of speed differences of like what the expectation when running like a Deforum would be between the three, like how long it takes, you know, to run like to generate a video? I guess there's too many factors to really answer.

There's a lot of factors. I got asked this today too. So there's a bunch of settings that will affect it. I would say that the main one is resolution. So if you're trying to make a higher resolution video, you're going to want a bigger box. And that's again where you figure it out on a smaller resolution you kind of understand what you want, you scale it up, you get something a little bit larger and then of course you can do the upscaling and so on after as well. You don't need to do the biggest one from the get go just like normal. So that is one of the things. The other thing is like, just like when you're generating an image, you know, there's how many steps you're using and in Deforum there's some other settings as well that will affect that. But then also how much stuff are you piling into it? You know, there's like hybrid video, there's 3D camera controls, there's all kinds of stuff. And so that complexity can really drive the, you know, the usage that you need.

Yeah. And you know, when I was, when I was demoing it on a YouTube video I was showing, you know, if you want to figure out, if you want to learn the various like rotations, like the clockwise rotation or the zoom in and the zoom out, that kind of stuff just generate like a, you know, 12, 15 frame video because then you can generate real quick just to kind of get the example of what it's going to look like. But then if you're trying to generate something that's 500 frames, that's going to take a hell of a lot longer than something that you're just generating, you know, 12 or 15 frames on. So yeah, makes a lot of sense. It was as I was asking the question about comparing the three, I realized the, the answer wasn't going to be so straightforward. So.

Yeah, there's there's a ton that goes into like determining kind of the speed of things. But you know, there's also so many awesome tools available. So. Yeah, just like recently we've been test there. The Deforum has been testing like ControlNet getting involved. There's hybrid video, which is awesome using like you can load your own videos into things. So there's a lot of interesting stuff.

Yeah, I see a lot of good questions. Oh man. People stole all the large servers. That's what's going on. Apparently. Apparently I need to use a medium. Okay. Okay.

That'S funny. Well, we're gonna have to reserve a few more of those bad boys. Let's see what we got here. We'll go for medium and that'll make it easier to, to see what a medium is really like here as well.

Cool. So when you first jump, I see people are already jumping in and making deform videos when you first jump in. It's very intimidating. There are a ton of settings. And this is like if you stepped into automatic and you were like, holy crap, there's a lot going on. You know, be warned that in deforum there's a lot going on as well. And it's not as immediate to get kind of the instant result you may want. And that's what I'm talking about. That experimentation, the curiosity.

I think we're getting scaled up in the background on servers, so we'll get that rolling. But yeah, I just know that when I first stepped in, like the first few videos I got were just like garbage. And sometimes like a blank, you'd get blank frames. You'd get like crushed, deep fried images. You know, you'd get like blurry images.

Yeah, there's, there's a lot going on. But yeah, like there's a comment in here. We are light years ahead of where we were six months ago. And just in the industry in general, we're light years ahead in the industry.

What people are doing now is incredible. Yeah. You mentioned disco diffusion, like the precursor to deform. Right. People are doing video in that. So it's just amazing to see how far things have come. And now that like hybrid video control Net.

Even the tools Runway is brought out, there's a lot more viability to using this in like even a professional sense video in that. So it's just amazing to see how far things have come. And now that like hybrid video control Net.

Even the tools Runway is brought out, there's a lot more viability to using this in like even a professional sense.

Okay. I'm up now, Matt.

There we go.

All right. Medium works. This is a medium.

Awesome. One thing while you're getting set there is you guys are giving away some credits to run to Fusion, so might as well give some of those credits away. I do have a little contest bot. I can't tell if it's working yet or not, so in order to test it, why don't you put the word deforum into the chat, and then I'll try to randomly pick somebody that types the word deforum and see if I can get this bot working to do the competition here.

All right, There we go. Spamming. Deep Love's asking if anyone used Deep Dream and gfp Gone. Yes. Back in the day, Deep Dream was the one that, like, I think everyone turned things into dogs, remember? Like, that was a good one.

It was like acid dogs. Like psychedelic acid dogs. Yeah. All right. So many people chiming in. What one do you want to give away, Matt?

That's a good question. Let's go for one of the big ones. We're giving away two $50 ones, and let's do one of the $50 ones.

There's a question from Dave. What price is the medium? It is 99 cents an hour.

All right, my little random picker isn't working here, so that's annoying. So I'm going to go ahead and just kind of throw a dart. I'm gonna kind of blindfold and point at the screen and scroll up and down real quick and then just go. All right, Dewaz Lafoon, that's the winner.

All right, so you won $50.

Nice one. That's. That's a lot of time. That's a lot of hours.

Yeah. So there you go. That's the. That's the first $50 giveaway there. I'm gonna go ahead and cross it off my list to let us know that we gave it away and to make a note of who gets it. So, Dawaz lafoon. All right, let's go ahead and start exploring some deforum here.

Yeah, so I'm just logged in. This on the side is our file browser. So you can see all your directories in here.

This is the standard automatic 1111 with this top bar added. So what I will actually do here is we can get a multiple session going. So let's launch another session.

And we'll just do this exact same thing. Medium, 99 cents an hour, 1.5 select, and we'll get it going.

So now while that one's spinning up, we can see it in the session here, and it lets you, you know, keep an eye on the timer, make sure you're not wasting anything by accidentally closing a tab and realizing the server's still up or anything. You'll see it here on the side.

Okay, so we're in the Deform tab. And, you know, I think first thing I really wanted to do, and I'm just gonna pop up my notes here was kind of go over like an example of using 2D camera controls. Just get something basic started. So, you know, we can keep a lot of these the same. There's a lot of different samplers in here. And for deforum, I actually particularly like Euler A just because it's a little bit of the wilder ones. I will say that, you know, what settings work for me may not work for you. And a lot of it comes down to personal taste. And that's where that curiosity, trying things out can really help because you'll find what works for you, not just what works for other folks.

So a lot of the settings are in this keyframes tab here. And I cleaned it up. 2D camera controls, just get something basic started. So, you know, we can keep a lot of these the same. There's a lot of different samplers in here. And for Deforum, I actually particularly like Euler A just because it's a little bit of the wilder ones. I will say that, you know, what settings work for me may not work for you, and a lot of it comes down to personal taste. And that's where that curiosity. Recently they did some fantastic work on organizing the ui. There's a few different modes here. So the first ones that I want to talk about are. I think a lot of people have seen some of the. More like rapidly like hyper growth, almost like flashy, flickery type of deform videos. And you can slow that down. So when I started, like a lobby, trying things out can really help because you'll find what works for you, not just what works for other folks.

So a lot of the settings are in this keyframes tab here. And they cleaned it up recently. They did some fantastic work on organizing the ui. There's a few different modes here. So the first ones that I want to talk about are I think a lot of people have seen out of what I was doing was around trying to slow things down and make it like a little bit.

A little bit like slower and more my speed, because that's my personal taste, right? I wanted something that was a little bit like smoother. So what I like to do is change this deforum cadence. Now there's a long explanation there, but the basics of it is if you have cadence one, it will basically do a diffusion on every frame. And if I do two, it'll be every other frame and so on to five. So here we're going to get much less of that hypergrowth and every five frames it'll start to change.

So we'll just rock 120 frames here we've got our steps. Strength is I would say, the next most important.

And this is because this is the cohesion between the previous frame and the next frame. And it will really help determine like how wild you get. It starts at 0.65 here, which is pretty good. I want to keep it again a little bit smoother. I'm going to put IT up to 0.75, but in some cases you might want to lower it depends how wild and crazy you really want to get.

Going down to motion here. So there's a number of different things and these are all very important. So this is for the 2D camera. Rotate clockwise or counterclockwise. You know, counterclockwise would be negative number here. We're just gonna leave that as is. Now this is where you kind of look at deform and go, what the heck is going on? So with this zoom, what you really can look at this as the mathematical equation of a wave. Now I am not a mathematician. My, you know, algebra teachers would be ashamed that I might not know what this is without pulling out some fancy calculator graphing Good old TI83. But what I'm gonna do is just remove that and replace it with a one and we'll show you some different tools that, that can auto populate that. I'm sure someone in the chat is already like ChatGPT. I'm just going to pop that in there and it'll tell me what it does, which is probably a good bet. ChatGPT and those other AI chat tools are really good for figuring this stuff out.

So the important thing to realize, you see there's this zero here. And the important thing to realize is like that is what frame that this takes effect on. And if you leave it at zero, it just means the whole time throughout the video we're going to be at Zoom 1. Zoom is this one setting that's very important to note that you cannot put zero on it. That will not stop it. Zero just basically, I don't know, it goes into like nothingness, infinity. It just like messes with it. You'll get an error. So keep it at one, which is essentially standstill zoom one not moving. Just keep that in mind and make it down to one for zoom if you want to keep things flat without a zoom. Now we can actually add in maybe some keyframes here. So here we've got on frame 20, you know, we're going to zoom in a little bit. Frame 50, we're going to pull out a little bit. Frame 100 will just stay back and still. And what it will do is it will smoothly move over the frames and blend that or tween. I don't. There's a technical term for it, but it will slowly move from 1 to 1.005 as the frames advance from frame 0 to frame 20. So that's just a really good idea to keep in mind that that's how it will smoothly move through.

Now, X, you know, this is the left right, as it says here. So we're going to go from negative 1.1 down to zero again. So it's going to move and then stop and then move again. And we'll do something similar with translate Y down, of course.

All right, now going on to noise. Noise is actually incredibly important. And the reason why in Deforum it's important is if you think about it in one image, you know, noise is not a big deal. You know, you can kind of mess with it, but it's necessary for making a diffusion. But when you're dealing with a video and you're using the strength to have coherence from one frame to another, what that means is that noise can feed back on itself and quickly get out of control. And you see this a lot where like, people are like, what happened? Like, my video started tiling and getting squished and the colors got all blown out and like very. I lost everything. A lot of times it might be because of the noise is too high and the strength is too high and they're combining together to just crush everything into noise. So for me personally, I like to keep a lower noise, not introduce as much. But if you're doing a more chaotic, more wilder one, then you can crank the noise up a little bit, try it out, find out what you like. You know, become a noise artist if you want. I'm going to try to minimize it as much as possible in this one.

So for coherence, this is the color settings. We'll go over some of them as we go through, like different modes and then there's an anti blur as well. I'm just going to leave those all blank. Config, you may already be familiar with from text to image or image to image. You know, lower values, more creative, higher values closer to the prompt. So we'll leave that as is. Well, actually, maybe we'll pop it up to nine or something like that. Okay, now, there's a bunch of other settings here which you can use to determine the different ways that it uses seeds, for example. So I'm going to keep it on iterative, which means that the seed will go up by one for each frame of animation.

You can, if you want, go back in and change it to a fixed seed. So you could even schedule that by a keyframe, which is awesome. Like, I love that they thought of this. My seed is, you know, random, so I'm not too worried about it. It's just going to pick a random spot and iterate from there.

All right, so let's put our prompt in. Now, the key thing about this, it's very important. Notes. I always read these. Very, very important.

So with these prompts, we're going to start at keyframe zero. And this uses JSON. So what I would recommend is if you're getting errors, double check that your JSON is validated, go in here, pop in your prompt. I see our other one's definitely ready to go. So this prompt, we know it works. I mean, this is the default one. We're going to try putting our own prompt in here.

Like so.

And now if we go to the JSON validator, we can pop it in here, hit validate, says, bingo. Looks good. The one that I see a lot in the discord is if you do multiple. Like, let's say we wanted to do one for.

Frame 100 like this, you know, and like cat here you don't put a comma.

But here you do put a. The last one doesn't have a comma. That's what always throws me. The last one doesn't have a comma. But if you do another line, you need a comma. So just keep that in mind. And if you're not sure, check it in the JSON validator. Very important. You'll see we also have negatives here. So there's two ways that you can do this. You can either put your prompts, positive here and your negative here, or you can put in your prompt here and then do the dash, dash, neg, and then put in your negative thing. So you'll see here it says, you know, do this one way or the other, but not both.

How we doing over there? Any questions so far? Anything coming up?

So far so good.

Yeah.

I think I'm seeing people having conversations in the chat, but saving some of the questions for later that we'll circle back around to. One thing I'm curious if you're able to do. If not, no worries. Are you able to zoom in at all? Can you, you know, control plus or something and get a little bit bigger on the screen there?

Is this a little bit better?

Yeah, I think that looks a little bit better. Cool, thanks.

Okay, cool. So this INIT image, we're going to use an image to start things off, which I find in my creative process is like hugely important. So we're going to go into Init. You see, I've got like a bunch of glitch video stuff here on the side from my synth in the background. So I'm going to click Use in it. Now, the strength of this determines like how close to the INIT image it is. So I'm going to try going up a little bit like six, five. And we're going to use this LZX one. So for the path, you can use a website path. For this I'm using. I've got the Creators Club private storage. So I'm just going to reference the storage on this cloud server that goes between instances and then the directory. Oh, wait, wrong directory. Init image.

Lzx1.

Okay, hybrid video, we're not going to talk about yet. Let's see. Whoops.

Anything else I need to do here?

I think that's mostly it. Okay.

Yeah, we're good to go.

So here you can see we've got it started to generate on medium. It's going to take a little while. So we can pop over to our other session.

When you start off the timer. I'll just go back to that. When you start off the timer, it looks like it's going to take a while and then it really comes down in speed. You saw it started at like 12 minutes and I was like, oh, let's switch tabs. But then I'm like, wait, this shouldn't take that long back 30 seconds.

Yeah, you see that in regular stable diffusion as well, when you're just generating images, right, it'll say like five minutes and then 18 seconds later it's done.

Yeah, exactly. So let's, let's just watch this because it's almost done. Let's see how it looks.

All right, so you can see there's a very slight smooth.

Movement. You know, I'm trying to reduce the amount of blur by Keeping the movement slow. You could use the anti blur as well. But here you can see, you know, that watercolor came out gorgeous. And when it slows down, you'll see, like, the details kind of clarify a little bit. So let's, let's see, let's go into here and make the frames a bit longer.

And we'll use a new feature.

To me anyways, but this is really important. So we've got this video that we created from the INIT image and maybe actually because it's so quick, maybe we just try a different one. So if we go to some of the more like rapidly like hyper growth, almost like flashy, flickery type of deform videos and you can slow that down. So when I started, like a lot of what I was doing was around trying to slow things down and make it like a little bit.

A little bit like slower. Lzx3.

Generate.

And more my speed. Because that's my personal taste. I wanted something that was a little bit smoother. What I like to do is change this deform cadence. Now there's a long explanation there, but the basics of it is if you have cadence, one, it will basically do a diffusion on every frame. And if I do two, it'll be every other frame and so on to five. So here we're going to get much less of that hypergrowth. And every five frames it'll start to change.

So we'll just rock 120 frames here. We've got our steps. Strength is, I would say, the next most important.

And this is because this is the cohesion between the previous frame and the next frame. And it will really help determine, like, how wild you get. It starts at 0.65 here, which is pretty good. I want to keep it, again, a little bit smoother. I'm going to put IT up to 0.75. But in some cases you might want to lower. It depends how wild and crazy you really want to get.

Going down to motion here. So there's a number of different. And these are all very important. So this is for the 2D camera. Rotate clockwise or counterclockwise. You know, counterclockwise would be negative number here. We're just gonna leave that as is. Now this is where you kind of look at deform and go, what the heck is going on? So with this zoom, what you really can look at this as the mathematical equation of a wave.

Now, I am not a mathematician. My, you know, algebra teachers would be ashamed that I might not know what this is without pulling out Some fancy calculator, graphing, good old ti83. But what I'm gonna do is just remove that and replace it with a one. And we'll show you some different tools that can auto populate that. I'm sure someone in the chat is already like chatgpt. I'm just going to pop that in there, it'll tell me what it does. It's probably a good bet. ChatGPT and those other AI chat tools are really good for figuring this stuff out.

So the important thing to realize, you see there's a zero here. And the important thing to realize is like that is what frame that this takes effect on. And if you leave it at zero, it just means the whole time throughout the video we're going to be at Zoom 1. Zoom is this one setting that's very important to note that you cannot put zero on it. That will not stop it. Zero just basically, I don't know, it goes into like nothingness, infinity. It just like messes with it, you'll get an error. So keep it at one, which is essentially standstill. Zoom one not moving. Just keep that in mind and make it down to one for zoom if you want to keep things flat without a zoom. Now we can actually add in maybe some keyframes here. So here we've got on frame 20, you know, we're going to zoom in a little bit. Frame 50, we're going to pull out a little bit. Frame 100, we'll just stay back and still. And what it will do is it will smoothly move over the frames and blend that or tween, there's a technical term for it, but it will slowly move from 1 to, to 1.005 as the frames advance from frame 0 to frame 20. So that's just a really good idea to keep in mind that that's how it will smoothly move through.

Now, X, you know, this is the left right as it says here. So we're going to. So we'll get that one going and see what the effect is by using a different one. Because it's going to change the colors, it's going to change the shapes you can see, you know, in this image here.

Pop it out.

Go from negative 1.1 down to zero again. So it's going to move. So it's really grabbing like these, these lines from the video synth and bringing them in, making it this abstract kind of thing. The strength is quite high, so it's not getting too wild. It's sort of sticking very close to that. Init Image. But once it's off that first frame, it's all on its own and it's just the strength that's keeping it together and then stop and then move again. And we'll do something similar with translate Y down, of course.

All right, now going on to noise. Noise is actually incredibly important and the reason why. And together like glue as it goes on.

So I think that other one should be done. Yep. So let's check it out. Hit update video. There it is.

We got like a spaceship or something in there.

A foot.

Uh oh.

If you guys are going to be in the quorum, it's important is if you think about it in one image, you know, noise is not a big deal, you know, e comments.

But yeah, you can see it's keeping pretty close to it. So if we wanted to make a little bit wild, you know, we could maybe lower this a little bit, go over to keyframes, bring this down to 6, 5. You can kind of mess with it, but it's necessary for making a diffusion. But when you're dealing and maybe put the cadence at four.

Thing with a video and you're using the strength to have coherence from one frame to another, what that means is that noise can feed back on itself and quickly get out of control. And you see this a lot where like people try again.

And this is where that experimentation comes in handy. Because what I find is depending on where you start and kind of what you're working with, your settings will be totally different. There's no like, this is the way to do it every time to make a great result. And there isn't in some text to image. You can definitely see that when there's like a prompt that's just amazing and like people pop it in, they all get the same are like, what happened? Like, my video started tiling and getting squished and the colors got all blossom result. With Deforum, you can get some incredible results in it by copying other settings. But what I find is it's really dependent on kind of blown out and like very I lost everything. A lot of times it might be because of the noise is too high and the strength is too high and they're combining together to just crush everything into noise. So for me personally, I like to keep a lower noise, not introduce as much. But if you're doing a more chaotic, more wilder one, like if you're what you're using as your source, your init image, your prompt and everything that'll determine the settings.

So here we go. Now you can see it's moving like a bit faster and it's going off of our init image faster. And then you can crank the noise up a little bit, try it out, find out what you like, you know, become a noise as well. Really goes into like an abstract sort of thing. This looks like a fruit. I don't know what's going on there. Right. But you can get the idea that it's like it's shifting a lot more. So artist, if you want. I'm going to try to minimize it as much as possible in this one.

Let's use guided images here. Now the way I'm going to use them is a little bit abstract because I like make abstract art. But you could try to. So for coherence, this is the color settings. We'll go over some of them as we go through like different modes. Bring it really close to it and make it like force it into a certain image that you've generated. And then there's an anti blur as well. I'm just going to leave those all blank. Config you may already be familiar with from.

To make like an animation. And that's like a very complicated thing. Like I know we saw that incredible like anime that the guys made forgetting their name, text to image or image to image. You know, lower values more creative, higher values closer to the prompt. What is it?

Oh, the corridor Crew guys corridor.

Yeah, yeah, yeah.

That was a. That was phenomenal.

Yeah, yeah, that stuff's like amazing. But you know, I try to stick to the more abstract stuff but you can force it to go down those roads. So there's a lot of difference that we'll leave that as is. Well actually maybe we'll pop it up to nine or something like that. Okay, now there's a bunch of other settings here which you can use to determine the different things that you can do to like kind of.

Have it do a specific type of thing. We'll go into hybrid video after to start showing you that because that's going to be very important ways that it uses seeds, for example. So I'm going to keep it on iterative, which means that the seed will go up by one for each frame of animation.

You can if you want. So there's a very important readme here. This only works with 2D and 3D mode. Go back in and change it to a fixed seed. So you could even schedule that by a keyframe, which is awesome.

Interpolation is actually something in specific, in stable or sorry, deforum and it is not frame interpolation. Which you might be familiar with frame interpolation, like basically takes like. I love that they thought of this. My seed is, you know, random, so I'm not too worried about it. It's just going to pick a random spot and iterate from there.

All right, so let's put our prompt in now the key of like a 120 frame movie. And it says like I'm going to make this 360 frames by multiplying the number of frames and then like linking them together. Sort of like how Deforum is doing this with the strength to make it match from one to another. But the thing about this, it's very important notes I always. It's like tweening those frames and putting them together.

So this is different. In Deforum, what it's doing is it's read these very, very important.

So with these prompts, actually tweening prompts, it's going between from one prompt to another, bringing it there. So just keep in mind that that is what interpolation mode is. We're going to start at keyframe 0 and this uses JSON. So video input mode. In the old one I was actually not. What I would recommend is if you're getting errors, double check that your JSON is validated to happy with. And yes, you can use any image to start the animation like I'm doing here. Go in here, pop in your prompt here. Just some random stuff I grabbed off my synth. But could. Could grab a new one from the Synth right now and pop it in here. No problem.

Or something I download.

Yeah, somebody was just asking if they can use, you know, something that they drew as the starting image. So yeah, just wanted to click see our other ones.

Definitely ready to go. So this prompt, we know it works. I mean this is the default one. We're going to try putting our own prompt in here.

Like so.

And now.

If we go verify. Yeah, any. Any image doesn't really matter.

Yeah. Or you could like take something you drew, pop it into control. Net and go to the JSON validator. We can pop it in here, hit validate, says bingo, looks good. Then put it in here. Right. So there's. There's like lots of capabilities.

The one that I see a lot in the discord is if you do the video input mode, I do not use. So I will just say that straight up. I wasn't too impressed with it. What I have used instead is hybrid video do multiple. Like let's say we wanted to do one for.

Frame 100, but there's a few specific use cases where you might want to use that. But anyways, see how it says set the init image tab strength higher. So recommended 0.65. And then like this, you know, and like cat here you don't. You need to set the seed behavior to schedule. So let's get the schedule piece going.

Seed behavior, put a comma.

But here you do put a. The last one doesn't have to schedule. And I will put in.

The seeds that I have here.

To do in a little bit longer.

And then we can grab comma. That's what always throws me. The last one doesn't have a comma. But if you do another. The prompt for this.

So what just entered in the seed schedule there line.

You need a comma. So just keep that in mind. And if you're not sure, check it in.

Where did that come from?

That. That's just something I've practiced with before to make sure this was working properly. But here I'll just walk through it. This is seed 5. It can be any number. I just chose 5 arbitrarily in the JSON validator. Very important. You'll see we also have negatives here. So there's two ways that you can do this. You can either put your prompt and then negative one would generate a random seed at frame 200. We're going to do seed 555 and then frame 300 will be random again. It's positive here and your negative here. Or you can put in your prompt here and then do the dash, dash, neg and then frame 400. The end will be seed five.

Gotcha. And the. The seeds are just fairly random, right? Those are arbitrary numbers.

Yeah, I have. And then put in your negative thing. So you'll see here it says, you know, do this one way. Haven't like, you know, gone in and investigated hundreds of seeds or anything like that. Gotcha. And we can put any number in there if we wanted here on the images. So what I've done is I've gone or the other, but not both.

How we doing over there? Any questions so far? Anything Coming up on.

Frame 0 is image 1, 2, 3 and then back to 2 and 1. You see that?

So far so good. Yeah, I think I'm seeing people having conversations in the chat.

But it does say you can make a loop. Now what I will tell you is that it takes a lot of work to make a loop in this.

Saving some of the questions for later that we'll circle back around to. One thing I'm curious if you're able to.

And you'll really have to jog your settings because if you think about it, we're starting with an INIT image. It's getting like the strength is generating these frames off of it with the cohesion and the noise. And then we're injecting another image into it at frame 100. It's going to try to blend there. So because this is very short, like.

400, if not, no worries. Are you able to zoom in at all? Can you control plus or something?

Frames is not long. Even if it's 15 frames per second, it's not long.

Get a little bit bigger on the screen there.

Is this a little bit better?

Yeah.

To make a loop will take a lot of work there. It will really have to go through.

I think that looks a little bit better. Cool, thanks.

Okay, cool. So this INIT image, we're going to use an image to start things. A lot of stuff and a lot of frames. You can do it. Just know that this is probably not going to look like a loop, even though we're going back to the original image off, which I find in my creative process is like hugely important. So we're going to go into Init, but with some like INIT image stuff, we could definitely do that. So it's not impossible. So since we've changed this to 400 frames. Right. Let's make sure we. You see, I've got like a bunch of glitch video stuff here on the side from my synth in the background. So I'm gonna click use init. Now, the strength of this determines, like how close to the INIT image it is. So I'm gonna try going up a little bit like six, five. And we've got that 400.

And enable guided images mode.

Okay.

All good there.

We will need to change the seed schedule or. Sorry, that's not the seed schedule, the motion. So the frames match up. So this zoom, actually we're going to use, you know, this LZX one. So for the path. You can use a website path for this. Need to change that translation X we're going to do this. So now it's going to frames 100 and 300 and y as well. Maybe we'll add a fourth hundred. This I'm using. I've got the Creators Club private storage. So I'm just going to reference, you know, the storage on this cloud server that goes between instances and then the directory. Oh, wait, wrong directory. Duh.

Here.

400.

So just to clarify here, so when on the zoom and on the trans image.

LZX1 translation x and on the translation Y, these series of numbers, the. So starting on frame zero, the one is. The one is basically no zoom. Is that correct?

Okay, hybrid video we're not going to talk about yet. Let's see. Whoops.

Anything else I need to do here?

Yes, but on translation x and Y, it means something else. It means zero means. No, I think that's mostly it. Okay, movement.

Gotcha. And so what we're doing here is from frame 0 to frame 99, essentially, it's not going to zoom. And then starting on frame 100, it's going to start zooming in slightly.

No. Okay, it's frame zero. Yeah, we're good to go.

So here you can see it's going to start slowly increasing the zoom till it hits the. The value of.05 on frame 100. Yeah, cool. That's for clarifying that. Yeah, yeah, because it's not. It's not quite obvious, but yeah.

Okay, so from frame 0 to 100, it's going to slowly zoom in until it hits that one point.

We've got it started to generate on medium. It's going to take a little while. So we can pop over to our other session.

It.

When you start off the timer, I'll just.

0,05. Okay, gotcha.

Yeah, let's go back to that. When you start off the timer, it looks like it's going to take a while, and then it like, really comes down in speed. So let's get that one bacon in the oven there. Three minutes on a medium. Again, these are very fast. You know, it's. It's. So you saw it started at like 12 minutes, and I was like, oh, let's switch tabs. But then I'm like, wait, this shouldn't take that long. Not our best card in here. But, you know, it's still able to generate all these pictures at a fast rate.

All right, do we have any questions? Maybe do another giveaway back. 30 seconds.

Yeah, you see that in regular, stable diffusion as well, when you're just, hey.

What do you think? Well, we're waiting for this bad boy to cook.

Yeah, let's do a giveaway. I did get my bot working, so now I can just pick from. From anybody who's been accurate generating images. Right. It'll say like five minutes, and then 18 seconds later, it's done.

Yeah, exactly. So let's. Let's just watch this because it's. It's almost done. Let's see how it looks.

All right, so you can See, there's.

A very slight div in the last 15 minutes. So if you've been chatting in the chat for the last 15 minutes, then smooth.

Movement, you know, I'm trying to reduce the.

You're eligible. So I'm going to go ahead and roll it real quick and. Well, it picked Run Diffusion as the winner. So let me roll it one more time here.

Well, no, we need. We need people to say a word.

Oh, okay. Yeah, we can do it that way. Right now, it's just said, as anybody who's been active, do a tween.

I want to hear people say tween queen. Like amount of blur by keeping the movement slow. You could use the anti blur as well. But here you can see, you know, that watercolor came out gorgeous. And when it slows down, you'll see.

Like W, E, E, n. Yeah. All right.

Such a weird word.

PW, E, E, N type tween.

Yeah.

Run Diffusion's pretty. Pretty happy that they. They won there.

And we're giving away a $30.

Details kind of clarify a little bit. So let's. Let's see.

Let's go into credits on this one. $30 in credits. All right.

Tween and Weezer, the good old bands.

All right. I'm debating when to press the button because the. The still rolling in.

Yeah, let people, you know, let's. A bunch of people on. Let's make sure we get everyone in there. Oh, we got some D here. And make the frames a bit longer.

Double, double dippers there.

Yeah, it only let them. It'll only see them once.

Okay. Okay, good, good.

All right, I'm gonna go. I'm gonna. I'm gonna press the button. So this is for $30 in credit here. So we got Tim Buck two. That's Tim Buck. And the number two. You just.

And we'll use a new feature.

To me anyways, but this is really important. So you. We've got this, like, video that we created from the INIT image. And maybe actually because it's so quick, maybe we just, like, try a different one.

So if we go to $30 in credits.

Nice. Hey, there's Nightbot. All right, awesome. Why don't we do. Why don't we do another one? Do a 20 or something like that.

All right, so. All right, so this is also going to pick from Tween, since everybody's on the tween train right now.

Yeah.

All right, so we got Sergio F. Mohar Masters nation.

That was LZX3.

Generate.

For a 20.

Nice. All right. Let's do a 10. All right.

And then let's see, for the 10, we got deep love, AI.

Hey. All right. Awesome. The tech priests will be pleased.

Just making some notes here, so I don't forget who got what.

So we'll get that one going and see what the effect is by using a different one. Because it's going to change the colors, it's going to change the shapes you can see, you know, in this image here.

All right, awesome.

I'm also. What I did was I. You can't see what I'm doing right now, but.

I took. I've got my little notepad here and I wrote down all the prizes. I'm just crossing them off as we give them away way so that I don't lose track here.

So it's really grabbing. Like these. These lines from the video synth. Great. Great. Yeah. We will make sure all those credits get handed out to you. If you're not signed up already, you will have to sign up, go into our discord, send us a message bringing them in, making it this abstract kind of thing. The strength is quite high, so it's not getting too wild. It's sort of sticking very close to that init image. But once it should just say, hey, you know, I won the prize. We'll verify it and then, you know, you can get the credits added to your account.

Yeah. So somebody's asking how to get them. Yes. Jump into the run diffusion discord, and they'll make sure to take care of you there.

Sweet. All right, we are good on this. Let's update the video, see what we got off that first frame. It's all on its own, and it's just the strength that's keeping it to get. Okay, so it's starting in the same place with the image, but let's see how it develops. Okay, you can see here it's very rapidly changing as it gets to those different images. But the strength and color. Oh, we got a little car in there. That's cool. The strength and color is gather like glue as it goes on.

So staying pretty coherent between these images, even though they're very different. So there's a few different settings that we can look at in there. We're almost done. And then it's like stopping. So it kind of blooms a little bit in the quality. You can see there's like a little bit of blur as it's tweening between the different things there. So if we go to guidance, that other one should be done. Yep. So let's check it out. Hit update video. There it is.

We got like a spaceship or something in there.

A foot.

Uh.

Oh, feet, guys gonna be in the good images.

There's a few different settings here. And so it's important, like, if you really want to nail kind of an exact image, this is where you would do that. We've got the tweening frames. Frames, schedule that word, man.

Image strength. Right. So, you know, how high do we really want this? Do we want it to affect our image a lot or not very much? How do we want it? Comments.

But yeah, you can see it's blended in. Now. The blend is an equation, and you see the equation listed there. But it's a mixture of the blend factor, scope and the tweening frames, keeping pretty close to it. So if we wanted to make a little bit wild, you know, we could maybe lower this a little bit, go over to keyframes, bring this down to 6, 5. Schedule here. So again, math not that great. ChatGPT probably has the answer. Maybe human can answer in a bit here when we go on. But just know that these are fun sliders to play with. And we can just try and maybe put the cadence at four, try some stuff out, you know, might as well. While we're doing this again, you want to make sure that.

You know, you've got some pretty consistent images, I would say, in here. But hey, experiment, get wild. Put some weird stuff in there, do a longer video, try it out. Or maybe do tons of images, like, try again.

And this is where that experimentation comes in handy. Every frame, like a different image, or every 10 frames a different image. You know, there's lots of different ways that you can play with this and try it out. Color correction. I kind of want to lower that. How close to get the colors of the input frame. Right. So actually bring that down and see if we can get some of these other colors that. Because we've got like some really wild colors in here. Andy. Because what I find is depending on where you start and kind of what you're working with, your settings will be totally different. There's no like, this is the way to do it every time to make a great result. And they're right. Let's see if we can. Well, these. These are actually pretty consistent. Isn't in some text image. You can definitely see that when there's like a prompt that's just amazing. And like people pop it in, they all get the same. So maybe instead we'll go four.

And five try that out. But we will have to move on to some other stuff here because I know I'll just like nerd out about this stuff. Like oh, this art's so fun. Awesome result. With Deforum you can get some incredible results in it by copying other settings. But what I find is it's really dependent on what you're using as your source, your init image, your prompt and everything that'll determine. I'm going to keep doing this. Now I do want to show the other two modes which will take a little bit. So we should probably.

Yeah. And 3D is one that when I was messing with the settings.

So here we go.

Now you can see it's moving like a bit faster and it's going off of our INIT image faster.

I stuck to 2D. I didn't even move into 3D because I wasn't ready to make a video with all of the various options that are available inside of 3D.

Yeah. I'm going to try to see if we can get a large. Let's see.

Run diffusion. Can you kick someone off for me? No, I'm just kidding.

I'm kidding.

Let's see what we can get here. Let's try a 2.1 select and launch.

So we'll see if we get something there. But.

We can go here and as well really goes into like an abstract sort of thing. This looks like a fruit. I don't know what's going on there. Right. But you can get the idea that it's like it's shifting a lot more. So let's kind of talk about a 3D version because the 3D camera is incredible. Like I am. I am really blown away and I will say I do want to give some shout outs here because the Deforum Discord is. Let's use guided images here. Now the way I'm going to use them is a little bit abstract. Incredible. Like the everyone there is super helpful. They're all so nice and they get bombarded with act because I like make make abstract art. But you could try to questions non stop. But the community's really come together and helped out there in a big way. So the shout outs I want to give are one Snack Pack. I saw you there in the channel. That's how I learned was like by looking at what Snack Pack does. They post their videos. They include the prompts you can go through figure out how things do what they do and it's incredible. Like big fan of Snack Pack.

All right. And we got a bunch of. A bunch of fresh LG cards loaded up. We do have this like scaling back end so, you know, we can technically go to 10,000. It just, you know, we're bring it really close to it and make it like force it into a certain image that you've generated to make like an animation. And that's like a very complicated thing. Like, I know. We saw that. Incredible. I guess our provider was a little caught off guard there.

But. So Snack Pack's one of the shout outs. The other one I want to give a shout out to and we'll talk more about it later, is really big name.

Like anime that the guys made, forgetting their name because they not only develop an ad to the Deforum extension, so some incredible stuff, but they also post all their videos with like details of how they did it. So both of those creators are just incredible and definitely worth looking up in the discord. Just go through their past messages and just see all the like crazy stuff that they've made because it's incredible. All right, so we are in this medium one.

Still spinning up the big box. But let's jump in here. So we're gonna do. What is it?

Oh, the corridor. Crew guys, corridor.

Yeah, yeah, yeah.

That was. That was phenomenal.

Yeah, yeah. That stuff's like amazing. But, you know, I. I try to stick to the more abstract stuff, but you can force it to go down those roads. So there's a lot of different.

Maybe we'll make it a little wide. Things that you can do to like kind of have it do a spectator.

I don't know.

Let's try it out.

And 20 steps.

Go to keyframes. So we're going to go 3D mode.

We are going to do wrap. So this is really important. And especially when you're doing 3D camera work, you have to visualize that the way the 3D works is it is going through.

And making a depth map of what is active and then using that in a 3D space. So there's almost like consider that there's a virtual camera and your specific type of thing. We'll go into hybrid video after to start showing you that because that's going to be very important. So it's a very important. Read me here. This only works with 2D and 3D mode. Training the camera on where to move in this 3D space. It's wild stuff. We did get our large box. Thank goodness. Let's. Let's do that in a large box instead. And just. Oh, it's interpolation is actually something. You know, we showed how it works on a.

In specific, in stable or Sorry Deforum. And it is not, you know, on a smaller box, a medium box, but.

Maybe make it wide screen. Oh, got to be in the right one here.

10.46 Hot frame interpolation, which you might be familiar with what frame interpolation basically takes. Let's say you have a 120 frame movie and it says I'm going to make this 363D mode frames by multiplying the number of frames and then linking them.

Cadence.

I guess we can put that back at 5 again.

And.

Maybe we'll do.

Together how Deforum is doing this with the strength to make it match from one to another. But.

Now I will say just as a note before I jump into the big box and start showing you all the speed and what it can do. If you go into the deform thing here, you can actually see all the individual frames of the image that you're making or the video that you're making. It's like tweening those frames and putting them together.

And that can really help you if we pop this up. So this is different in Deforum what it's doing is. It's open. That can really help you when you're making a new video. So this is also a folder that I delete quite often. Like I will go in here, I will just go search video, hit enter. Now I can see all the videos that I did right. Like you know, looking at this actually tweening prompts. It's going between from one prompt to another one. We can go in and see there's our video that we had before. So I will get all the videos that I want. But here you can see all and bringing it there. So just keep in mind that that is what interpolation mode is. All the frames for everything that we've done so far. So it takes a bit. It's best if you like delete it when you're done using it and start up with a fresh one. But I will say that you can scroll video input mode in the old one I was actually not too happy with. And yes, you can use any image to start the animation like I'm doing down here and see in advance like oh, this video is looking like it's not going to be so hot here. Just some random stuff I grabbed off my synth. But could could grab a new one from the. Maybe it's more like a hot mess. And then you can stop your video synth right now and pop it in here, no problem.

Or something I download but the other way you can do this to, you know, reduce the amount of time that you're experimenting and testing things out is you can do a much smaller amount of frames and then take a look at how it's looking in the first like 30 seconds or so and then try it again with a longer amount and again with a longer amount. And then.

Yeah, somebody was just asking if they can use, you know, something that they drew as the starting image. So. Yeah, I just wanted to clarify. Yeah, any. Any image doesn't really matter.

Yeah. Or you could take something you drew, pop it into control. Net and then put it in here. There's lots of capabilities.

That will help you cut down on server costs or time if you're doing it locally, but it will really help you just at the beginning, make sure that what's coming out looks great and then do it. It's nice because in Deforum, you know, if it's bad, pretty fast, whereas when you're.

You generally don't. The video input mode I do not use. So I will just say that straight up. I wasn't too impressed until much later. By that time you've already trained most of it. So.

Yeah, so let's go into this and just go into some more settings here. If you want to do another giveaway or something, I'll.

Yeah, I actually want to pull Human onto the stream if, if he's available now.

So I want to pull him on.

To mess with it. What I have used instead is hybrid video, but there's a few specific use cases where you might want to use that.

But anyways, maybe help with some of the concepts and explain some of this stuff. Now we got Human on here. Who. Human is one of the original creators of Deform. He actually gave Deform its name. He runs the Deforum Twitter. He's inside the Deforum Discord. He is the man over at the forum, so thought it'd be cool to bring him on and he can help answer some questions as we're going through.

Set the INIT image tab strength higher. So recommended 0.65 and then you need to set the seed behavior to schedule. So let's get the schedule this and.

Maybe help clarify some of the concepts that you're. You're walking through as well. So thanks for, for joining us today, Human.

Thanks for having me. It's a pleasure to be here. And yeah, like lots of questions in the live stream chat. I've been trying to answer them. There's a lot going on under the hood.

Little piece going.

Seed behavior, you Know.

At Deforum, you know, we're, we're a community and we want to give people to schedule.

And I will put in.

The full access to as many tools as we possibly can. Unfortunately, that means that there's a ton.

Of settings.

But we just try to make everything available so you have complete control.

Now. Is there anything that's.

That sort of seeds that I have here?

Do one a little bit longer.

And then we can grab.

Come up so far that, that maybe you feel you want to clarify a little bit more or, you know, bring, bring maybe some more attention to. Because I noticed we were kind of in the, in the, in the background chat here. I've got my, my head on a swivel here. I've got, you know, chat over here. And I'm trying to, you know, look at the cameras and pay attention to revolved what he's talking about. So I'm, I'm kind of looking all over the place. But is there anything that, that sort of came up so far that you want to dig a little deeper on or clarify it?

The prompt for this.

So what you just entered in the seed schedule there, where did that come from?

That's just something I've practiced with before to make sure this was working properly. But here I'll just walk through it. This is seed five. It can be any number. I just chose five arbitrarily.

Yeah. So one thing that I think is really important to understand is sort of like how these diffusion models models are working, especially like in the context of these stable diffusion animations. So I guess like the, the key thing to understand is that this is essentially a, like an initial image animation. Right. So the diffusion model will take.

And then negative one would generate a random seed at frame 200.

We're going to do noise and with a text conditioning, like produce some image.

And instead of starting from noise, we can start from an initial image and we can still use that same conditioning to sort of like modify and augment the, the starting image. And essentially like all these need 5, 5, 5.

And then frame 300 will be random again. And then frame. Frame 400, the end will be seed 5.

Gotcha. And the seeds are just fairly random, right? Those are arbitrary numbers.

Yeah. I haven't gone in and investigated hundreds.

Of seeds animations are, is we're taking the output of a generation and we're putting it into another generation. And we sort of like have this feedback loop and, and we have clever ways to sort of augment and like translate, rotate, zoom, the output of one.

Video or anything like that.

Gotcha.

And we can put any number in there if we wanted here on the images. So what I've done is I've gone.

Frame 0 is image 1, 2, 3, and then back to 2 and 1.

You see that generation, put it into the next. And we sort of like create this illusion of like a camera moving through 3D space or 2D space. And then, you know, honestly, like, you could just keep going, right? There's. There's so many things you can do.

So it's kind of like. It's. It's kind of like the sort of stable diffusion image to image. But it's just kind of. It makes one image and then sort of takes that image and then makes another.

It does say you can make a loop. Now, what I will tell you is that it takes a lot of work to make a loop in this. And you'll really have to jog your settings because if you think about it, we're starting with an INIT image or.

One that's based off of that. And that's why each sort of thing blends so nicely. Because the next image is sort of based off the one that was previously made.

Yeah, exactly. I mean, it's exactly image to image. So it's image to image, but you're just taking one output and putting in into another image to image. And then, you know, doing those little changes, right? Like move.

It's getting like the strengths is like generating these frames off of it with.

The output or zoom into the output just like a teeny bit and then.

The cohesion and the noise and then we're injecting another image into it at frame 100.

You do this enough times, you create enough frames, you stitch them all together into a video, and then now all of a sudden you have a zooming video.

Very cool. This is actually a question that I had for you. Nathaniel's asking it here as well. But where did the name Deforum come from? I know that was.

And so it's going to try to blend there because this is very short. Like 400 frames is not long. Even if it's 15 frames per second, it's not long.

You were the one who came up with the name. So I'm curious where the name's from.

Yeah, so I actually started sort of messing around in the AI space in, I guess, like late 2021 and, you know, before I found all these amazing communities on Discord, I sort of like had this idea in my head that I wanted to figure out a way to, like, program.

So to make a loop will take a lot of work there. Like, it will really have to go through a lot of stuff and a lot of frames. You can do it. Just know that this is probably creativity.

And that was like, a really stupid idea. But.

Like, I started exploring and experimenting with code and the first two folders I made.

When, you know, starting my AI journey was. Was Human and Deform not spelled the way that they are not going to.

Look like a loop. Even though we're going back to the.

Original image spelled now. But I've sort of had these names sort of from the beginning, but with.

Some like, init image stuff, we could definitely do that. So it's not impossible.

And Deform sort of just worked and it fit. And.

It'S a combination of design and forum. But also it's like a play on words like, you know, with this AI.

So since we've.

Changed this to 400 frames, right, let's make sure we've got that.

400.

And enable guided stuff.

We're like deforming these images and augmenting them with AI. So that's really like how the name came together.

Gotcha. Very cool. Very cool.

Images mode.

Okay, cool.

Well, so Revolved was mentioning another giveaway. I do want to say something real quick. I know it was Judge works, I guess when I was messing with Nightbot the first time around and it didn't work, he won one of the things. So I'll make sure that, you know, he gets one of the prizes as well. So I just want to say that real quick before I go any further and, you know, somebody gets upset with me on.

On that.

I'll get there.

You won one, too. Very cool with the forum. If we go back over to some of the settings and maybe start messing with the 3D, I want to dig into the 3D and it would be cool to have.

We will need to change the seed schedule or, sorry, not the seed schedule, the motion.

The frames match up. So this is zoom, actually both, you.

Know, revolved and human on, you know, on voice chat here, as we're going through it, to talk through some of the. What's going on as we're. We're messing with some of the 3D stuff because, you know, 2D to me was fairly straightforward when I was messing with it. There's only, you know, so many motions you can do with it. When you start getting into the 3D, you know, there's quite a bit more knobs and buttons you can press. And that's where it was starting to.

Get a little to change that. Translation X, we're going over my head.

So what we're seeing on the screen right now, this is. This is a 3D generation.

Yeah. So it's some of my earlier work in the 3D. You can really see the DAP and do this. So now it's going to frames 100 and 300 and Y. Then, like how the camera is moving around. I actually wanted to bring this one up because there's some interesting stuff with it. So I used an INIT image here. Right. So this is some of my video art. And then. Well, maybe we'll add a 400.

Here. 400, like, kind of goes off and you see these, like, lines, you know, actually human. I was kind of curious. I know this one comes up in the discord, but those lines are kind of when the camera goes out of bounds. And there's a few different ways. I think it's like you adjust the strength setting. You can kind of tone those down. I mean, I kind of like the effect. But.

So just to clarify here. So when on the zoom and on.

The trend, you know, do you know why that's. That's sort of happening?

Well, so I think these are just like artifacts of the diffusion model. And I personally love them. I kind of try to accentuate them as much as I can just because they're sort of like this immersion translation.

X and on the translation Y, these series of numbers, the. So starting on frame behavior of these.

Models and you know, they're not. They're not desired necessarily, but they're certainly cool.

Yeah, I agree. You can get some really awesome textures with it. It's. It's a ton of fun.

Zero. The one is. The one is basically no zoom. Is that correct?

This one's more of like a sideways sort of one, but still use. Yes. But on translation X and Y, it means something else. It means zero means no zing. The depth map here a little bit. You can see there's the camera, like turning and it kind of twists everything around. I really love it because it kind of gives us like zero gravity feeling. You know, you're like floating through space and there's some of the artifacts. And you see here, they got interpreted as desert dunes, you know, movement.

Gotcha. And so what. So what we're doing here is from frame 0 to frame 99, essentially, it's not going to zoom. And then starting on frame 100, it's going to start zooming in slightly.

No.

Okay.

It's frame zero. It's going to start slowly increasing the zoom till it hits the value of one. There's these little flashes that come from this. This is actually from the frame interpolation that I did afterwards to make it go smooth. So I use an. This is. Prior to frame interpolation being added into Deforum, I used a free application called Flow Frames, or I think they might have, like, a donation or something like that. But Flow Frames was what I used, and it does kind of make things a little bit flickery in that area. But otherwise, I mean, you get this, like, nice, smooth animation, which is really cool.

We can dive into the settings. I just wanted to show off some of this other work while we were chatting.

This is awesome.

The initial art that you were doing, that's all the.

On frame 100. Yeah, that's for clarifying that. Yeah. Because it's not quite obvious.

Okay. So from stuff that's in your background there, you make it with all the. All of your. I'm not quite sure what you call all the crazy wires and cables and stuff going all over the place in the background, but that's. That's built with that stuff.

Yeah. The initial image. Yeah. Actually, if you change to my camera here.

This is the video frame 0 to 100, it's going to slowly zoom in until it hits that 1.005. Okay, gotcha.

Yeah. Synth that I have running right now. So then over here, I can, like, tweak these settings to get, like. This is a synth oscillator. And then if I want, I can change, like, the blending between layers. So let's get that one bacon in the oven there. Wow. Or I can. This one uses, like, shaders, which you can. Shaders are amazing. You can get three minutes on a medium. Again, these are very fast. You know, it's. It's not our best card in here, but, you know, it's still able to generate all these pictures at a fast rate.

Like, look at, like, programs like vdmx, and there's. There's, like, Shader Toy. Actually, if you're looking for, like, cool stuff to run through Deforum, check out Shader Toy, because there's a ton of cool, like, applications that you can do to make, like, awesome little weird animations and stuff like this. And there's a whole community around making these things. You can also run videos through Shader Toy.

All right, do we have any questions? Maybe do another giveaway. What do you think while we're waiting for this bad boy to cook?

Yeah, let's do a giveaway. I did get my bot work, so now I can just pick from anybody.

Who'S been acting to give extra effects. So.

Wow, that's crazy. That's so cool. Yeah, that's like a whole world I've never even dived into before, right?

Yeah, there's like, there's a ton of stuff out there in the video world and it's neat to see, like, areas where it sort of impacts with AI stuff. So maybe switch back and active in.

The last 15 minutes. So if you've been chatting in the chat for the last 15 minutes, then.

I've got Shadertoy on the thing here. You can see that there's different fun things and then the codes here. So you can actually go in and be like, oh, I'm going to change.

You're eligible. I'm going to go ahead and roll it real quick.

Some of these vectors and make this do other things.

Oh, wow.

And then you could load that video into.

Into it. You can use a camera input. There's all kinds of crazy stuff in here. So lots of fun to experiment with.

Wow, that's really cool. Real quick question for human Deep Love, AI is asking if there's. If you see like any sort of Adobe After Effects extensions, it picked Run to Fusion as the winner. So let me roll it one more time.

Well, now we need to form or anything like that in the future.

So I was actually thinking about, we.

Need people to say a word.

Oh, okay. Yeah, we can do it that way. Right now it's just said, as anybody who's been active, do a tween.

I want to hear people say tween.

Tween, Like PW this earlier.

I think the next sort of evolution.

To U E, E, N. Yeah, all right.

It's a weird word.

P W E E N type tween.

These animations is actually going to be the Dream Studio Pro. I'm not sure if either of you are familiar, but.

So from the Disco Diffusion days, which is like the precursor to Deforum, they were using like sort of different diffusion models but like similar ideas where you're.

Using initifusion's pretty, pretty happy that they.

To create these animations. A lot of the developers of Disco Diffusion are now working for stability and actually working on Dream Studio Pro.

But they won there.

And we're giving away a $30.

I know there's like a lot of.

Like development going into sort of others.

And credits on this one. $30 in credits. All right.

Tween and Weezer trying to make as smooth animations as Possible.

But who knows when they're actually going to end up releasing that. They've been talking about it for months.

I've seen on Twitter some little leaks, some little good old bands.

All right. I'm debating when to like people trust.

Testing out the new goods. So we've got sdxl. I've seen people testing that.

Press the button because the. Still rolling in.

Yeah. Let people, you know, there's a bunch of people on. Let's make sure we get everyone in there. Oh, we got some double double dippers there.

Yeah, it only let them. It'll only see them once.

Okay. Okay, good. I saw one guy say, like, this means no one will use 1.5 ever again, which is a big claim. We'll see if, you know, the anime waifu stuff goes over into sdxl. Maybe, maybe not. But I've also seen that there's another thing beyond that, which is SD 3.0, which. Good.

All right, I'm going to press the button. So this is for the.

The consumer version of it, which sounds fascinating. So lots of exciting stuff going on.

Emad still taking a ton of flack, though, actually. It's funny, you know, he's taking so much flack for 1.5 because it uses the artists. And he was posting on Twitter, he's like, I didn't even make that, like.

Runway $30 in credit here. So we got Tim Buck 2. That's Tim Buck and the number 2.

You just made 1.5 and released it. But they get no flack. I'm just like a person. So everyone attacks me instead of the company, you know.

So, yeah, I'm glad he's still out there building, you know, despite the haters, he's out there making stuff. So, yeah, I know Dream Studio is going to be coming out with some fresh hotness and then I'm assuming $30 in credits. Nice. Hey, there's Nightbot. All right. Awesome.

Emad's got that lovely, thankfully, attitude about open source. So the tools will be coming to the general population as well. Which means. Why don't we do. Why don't we do another one? Do a 20 or something like that.

All right, so. All right, so this is also going to pick from Tween, since everybody's on the Tween train right now.

Yeah.

All right, so we got Sergio F. Mohar masters Nation automatic 1111 will be.

Able to use it too. So that's exciting stuff. I don't know if you've got any inside scoops there, human, but interested to hear Your thoughts on the next generation.

Of for a 20.

Nice. All right, let's do AI tools with stable Diffusion.

So I actually have access to Stable Diffusion XL and it's amazing.

It's amazing. It's so good. It's like you get all of the Benefits of version 2 plus all of the benefits. All right.

And then let's see for the 10.

We got Deep Love AI from version one. It's really, it's really amazing.

Awesome. Wow, that's big. That's big hype.

Is there, is there any word around. I know we're sort of getting off topic here, but is there any word around like when the expected releases for the general public.

So I, I heard.

Hey, all right, awesome. The tech priests will be pleased.

Rumors. And of course, you know, things are super dynamic. So you know, I, I just making.

Some notes here so I don't forget who got what.

No idea if this is gonna happen, but I, I would feel like in the next month. But again, I'm not.

All right. Awesome.

I, I don't know. I, I really don't.

Yeah, I saw he. The nice thing is because he took all this flack for like, you know, runways 1.5 release, but he's made an opt out model and then opt in. So if you're an artist and you wanted your art in the model, you can reach out. He's got, there's like a website which is a third party website that artists put together and he started working with them.

I'm also. What I did was I. You can't see what I'm doing right now, but I took, I've got my little notepad here and I wrote down all the prizes. I'm just crossing them off as we give them away way so that I don't lose track here.

Great, great. Yeah, we will make sure all those credits get handed out to you if you're not signed up already. So you can opt out or opt in of having your art. You will have to sign up, go into our discord, send us a message in the next 3.0 release, which I think is, is really great. You know, it's not like he's just like, you know, just saying screw the haters, we're going to do whatever we want. Instead, he's like actually working with people and trying to build bridges and not like, you know, move fast and break things like some other.

I think that's a good way to go. I, I feel like just in the last, you know, four weeks or so, I've seen a Lot of the AI hate sort of die down. It feels like everything moves so fast. Right. But even just on my YouTube channel, I used to get comments all the.

Just say, hey, you know, I won the. The prize. We'll verify it.

And then, you know, time from people going, oh, AI is stealing from artists. And this is, you know, what you're doing is evil. And I used to get all those comments, and a lot of that has sort of. Sort of phased out a little bit just in the last several weeks, which I find very interesting.

Yeah, I know that there's. Oh, this one. I'm gonna extend this one too.

I know that there's. Yeah, there's. There's still a ton. And get. Get the credits added to your account.

Yeah. So somebody's asking how to get them. Yeah. Jump into the. The run diffusion discord and they'll make sure to take care of you there.

Hey, Doug Emad. I mean, deactivating his Twitter was. Was pretty. He. He made, like, a faux pas. I think he was talking about, like, firewalls, and everyone thought he was, like, admitting to stealing. But what he was talking about was, like, working with companies like Getty or. Or other companies to, like, have their.

Own, like, sweet. All right, we are good on this. Let's update the video, see what we got. Models. And so they are working to, like, work with companies as well that have their own, like, copyright concerns. Obviously, if you've done any amount of work in 1.5, you've probably seen. Okay, so it's. It's starting in the same place with the image, but let's see how it develops in the Getty Images watermark on there. So.

I just wanted to point out one thing really quick on this. So this is the guided images thing that we did in Nileps. Okay. You can see here it's very rapidly changing as it gets to those different images. You see, the color comes out a lot more.

From the individual images that we injected. And there you can see that it's closer to the images that I put in. So just by. But the strength and color. Oh, we got a little car in there. That's cool. The strength and color is staying pretty coherent between these images, even though they're very different. So there's a few. Adjusting some of these blend image strength, lowering the image strength. We were able to kind of get it a little bit closer to the. The images.

And I think we've got a few minutes on our 3D1.

Human, was there anything you wanted to chat about or should I just Jump into some Hybrid stuff and start spinning that up. What do you think? Two different settings that we can look at in there. We're almost done. And then it's like stopping. So it kind of blooms a little bit in the quality.

Yeah, I think Hybrid sounds. Sounds neat.

Yeah, it's all great. And this is so hybrid is something that like. And correct me if I'm wrong here, but I think a really big name contributed to the project and, like, made this on his own and then it was brought in via GitHub. Is that. Is that right?

Yeah. So I was.

You can see there's like a little bit of blur as it's tweening between the different things there. So if we go to guidance, actually.

Just scrolling through the user discord and I saw a really big name sort of just doing this incredible, like, ransack hybrid video stuff. And I was like, hey, like, we got to add this. This would be amazing. And he worked really hard and he put together.

Like a pull request to update the repo images.

There's a few different settings here. And so it's important, like, if you really want to nail kind of an exact image, this is where you would do that. We've got the tweening frames. Schedule that word, man.

Image strength. Right? So.

And boom, now we have Hybrid video. And actually, like, his work is cited by Runway ML, I think, in their, like, Gen one page paper.

Oh, but it's.

It's incredible that, you know, just you can be in the community, you can have a cool idea, you can, you know, get help from people, and all of a sudden, you know, you've just made some groundbreaking, you know, how high.

Do we really want this? Do we want it to affect our image a lot or not very much? How do we want it?

AI Tech.

Yeah, I saw the craziest story on Reddit too, where like a guy who had some, like, he's not a programmer, but he knew like, something about, like, video, I guess, and he just used Bing and Chat GPT and coded his own anti flicker. Like, he's got posts of Blended in. Now, the blend is an equation, and you see the equation listed there, but it's a mixture of the blend factor scope and the tweening frame schedule here. So again, math not that great. ChatGPT probably has the answer. Maybe human can answer in a bit here when we go on, but just know that these are fun sliders to play with and we can just try the current build on Reddit right now. So this is just like such a quickly moving Space. But what I love is it's so different from some of these other big movements we've had in the past where there's that like, early Internet feel of collaboration, which is. Which is a really incredible thing. So, you know, fingers crossed we don't end up with corporate overlords ruling everything. But at this stage anyways, we've got this like fun creative space where anyone can jump in, anyone can contribute to a project, try some stuff out. You know, might as well. While we're doing this again, you want to make sure that people like talking on GitHub about problems they're having and fixing things. Like those are the real heroes that are keeping the kind of like open source movement alive. So that.

You know, you've got some pretty consistent images, I would say, in here. But hey, experiment, get wild. Put some weird stuff in there, Do a longer video, try it out. Yeah, if you think you can help, jump in, by all means. Because the space definitely needs as many, you know, people working on it as possible.

Absolutely.

We've got our 3D video. So let's take a look at it. Took about 12 minutes there. 2,000 frames. It'll take a second to load it into the RAM here or maybe do tons of images like I believe to play the video. There we go.

And if I wanted full screen, I could just like navigate to the folder and pull it up. But we've got a full minute of video and you can see the 3D happening here.

I love the color palette of this one.

Yeah, yeah, it's fun. I tried to like throw in every frame, like a different image or every 10 frames a different image. You know, there's lots of different ways that you can play with this and try it out. Color correction. Correction. I kind of want to lower that. How close to get the colors of the input frame. Right. So let's actually bring that down and see if we can get some of these other colors that. Because we've got like some really wild colors in here. And some sci fi elements have some windows with planets behind them.

It's very contrasty too. It's got some nice contrast to it.

Yeah, like the like Michael Bay lens flare action going on there. I think a lot of that's coming because I put in the prompt, like astronomical. Right. Let's see if we can. Well, these, these are actually pretty consistent. Well, phenomenon or something like that. So it's got like, you know, galaxies and clouds and strain. So maybe instead we'll go four.

Strange things. But. But yeah, this is based off of a really big name prompt where he did this incredible one of like human. You probably saw this one. The like never ending Egypt and five try that out. But we will have to move on to Shin Obelisk exploration video. It's incredible. If you go into there and check it out in the. You'll see the prompt for. For this like amazingly good work on the prompt there. All right, so that's the. Some other stuff here because I know I'll just like nerd out about this stuff. Like, oh, this art's so fun. 3D video. Might as well do this, generate another one and we'll go over to our large instance. I spun up here.

Now can you clarify for me? We know what the difference between 2D and 3D is? What was the difference with a hybrid?

Yeah, I'm going to keep doing this now. I do want to show the other two hybrid. So really big name wasn't quite satisfied with video input. He had been talking about it and to him it didn't really do much. And I agreed it was like it was kind of modes which will take a little bit. So that we should probably.

Yeah. And 3D is one that when I was messing with rough around the edges.

The video input mode and it didn't really.

I stuck to 2D. I didn't even move into 3D because I wasn't ready to make a video with all of the various options that are available.

Like there was no control that there was no nothing. Right. So he wanted to bring in the video and then use like. Like I showed you on the mixer where on the video mixer that I have there to use those elements to like blend the input video well inside of 3D. Yeah. I'm going to try to see if we can get a large. With the generated prompt and make it so that you can control that parameter and you can sort of effectively really add the generated piece over top of the video input piece. And then he took some work that was pioneered by. Let's see.

Run diffusion. Can you kick someone off for me? This fellow, I forget his name. It's like S X E L A. Mm.

Oh, I know.

Yeah.

Alex. Alex is.

That's right. No, I'm just kidding. All right, let's see what we can get here.

Try a 2.1 select and launch. And he's got a Patreon where he does warp fusion, which is on a collab that you have to be on the Patreon to do. But.

So we'll see if we get something there. But.

We can go Here and he's got some brilliant stuff. He's just solo dev crushing it. Like I'd see like daily builds all the time. Time. The guy's like nuts. Like.

So is the Hybrid similar to what like Runway is trying to do with their Gen1? Is it kind of that similar idea?

Yeah, it is. I mean the Gen one really took it to the next level where they're bringing in things almost like instruct pics to picks and kind of talk about a 3D version because the 3D camera is incredible. Like I am, I am now we're seeing this sort of like interesting space come out where like Hybrid is like an early version of this stuff that's I think still really great to use. Gen 1's got this like very like Chris really blown away. And I will say I do want to give some shout outs here because the Deforum Discord is incredible. Like the everyone there is super helpful. They're all so nice and they get bombarded with professional like full replacement mode. Instruct pix2 pix and controlnet are sort of like different ways that you can break it down per frame and reimagine it. So a lot of people before.

You know, the Deforum tools came around, they're making videos by like just grabbing large questions nonstop. But the community's really come together and helped out there in a big way. So ounce of frames and then piecing them together like stitching them together with a video program to take all the raw images in a row and then stitch them together. They're still doing that with.

Control Net stuff right now.

Human. We should definitely have you talk about ControlNet maybe after the Hybrid piece because I'm so curious about what's going on there. But there is here shout outs I want to give are one Snack Pack. I saw you there in the channel. That's how I learned was like by looking at what Snack Pack does. They post their video where you can take. I'm in the output tab on Deforum. You can take frames and turn them into video right from a batch here. So I would just like, you know, put in my path, et cetera here. And then it would stitch them together. So you could do that for like control. Net.

Or they include the prompts you can go through, figure out how things do what they do. And it's incredible. Like big fan of Snack Pack.

All right. And we got a bunch of a bunch of fresh LG cards loaded up. We do have this like scaling back end so you know, we can using the Instruct Pics to picks.

Yeah. So let's jump into hybrid here.

This, this generate input frames. You have to do it once. But that's basically like doing what we were just talking about. Like, it's generating. It's taking all the frames from the video you put in and breaking them all down, splitting them all up and putting them in a full technically go to 10,000. It just, you know, we're. I guess our provider was a little caught off guard there.

But so snack packs. One of the shout outs, the other one I want to give a shout out to and we'll talk more about it later, is really big name Boulder. So you do it once.

And yes, I'm seeing a comment. Hybrid mode has been really confusing. It is a tough one, I have to say. There's a lot of complexity in even a small amount of settings. It's still very complex. So we're going to try it out. I'll show you some examples that I've done. Because they not only develop and add to the Deforum extension some incredible stuff, but they here, actually, I have a good example here that I can kind of pop up.

Whoops. But I wanted to take my glitch art that I do and.

Put it in here. So also post all their videos with, like, details of how they did it. So both of those creators are just incredible and definitely worth looking up in the disc. This is the glitch.

Oh, I think I might have just lost audio on you.

Yeah, I can't hear anything.

Oh, sorry, we're back. That one had audio on it. So maybe like, it switched focus to the. Oh, that might have been. It had, like, music.

But just go through their past messages and just see all the, like, crazy stuff that they've made because it's incredible here. Actually, I have a good example here that I can kind of pop up.

Whoops. But I wanted to take my glitch art that I do and.

Put it in here. So. All right, so we are in this medium one.

This is the glitch.

Still spinning up the big box. But let's jump in here. We're going to do.

Maybe we'll make it a little wide. That was the raw input video. And then here I've reimagined it. This is an example of the first example that I got of it working with Hybrid. You can see how it took that glitch stuff, turned it into an art nouveau painting. Like, you can see the colors, like the divider.

I don't know.

Let's try it out.

And glitch aspects turned into like faces and clouds. Really awesome stuff. Here's another example using that same one, but I like toned it down a bit. You can see there's like a figure coming out of the clouds. This one I used a custom model that I made.

So it can make some really incredible stuff.

And 20 steps.

Yeah.

I want to explain something real quick. So in the original video input, right. All you're doing is go to keyframes.

So we're going to go 3D mode.

You're taking your video and you're separating it into frames and then you're doing individual.

We are going to do wraps. So this is really important and especially.

Generations on each frame. And so what this hybrid mode is essentially doing is it's. It's splitting the video into frames, but it's also.

Like getting more information from the video. It's getting information on like how things.

When you're doing 3D camera work, you have to visualize that are moving frame.

To frame and it's like using that information in addition to the frames in order to like make these sort of warpy videos.

Yeah, yeah. Like sort of like how strength works by going from like one image to another. There's the way the 3D works is it is going through like it's picking up some of that information as well. Would you agree with that human?

Yeah, yeah, definitely.

Okay, cool. One of the problems I had with my video, which is like, it's really glitchy, like it's coming from like a tv, low res, like lots of noise, lots of like random stuff in there. So.

And making a depth map of what is active and then using that in a 3D space. So there's almost like consider that there's a vertical virtual camera and you're. When I was trying a lot of other modes, it wasn't really working very well. But this one was able to with the hybrid mode, like really come up with some creative ways of using it.

Oh, I think I might have just lost audio on you.

Yeah, I can't hear anything.

Oh, sorry, we're back. That one had audio on it. So maybe like it switched focus to the.

Oh, that might have been.

It had like music.

But that was the raw input video. And then here I've like reimagined it. So this is. Which were pretty awesome. This one was like a. More like a weird subterranean cave, ancient ruins sort of thing I had going on, which was. Which was really fun. And this is an example of the first example that I got of it working with hybrid so you can see how it, like took that glitch stuff, turned it into kind of an art nouveau painting. Like, you can see the colors, like the different glitch aspects turned into like faces and clouds. Training the camera on where to move in this 3D space. It's wild stuff. Ones were more similar inputs, but like, looking at. Doing more of like faces and portraits. In 1.5.

One of the things I found in my work is like, if I do a lot of Interpol, we did get our large box. Thank goodness. Let's. Let's do that in a large box instead and just relation or I'm working with hybrid mode. Sometimes I'll get this sort of like really awesome stuff. Here's another example using that same one, but I like toned it down a bit. You can see there's like a figure coming out of the clouds. This one I used a custom model that I made.

So it can make some really incredible stuff.

I wouldn't describe it any other way than jello. I got this like, sort of jello going on where, like, you know, we showed how it works on a.

You know, on a smaller box, a medium box, but.

Maybe make it wiggles. So that's the interpolation going overdrive. Because it's not like using the.

Information from the video, it's using the frames. So then the pixels are moving in a way that's not consistent with the way that the video is moving.

Love that.

Yeah.

I want to explain something real quick. So in the original screen, got to.

Be in the right one here.

10.46. And that's really important for hybrid video. So if we go back here, we're going to do this hybrid composite, which means it's going to be that original video and generated video on the same thing. We're going to use the first frame as the init image. Right. We want it to start with what the image looks like. Now this is where it gets interesting. This is where it's able to take in the hybrid motion and flow. It's able to take that information from the video on the movement in the video and transform it.

3D mode.

Cadence.

Into the generated video. That's where sometimes I'd get the jello, sometimes I get movement. That was beautiful.

Um, like video input.

Right.

All you're doing is you're taking your video and you're separating it into frames and then you're doing individual generations on each frame. And so what this hybrid mode is essentially doing is it's. It's splitting the video into frames. But it's also.

Like getting more information from the video. It's getting information on like how things.

This optical flow model, if you really nail the settings, is incredible. Like you can get some brilliant stuff. And a lot of what I showed you uses this optical flow model with dis medium or far and back. I guess we can put that back at 5 again. And I don't do much masking at the moment, but it's definitely possible. But I will say, unless you're putting a mask thing in, it's hard to get it working properly for me.

Maybe we'll do our moving frame to frame and it's using that information in addition to the frames in order to make these warpy videos.

Yeah, yeah. Like sort of like how strength works by going from like one image to another. There's like it's picking up some of that information as well. Would you agree with that human?

Yeah, yeah, definitely.

Okay, cool. So we're going to start with hybrid motion. None for this input video. But.

Now I will say just as a note before I jump into the big box and start like showing you all the speed and what it can. One of the problems I had with my video, which is like, it's really glitchy, like it's coming from like a tv. Just keep in mind that these other methods are really great for capturing motion. And so.

Do. If you go into the deform thing here, you can actually see. I'll test them out, try them out for sure. So let's make sure we have all of our other settings in place. So the. For the individual frames of the image that you're making or the video that you're making frames thing. Keep in mind that it ignores the max frames when you do a hybrid video. And it will only use the input video as the number of frames. So when you're testing this out, you don't want to sit around forever waiting for a generation. Make sure that you use a short video, like 10 seconds. Take a clip from the start of the video that you want to use, pop it in and then you won't be wasting a lot of time. I can't.

And that can really help you if we pop this open. That can really help you when you are making a new video. So this is also a folder that I delete quite often. Like I will go and or money doing this. So we are going to just go through this quickly. Max frames is fine. Cadence, you want it one. And the reason why is because hybrid video is trying to generate and diffuse off of Every single thing. So if you do less, it's going to have more jello. In my experience, it's going to be guessing and tween here. I will just go search video hit enter now and causing that, like, goop that's caused by not moving in motion. But if you're having every frame generate, I can see all the videos that I did right. Like, you know, this is. Again, I don't know if my logic is correct a little bit. Banging rocks sometimes with hybrid video to make fire. But.

Cadence one, I find is best. So we're going to use the 2D animation mode now with strength, you want it really high. Like, really high. This one we can go in and see. There's our video that we had before. So I will get all the videos that I want. But here you can see all the frames for everything that we've done so far. So it takes a bit. If you consider like Strength one. Like, nothing will show up except for. For, you know, what. What you put in.

Or. Sorry, what would I. How would you classify Strength one human if it's like.

Yeah, so I'm gonna potentially like flip the directions. But yes, please.

Essentially you have. You have two ends, right? You have best if you like, delete.

It when you're done using it and start up with. With a fresh one. But I will say that you can.

Scroll only random noise as an input to the generation, or you have just the image as input to the generation and then there's this entire spectrum in between. But essentially, like at 0.5, you have half random noise and you have half input image. And so what I think you're saying.

Right, is that down here and see in advance, like, oh, this video is looking like it's not going to be so hot. Maybe it's more like a hot mess. And then you can stop your video.

For these hybrid videos, you want to be closer to the image that you're putting in than you want to be to the noise.

Yes, exactly. Thank you. Worded much more gracefully than I could have. But the other way you can do this to reduce the amount of time that you're experimenting and testing things out. It's awesome to have you here for sure.

I'm going to turn up the config because I want it to be pretty close as you can do a much smaller amount of frames. And then to my prompt, which I'll enter in. So we're going to make this extra brackets here, we're going to make this input video, which I'll play in A second. We're going to make it comic book style.

Which should be pretty cool. I'm going to take a weird 80s movie trailer. I will say I showed you Shader Toy. I will say as well. Check out archive.org, go into video. There's a time, take a look at how it's looking in the first like 30 seconds or so and then try it again with a longer amount and again with a longer, longer amount and then kind of stuff here. Like I love archive.org it's amazing. And you can really find like a lot of Creative Commons video here that you can use. So I grabbed a Creative Commons that will help you cut down on, you know, server costs or time if you're doing it locally video and you can use it for hybrid video or art, no problem. But it will really help you just at the beginning, you know, make sure that thing is all good there.

Very cool. Actually, that brings up a question that popped up in the comments. This might be a question for Human, but any videos that we generate into forum, you do have the rights to use them however you want, commercially or otherwise. Correct?

Yeah.

So like low res, like lots of noise, lots of like random stuff in there.

So Deforum is distributed under MIT license, which means that you can what's coming.

Out looks great and then do it. It's nice because in Deforum you kind of know if it's bad, like pretty fast. Whereas when you're training you generally don't know until much later. By that time you've already trained most of it. So.

Yeah, so let's go into this and just go into some more settings here. If you want to do another giveaway or something, I'll.

Yeah, I actually want to pull Human onto the stream if.

If he's do anything you want with it. And you know, the people who make it, who make deform aren't liable. But.

Essentially the rights.

To generate or use any of the outputs from Deform for like commercial purposes, I guess.

Falls under some sort of like the licensing of the model itself.

So available now. So he I want to pull him on to.

That's completely okay with stable diffusion models. Gotcha. Like the stability released checkpoints of the stable diffusion models.

Gotcha now, I guess. Would there be some like gray area though? I guess if let's say you were to take a commercial video that somebody else created that you don't have the rights to and then sort of remix it with Deforum, then I feel like you might start getting into some gray areas there.

Right, Definitely.

Maybe help with some of the concepts and explain some of the stuff. So now we got Human on here. Who. Human is one of the original creators of Deform. He actually gave Deform its name.

When I was trying a lot of other modes, it wasn't really working very well. But this one was able to, with the hybrid mode, like really come up with some creative, creative ways of using it, which were pretty awesome. This one was like a. More like a weird subterranean cave, ancient ruins sort of thing I had going on, which was, which was really fun. And these ones were more similar inputs but like looking at, doing more of like faces and portraits. In 1.5, one of the things I found in my work is like if I do a lot of interpolation or I'm working with hybrid mode, sometimes I'll get this sort of like.

And I think it's really important to like.

Try to understand this stuff as best as you can because you know you'll, you'll be saving yourself from like accidentally doing something you're not supposed to.

Yeah. And you know, the nice thing is there is like some protection in the form of fair use laws.

He's in the default. He runs the deformed Twitter. He's inside the Deforum Discord. So he is the man.

I wouldn't describe it any other way than Jello. I got this like sort of jello going on where it, like it wiggles. So that's the interpolation going overdrive. Because it's not like using though. Just make sure you understand those rules if you are going to be using this stuff. When it comes to art though, I mean, I make music. I'm like make hip hop, sample things. So, you know, my understanding of things is a little bit different. But I make it for personal use. I'm not releasing it for tons of money or anything. It's just for fun. So. So yeah, I'm. I have no qualms using this. This movie is amazing, by the way.

Over at the forum, so thought it'd be cool to bring him on and he can help answer some questions. As we're going through.

Information from the video, it's using the frames, so then the pixels are moving in a way that's not consistent with the way that the video is moving through this and.

Maybe help clarify some of the concepts that you're. You're walking through as well. So thanks for, for joining us today, Human.

Thanks for having me. It's a pleasure.

And that's really important to be here.

And yeah, like lots of questions in the live stream chat.

I've been absolutely twisted Strange Sci fi from the 80s Liquid Sky. It's like half.

Fashion 80s nightclub aliens come that have to like have sex with you to live or something like I forget what it's like. Very, very strange.

But trying to answer them. There's a lot going on under the hood.

I liked this because it's just like face center. Thought it'd be a good example person.

You know, I think you know, at Deforum, you know we're. We're a community and we want to give people.

A lot of people in the beginning are maybe not doing weird abstract video stuff like I am. So they want to get like maybe a person or a style. So let's take a look at that and pop it in here. I popped it in the video in.

It past people access to as many tools as we possibly can. Unfortunately that means that there's a ton.

Of settings for hybrid video. So if we go back here we're going to do this hybrid composite which means it's going to be that original video and generated video on a few other settings that are really important. You'll see. I noticed the noise anti blur. So this is something that really big name is not a fan of when I saw his tutorials. So reduce it to zero.

But we just try to make to.

Get a better image because you're not doing some of the things that in normal deforum the same thing. We're going to use the first frame as the init image. Right. We want it to start with like what the image looks like. Now this is where it gets interesting. This is where it's able to take in the hybrid motion and flow. It's able to take that information from the video on the movement in the video and transform it.

Into the generated video. So that you'd be doing like with tweening or so maybe. But the anti blur is going to just get in the way for seed. We're going to do iterative and.

Strengths already really high. I think we're pretty much ready to go. We've got our prompt now.

One thing, everything available. So you have complete control.

Now. Is there anything that's come up so far that maybe you feel you want to clarify a little bit more or.

Bring maybe some more attention to? Because I noticed we were in the, in the. In the background chat here. I've got my, my head on a swivel here. I've got, you know, chat over here do.

Since we're using this, we're not going to click use init. We are going to increase this because it's going to use that initial.

Initial image from the video as the INIT image. So we're going to increase the strength so it shows up.

Yes, fan art, definitely what we're doing.

And again I'm trying to, you know, look at the cameras and pay attention to revolved what he's talking about. So I'm. I'm kind of looking all over the place. But is there anything that sort of came up so far that you want to dig a little deeper on or clarify at all?

Yeah. So one thing that I think is really important to understand is sort of like how these diffusion and this input.

Frames, you only need to do it in the first. Now this is where there's a lot of important settings under hybrid schedules. So comp Alpha. So alpha means transparency. There's no tool tips for these. We're going into the unknown here. We're going to step out into this, you know, area where it's a little bit shakier ground.

But we're going to have models are working especially like in the context of these stable diffusion animations.

Some fun with it. If you do have a mask blend, then this will be used contrast. You don't want to use this unless there's certain cases like it's the light in the frame. So you could use it to make like a transition. You could like brighten it up. It's going to use that same like.

So I guess like the. The key thing to understand is that this is essentially a like an initial image animation. Right. So the diffusion model will take when.

You'Re going from one frame to another, it'll like slowly increase. So you could do like a flash, you could do like a darken like end scene. So very important to keep that in mind. Maybe you want to like at the end of your video it slowly gets darker and then you can use another clip to go to another scene. I saw some guy earlier in the comments who wanted to make a Netflix video on Deforum like full length noise.

And with a text conditioning like produce some image.

And instead of starting from noise we can.

You're going to need to do a lot of separate clip renders on this. So get your transitions in. Now with this alpha. This is the blending of those two videos. So sorry, not of the two videos. This is the blending of the input video and the generated piece of it. So you really need to make sure that this is closer to a lower number to get half. Half basically when I was doing the.

Start from an initial image, and we can still use that same conditioning to.

Sort of like these videos. This is actually very low. So this was like point three, so you can hardly see any of the original video in it. But when we're looking at some of the other stuff, this one was also actually quite low. So just keep in mind that you're going to want to try different settings.

Oh, and here's the modify and augment the starting image. And essentially, like all these animations are. Is we're taking the output.

I mean, everyone wants to see the Jello. This is the Jello. So I was making some, like, underwater stuff with it, but that's what high frame interpolation looks like. Okay.

So we're going to lower that comp alpha and.

Of a generation, and we're putting it into another generation. And we sort of like have this feedback loop and we have clever ways to sort of augment and like translate, rotate, zoom the output of one.

I think we're good. Let's give it a go. Fingers crossed. All right. Seems to be cruising.

We did generate another video here with the 3D, so let's check it out.

Give it a second to load there.

Should we do some giveaways?

Yeah, let's do. I got another 50, a 20 and.

Four tens to give generation. Put it into the next. And we sort of like create this illusion of like a camera moving through 3D space or 2D space. And then, you know, honestly.

So we got.

Let's do another 50. Why not?

Yeah.

All right.

Should we do another keyword then? What keyword should we have them throw in?

Hybrid.

All right, let's see. Hybrid.

Let's see. And then also we do. You know, it'd be good to do some. Some Q and A here too. So maybe actually after we do the hybrid, we'll do the Q A, because otherwise I'll probably miss the questions.

Yep, definitely. And then I'll just run through really quick as well. We'll just. I know we've gone like. We've gone quite a while here, but.

I think there's like, you could just keep going, right? There's. There's so many things you can do.

So it's kind of like.

It's some really important tools that we can just like, maybe throw in at the end, like a few mentions for some. Some cool tools in the kind of.

Like the sort of stable diffusion image to image. But it's just kind of. It makes one image and then sort of takes that image and then Makes another one that's based off of that. And that's why each sort of thing blends so nicely, because the next image is sort of based off the one.

That was ecosystem cool. Can I full screen this? No, I think there's a way.

Anyways, that little square there next to the three dots doesn't do it.

It's grayed out now. It's like. Yeah, it doesn't do it in here. You can see we've got the 3D stuff I really love. It's like. It's not perfect, the depth map. And I think Control Net is gonna swear sometimes I'd get the jello, sometimes it get movement that was like, beautiful, like previously made.

Yeah, exactly. I mean, it's. It's exactly image to image. So it's image to image, but you're just taking one output and putting in into another image to image.

And then really change the game for 3D.

Because it's going to use like a much better depth map, you know.

Doing those little changes, right, like move.

I've actually seen lots of other new depth map models coming that have more detail in them as well. So I'm guessing SDXL and other things are really going to take this 3D.

Realm into a new.

The output or zoom in to the output just like a teeny bit space.

All right, so we have a winner. We have Lionel. A L A I N O L. You just won $50 credit to run diffusion.

You do this enough times, you create enough frames, you stitch them all together into a video.

This optical flow model, if you really nail the settings, is incredible. Like, you can get some brilliant stuff. And a lot of what I showed you uses this optical flow model with dis. Medium or far and back. I don't do much masking at the moment, but it's definitely possible. But I will say, like, unless you're putting a mask thing in, it's hard to get it working properly for me.

So we're going to start with hybrid motion. None for this input video. But just keep in mind that these other methods are really great for.

Let me make a quick note of that.

Big bucks. All right, Lionel, expecting a movie. Full movie now, no pressure.

And then I'm going to roll it one more time for a 20.

We got Gert K. Nielsen. You just won 20.

All right, Gert.

Now all of a sudden you have a zooming video.

Very cool. So this is actually a question that I had for you. Nathaniel's asking it here as well. But what. Where did the name deforum come from. I know that was.

All right.

I got a. I got a handful of $10 left and I'll. I'll save those for towards the end. But now I think would be a good time if you do have questions while we're waiting for some of this stuff to process, throw your questions in for. For human and for revolved and we'll make sure we get them answered here.

This one ended up cool. Cool.

You know, you were the one who came up with the name. So I'm curious where the. Where the name's from. Whoa.

This is like some of the settings. I messed around with it a bit. So you can see it's really getting a bit wild.

Yeah. So I actually started sort of messing around in the AI space in I guess like late for now. Woo.

Very fun.

That's cool. Somebody was asking this is a question that popped in earlier. How do you add the music to it? I know there's a feature in there to be able to add some music to the videos as well.

Yep. In this output you can specify a soundtrack to add to it. Capturing motion. And so.

Test them out, try them out for sure. So let's make sure we have all of our other settings in place. So the frames thing, keep in mind that it ignores the max frames when you do a hybrid video and it will only use the input video as the number of frames. So when you're testing this out, you.

Don'T want to sit around 20, 21. And you know, before I found all these amazing.

I personally like doing it after the fact because.

I can actually line it up to the video a little bit better. Which brings me to talk about a few cool things that you can do with music. So let's just go over those really quick.

There are a number of different tools for it. So they're third party. So here's an example of keyframe generator.

Using communities on Discord. I sort of like had this idea in my head that I wanted to figure out a way to like program creativity. And that was like a really stupid idea. But.

Like I started exploring and experimenting with code and the first two folders I made.

When you know, starting my.

So you can. This will actually parameterize and give you a math equation or this. You can see that the string is up there.

Or sorry, down here there's a string. So that saves you a lot of time if you're trying to figure things out. There is an audio keyframe generator link and forever waiting for a generation. Make sure that you use a short video like 10 seconds. Take a clip from the start of the video that you want to use, pop it in and then you won't be wasting a lot of time from there as well.

So you wait, go back real quick. The string that you just generated, what are you actually generating that for again? Just. I want to.

Oh. So let's say I wanted like, let's say I had 100 frames and I.

Wanted to adjust my AI journey was. Was human and deform not spelled the way that they are.

The strength parameter over the course of that.

Ah, okay.

So now I've got my idea of what I want this to increase to 30, then 64, then 77 now.

But I've sort of had these names sort of from the beginning, you know, or that.

That's the frame number, I guess. But if you come down here, you can see that's the keyframes and I could pop that into the quote. You keep it in disco format, but not into the. Into the. Into the parameter. Whatever parameter I want could be X axis movement, Y axis movement strength, hybrid schedule, whatever it is.

Deform sort of just worked and it fit and.

It'S a combination.

That's really cool.

Yeah, it's great for getting wild things really fast.

Huge time saver.

Yeah. Now that's an early one. Here's a really wild one. So this one's more for like beat generated stuff. So if you know the BPM you can like pop it in here. 120 is really popular for like house.

And stuff like that of design in forum. But also it's like a play on words like you know, with this AI and or money.

I'm doing this. So we are going to.

Just go through this quickly. Max frames is fine. Cadence, you want it one. And the reason why is because hybrid video is trying to generate and diffuse off of every single thing. So if you do less, it's going to have more jello. In my experience it's going to be guessing and tweening and causing that like goop that's caused by not moving in motion. But if you're having every frame generate, this is. Is again, I don't know if my logic is correct a little bit. Banging rocks sometimes with hybrid video to make fire. But.

So obviously we don't want it that fast. But you can also change the frame rate. So since we're on 15 could be like something like that stuff.

We're like deforming these images and augmenting them with AI. So that's really like how the name came together.

120 frames. So the sync rate, this will be really familiar to music producers and stuff. And then you can actually. This is stuff that I do on my synths all the time, where I'll modulate it with another wave so then the waves will change and collide with each other and create something new. But you can also load a file here if you wanted. We could load.

Gotcha. Very cool. Very cool.

Cadence one, I find is best. So we're going to use the 2D animation mode now with strength, you want it really high. Like, really high.

Well, so Revolved was mentioning another giveaway. I do want to say something.

Some music. Lock in the tempo and frame rate and have it, like, automatically generate this real quick.

I know it was Judge works, I guess when I was messing with Nightbot the first time around, and it didn't work work. He won one of the things. So I'll make sure that, you know, he gets one of the prizes as well.

And then, boom, we've got down here all of our frames.

Oh, that is so cool.

Beat generated.

Yeah, that is so cool. So now. So now the video that it spits out from Deforum, like, some of the transitions and the movements or the changes in angles or whatever you're doing will change with the beat of the music.

Yep. And it will be any parameter you want. So you could have it like camera movements. You can have it like.

So I just want to say that real quick before I go any further and, you know, somebody gets upset with me on that.

If you consider, like, Strength one, like, nothing will show up except for.

You know, what. What you put in.

Or. Sorry, what would I. How would you classify strength one human if it's like.

Yeah, so I'm going to potentially, like, flip the directions, but.

Yes, please.

Essentially, you have two ends. Right. You have.

Only random noise as an input to the generation, or you have just the image as input to the generation, and then there's this entire spectrum in between.

So you won 1, 2. So very cool. So with. With the forum, let's.

If, you know, get to a certain point and then, like, change it at a certain frame, put in some different movements in there. You can make it, like, beat. Maybe like beat, like generating, like, it'll go like faster generations and then slower generations depending on the beat and stuff like that. So a lot of really, really awesome stuff.

Just for like, a real practical example, can you show us where you would copy from? Here we go back over to some of the settings and maybe start messing with the 3D. I kind of want to dig into the 3D. And it would be cool to have both, you know, revolved and human on, you know, on voice chat here, as we're going through it to time, where you would paste it inside of deforum.

Yep. So copy keyframes or formula. So you see here we get the complex math version.

Then I just go back to here and let's say down here, I wanted to make it the strength parameter. Boom.

Very cool. You could set that on, talk through some of what's going on as we're like the x axis movement or the y axis movement or the zoom or any of that.

Exactly.

That math function is exactly what you're seeing up above the graph. So it's, it's. It's an equation that describes exactly this.

That is cool. This. That just blew my mind right there. These last two websites, messing with some of the 3D stuff because, you know, 2D to me was fairly straightforward when I was messing with it. There's only, you know, so many motions you can do with it. When you start getting into 3D, you know, there's quite a bit.

But essentially like at. At 0.5, you have half random noise and you have half input image. And so what I think you're saying, right, is that for these hybrid videos, you want to be closer to the image that you're putting in than you want to be to the noise.

Yes, exactly. Thank you. Worded much more gracefully than I could have. So definitely it's awesome to have you here for sure.

I'm going to turn up the config because I want it to be pretty close to my prompt, which I'll enter in. So we're going to make this extra.

Bracket, more knobs and buttons you can press. And that's where it was starting to get a little here.

We're going to make this input video, which I'll play in a second. We're going to make it comic book.

Style, but over my head. So what we're seeing on the screen right now, this is a 3D generation.

Yeah.

So it's something showed me are just like, holy crap, man, that's so cool.

Oh yeah. Now there is extra hard mode for super advanced.

Which should be pretty cool. I'm gonna take a weird 80s movie trailer, folks. I will just mention it, but there's this tool called Parsec that they brought in recently, which is completely nuts. Um, and vam of my earlier work in the 3D, you can really see the depth and like how the camera is moving around. I actually Want Very, very intense. Where is parsec here?

I haven't used it much at all.

Motion. That's right. Now, where is it human? Where is Parsec?

I'm not sure. They. They rearrange things.

They do rearrange things all the time. Wanted to bring this one up because there's some interesting stuff with it. So I used an init image here, right? So this is some of my video art. And then it's.

Sdparsec. See the parsec section in the keyframes tab? Guided images.

That just disappear. Oh, oh, you know what? It, like kind of goes off and you see these, like, lines, you know, actually human, I was kind of curious. I know this one comes up in the discord, but I think we rolled out a fix to deform prior to the show and it might have disabled parsec. But I'll show you what parsec is because I will say, I showed you Shadertoy. I will say as well. Check out archive.org, go into video. There's a ton of stuff here. Like, I love archive.org, it's amazing. And you can really find, like a lot of Creative Commons video here that you can use. So I grabbed a Creative Commons video and you can use it for hybrid video or art, no problem. Those lines are kind of when the camera goes out of bounds. And this is. If you thought that was crazy, wait till you see this stuff. So this is like mega. A few different ways. I think it's like you adjust the strength setting, you can kind of tone those down. I mean, I kind of like the effect, but complicated, heavy math stuff. But you can basically put everything in here, you know. Do you know why that's. That's sort of happening?

Well, so I, I think these are.

Just like settings, save settings, audio analyzer.

Artifacts of the diffusion model. And I personally love them. I kind of try to accentuate.

And you can do a lot of crazy stuff, like more.

Working with frames and time. And you can be like, okay, at this time, I want to change this.

You know, people as much as I can, just because they're sort of like this emergent behavior of these models. And you know, they're not, they're not designed.

Ask me that a lot, like, how do I do frames versus time and that kind of stuff. Like here you can see a graph. That thing is all good there.

Very cool. Actually, that brings up a question that popped up in the comments. This might be a question for human, but any videos that we generate into Forum, you do have the rights to use them however you want, commercially or otherwise. Correct?

Yeah. So like Deforum is distributed under MIT license, which means that you can hired necessary, but they're certainly cool.

We'll show you.

Frames and where you're at. So I don't do much of this, to be honest, in my like work and what I'm doing with video art, like I find like a lot of the automatic beat stuff to me doesn't work with the art style. Yeah, I agree. You can get some really awesome textures with it. It's. It's a ton of fun that I do. So I'm just more like. I like to turn knobs at the speed that I'm generating stuff. And when we get to.

This one's more of like a sideways sort of one, but still use.

Do anything you want with it. And you know, the people who make it, who make Deform aren't liable, but.

Essentially the rights.

To generate or use any of the outputs from Deform for like commercial purposes, I guess.

Distilled. I heard someone mention at some point the 30 frames per second.

Stable diffusion stuff. So we'll be able to. We'll get some crazy like you'll be able to do this live like with knobs and it's going to be nuts.

Wow.

So here's our.

The depth map here a little bit. You can see there's the camera like turning and it kind of twists everything around. I really love it because it kind of gives us like zero gravity feeling.

Hybrid one which came out, you know, like maybe it's a bit. It's a bit flashy. It's going to be a little bit like glitchy, but you can see like a lot of her. The original video came through and it's a bit crushed. So maybe we can try this a little bit lower and adjust. You know, you're like floating through space and there's some of the artifacts and you see here, they got interpreted as desert. The.

Strength a bit.

Not sure higher or lower actually at this point 8, 6.

But we'll give it another dunes, you know, another go here. See what we come up with.

Was there other. There's these little flashes that come from. This is actually from the frame interpolation that I did afterwards to make it go tools I wanted to mention. Oh yes. So.

There's a fantastic docs thing here.

This is prior to frame interpolation being.

Added into Deform falls under the licensing of the model itself. That's completely okay with stable diffusion models. Like the stability released checkpoint. So of the stable diffusion models.

Gotcha now, I guess. Would there be some, like, gray area, though? I guess if. Let's say you were to take a commercial video that somebody else created that you don't have the rights to and then sort of remix it with the forum mention.

So this guide is pretty good. There's also docs on the discord, but this one goes over a lot of the settings in detail.

I used a free application called Flow Frames or I think they might have like a donation or something like that. I highly recommend it. We can, like, probably pop these in the video description. Hey, Matt.

Yeah? Any sort of links that we mentioned, we'll try to get into the description once. Once we're done recording here.

Yeah. All right, cool. Should we go to then?

I feel like you might start getting into some gray areas there, right?

Definitely. And I think it's really important to, like.

Try to understand.

But Flow Frames was what I used and it does kind of make things a little bit flickery in that area. But otherwise, I mean, you get this, like, nice smooth questions and I'll just keep working on stuff while we're chatting.

Yeah, if, if you have any questions, pop them into the chat and we'll. We'll do some Q and A here while we got human. And revolve both on this stuff as.

Best as you can because, you know, you'll.

You'll be saving your animation, which is really cool. So.

So sticks is.

Yeah, we can dive into the settings. I just kind of wanted to show off some of this other work while.

We were chatting, asking how do we find Parsec? So what when it is actually available? It's.

It's like on the bottom of this keyframes tab. It's like down here under Parse, perspective, flip, I believe. Yeah.

Let's see. David is asking when you hit Generate, where's the processing happening on your local machine? This is awesome. So the initial art that you were.

Doing, that's all the stuff from, like, accidentally doing something you're not supposed to.

Yeah. And you know, the nice thing is there is like some protection in the form of fair use laws, stuff that's.

In your background there. You make it with all the. All of your. I'm not quite sure what. You know, this is on Run diffusion. So Run diffusion is actually cloud gpu. So it's. This is all happening in the cloud.

Yeah, which is great because if we were trying to crush these videos locally, we might be like glitching out on the live Stream and stuff like that.

Oh for sure. In fact this is so tinkered thinking is asking can you explain what a seed is? And this is actually something. All the crazy wires and ca. Cables and stuff going all over the place in the background. But that's, that's built with that stuff thing that I get asked quite a bit, especially when I'm talking about things like mid journey and stable diffusion is what the hell is the seed? Maybe one of you guys could probably explain it better than I can because.

Yeah, the initial image. Yeah. Actually if you change to my camera here.

This is the video synth that I have running right now. So then over.

I've tried a few times, I could explain it.

Yeah, please.

Yeah.

So.

When you're making like a diffusion, right you're starting from noise and you're sort of like.

Diffusing the image from that starting noise.

But that starting noise can be sort of like any rank here.

I can like tweak these settings to get like this is a random pattern.

You want and to sort of keep the model scientific. What they do is they use like.

A suit oscillator and then if I.

Want I can change a random number generator which takes a number or a seed as an input to sort of control the randomness of this noise generating process.

So just make sure you understand those rules if you are going to be using this stuff. When it comes to art though, I mean I make music. I'm like make hip hop, sample things. So you know, my understanding of things is a little bit different. But I make it for people personal use. I'm not releasing it for tons of money or anything. It's just for fun. So. So yeah, I'm. I have no qualms using this. This movie is amazing by the way. Like the blending between layers.

And so when they train the model they're training it on random noise.

Wow.

Or I can. This one uses like shaders which you can. Shaders are amazing. You can get.

When you're inferencing the model you can specify a seed number which will essentially determine what that noise pattern will look like.

Does that sort of explain it?

Yeah, no, that's great. That's a great description of it. You know my knowledge of what is absolutely twisted strange Sci fi from the 80s Liquid sky, it's like half.

Fashion 80s nightclub aliens come that have to like have sex with you to live or something Like I forget what it's like. Very, very strange. But I liked this because it's just like face center. Thought it'd be A good example person. You know, I think a lot of people in the beginning are maybe not doing weird abstract video stuff like I am. So they want to get like maybe a person person or a style. So let's take a look at that. Like, look at like programs like vdmx and there's, there's like Shader Toy. Actually if you're looking for like seed is mostly comes from like playing games with random map generators. Yeah. You know, like back like Minecraft.

Exactly.

In a seed. And if you put in the seed, you get the same thing. It's like a point in the randomness that you can, you can choose from. So. So if you choose the same seed, that's a way to get consistent generation cool stuff to run through Deforum. Check out Shader Toy because there's a. And pop it in here. I popped it in the video init path ton of cool like applications that you can do to make like awesome little weird animations and stuff like this. And there's a whole community around making a few other settings that are really important. You'll see. I noticed the noise in these things. You can also run videos through shadertoy. If you do a fixed seed and then you specify it. Then when you're creating a deformed video, if you're like, oh, you know, I kind of like this one. I want to get it like pretty close, but maybe I want to change like the strength or the frame rate or you know, anything like that. You can go back to that same seed because you use fixed and then carry off from where you were or to give extra effects. So.

Wow, that's crazy. That's so cool. Yeah, that's like a whole world I've never even dived into before.

Right? Yeah, there's like, there's a ton of stuff out there in the video. Redo it.

Yeah, the, the corridor crew video that you mentioned earlier as well, where they're explaining how they did that whole anime thing. They actually have a, like a, a good like 4 minute explanation of how the scene seed works and how the, you know, it starts off with noise and basically they're, they're making something that's really noisy and then removing the noise down to the picture they're trying to generate and the random world.

And it's neat to see like areas where it sort of impacts with AI anti blur. So this is something that really big name is not a fan of when I saw his tutorials. So reduce it to zero.

To get a better image because you're not doing some of the things that in normal deforum you'd be doing, like with tweening or so maybe, but stuff. So maybe switch back and seed, I.

Guess, kind of generates that initial noise that they're then sort of taking away to get the image. Their explanation was really, really good. So I highly recommend that. But yeah, the only reason that the term seed clicked with me was also I've done a lot of Minecraft in the past.

You get a.

You get a level, you get a random seed for the level. You could give somebody else that seed and they'll get the exact same work.

Just got. I've got shader toy on the thing here. You can see that there's like different fun things and then the codes here. So you can actually like go in and be like, oh, I'm going to.

Change world that you played on. So.