Stability AI released Stable Video Diffusion, an AI model that turns still images into short videos. SVD generates 14 frames, SVD XT generates 25 frames, both at up to 30 fps. The model is available on GitHub and Hugging Face for research use.
November 28, 2023.
14 frames per video.
25 frames per video.
3 to 30 frames per second.
256×256 pixels.
I've been really excited about Stable Diffusion, the AI Image generator from Stability AI. Now they've launched something even cooler. Their first foundation model for generative video named Stable Video Diffusion. And it's incredible. I'll explain all about this new AI video generator in this video, how it functions, its capabilities and why it's making a big impact in the AI and tech world. Stable Video Diffusion is a top notch AI tool for making videos. It creates lifelike and captivating videos about anything using just a bit of text. You can make videos out of still pictures too. For instance, take a photo of a landscape and it can turn it into a video with moving clouds, trees and animals. Or use a person's picture and it can make a video where they talk, smile, or show different facial expressions. But there are two types of stable video SVD and SVD XT SVD the basic version can make 14 frames per video and you can adjust the speed from 3 to 30 frames per second. SVD XT the advanced version can do 25 frames at the same speeds. Both can create videos in 256x256 pixel resolution and the videos are in MP4 format. You can pick the video quality from low to high based on what you like and the capabilities of your computer. SVD actually works by using technology similar to original Stable Diffusion model. It starts with a noisy picture or video and slowly makes it clearer and more realistic. Neural network learns from training data to do this. Now there are three main steps in training Stable Video Diffusion. First, it learns to create images from text descriptions using a big collection of text and image pairs. This helps it understand the meaning of the text and what the images look like. Next, it learns to make videos from images using a bunch of video clips. This teaches it about how things move and change over time in videos. Finally, it gets better at making videos in specific areas like landscapes or animals. This improves the quality and variety of videos it can make and that's why it actually beats older models from companies like Runway, Pika Labs, Meta, Google and Adobe by being better in quality, speed and versatility. Plus it works with a broader range of text and image prompts and can make videos in different frame rates and qualities. It's also quicker and works more efficiently thanks to its unique diffusion method and well designed structure. But even SVD isn't without flaws. Right now it's only for research and isn't ready for real world or business use. Stability AI the creators say it's still being worked on and they need more feedback and ideas. They also caution that it might create content that's not okay, like violence, nudity or hate speech, and they're not responsible for any problems that come from using the model. It might struggle with. Very complex or abstract videos could show bias or unfairness. This means it might create videos with stereotypes or negative views about certain people or topics, which could lead to problems like damaging reputations or spreading false information. Now if you want to try Stable Video Diffusion Stability AI has shared the code on GitHub and you can find what you need to run it on their Hugging Face page. They also have a waitlist for a new text video tool on the web, but to use it you can need some tech skills and a strong computer setup like a good GPU or cloud service. Without this you might face slow performance or errors. Stable Video Diffusion is part of stability AI's wide range of open source models covering images, language, audio, 3D and code. They keep updating their models and welcome community feedback. They're also planning more models like one that can make videos from any text. This is actually a big step in creating videos with AI and opens up new ways to make and use content, but it also makes us think about the ethical side of AI videos and how to use them responsibly. Stability AI reminds us that AI is a powerful tool that can be used for both good and bad. Have you actually tried Stable Video Diffusion? What kind of videos did you make and how did they turn out? Let me know in the comments if you like this. Please like and subscribe for more AI content. Thanks for watching and see you in the next one. SA.
