Mochi 1: The BEST Open Source Video Generation AI Yet! (Genmo AI)

Today, an exciting new step in video generation has arrived with the release of Moki 1. This open source model is pushing the boundaries of what AI can do in video creation, showing impressive advances in how smoothly characters move and how well it follows your prompts. Mochi is designed to be accessible to everyone, whether you're working on personal projects or something commercial. Now, what's great about this, this is not just another AI announcement. If you're actually curious to try out this model, genmo has launched a free hosted playground where you can experiment with this model. And of course, if you want to dive deeper into exactly what's going on, the weights are available on Hugging Face. Now, what's interesting about this is that on their blog they've actually stated that their goal is to unlock the right brain of artificial general intelligence on or AGI. Just like the right side of the human brain is associated with creativity and imagination, Genmo wants to bring those qualities to AI. Now, Moki 1 is the first tangible step towards this vision, enabling AI to not only generate videos, but act as a immersive world simulator capable of imagining anything. Capable of imagining anything whether it exists in reality or not. Now, by focusing on creativity, genmo aims to create AI that can visualize new possibilities, tell compelling stories, and bring imaginative ideas to life in ways that were previously out of reach. Now, the model is clearly stunning. We've never seen open source perform at this level before, but genmo did something rather different. You see, today there is an enormous gap between video generation models and reality. The current models that we do have, and this is a problem I've spoken about on several different occasions, is that they often struggle with making movements look natural and accurately following the user's instructions. These are two key areas, motion quality and prompt adherence, where videos are still falling short, which results in video outputs that seem jerky or or fail to match what the user envisioned. Now, Moki 1 sets a new best in class standard for open source video generation and also competes with, well, leading closed models. What we do have is the 480p preview of Moki 1, and this version excels in the following areas. So we do have prompt adherence. Moki1 actually demonstrates exceptional alignment with the prompt that it is given, meaning that the videos it generates actually closely match the instructions provided by the user. This allows for detailed control over elements like your characters, your settings and the actions. And to ensure this high level of accuracy, Moki 1 benchmarked using an automatic metric, a vision language model similar to the approach used by OpenAI with Dall E3 acts as a judge to evaluate how well the generated content matches the prompts, and they used Gemini 1.5 Pro to evaluate how these generated videos performed, ensuring they're consistent with the user's intended descriptions. For those of you that might not understand how good this new open source model is, we can look at the prompt adherence leaderboards, take a look at Open sora, pyramid flow, PCoLabs, Runway ML Gen 3, even cling Luma Dream Machine. And at the top the new leading model is genmo Moki One Preview. I think that deserves a round of applause, although there aren't enough people to do that, but being able to surpass all of these other models in such a short space of time with an open source model shows that you can still catch up to the leading labs even if you are a different team. This is where you take different routes, such as focusing on prompt adherence and actually ensuring the user gets exactly what they want. And we can see by this graph that this is exactly what occurs. Now this also performs well in the motion quality. Machi1 also delivers significant improvements in motion quality. The smoothness of character movements in generated videos has been pretty challenging for most AI models, which leads to unnatural or robotic looking actions. And Moki 1 addresses these issues by creating fluid, lifelike motion that enhances the overall realism of the content, making it more engaging and visually pleasing. And when we also once again take a look at the motion quality Elo score, we we can see that once again Genmo's Moki 1 preview is right up there with the likes of Kling managing to surpass even the very infamous Runway gen 3 and the Luma Dream Machine and even Kling, which is rather surprising. Which means that Moki 1 and the Genmo team have done something absolutely incredible. If you would have told me that an open source team was about to deliver a new video model to that could potentially surpass the rivals in the industry in terms of prompt adherence and motion quality. I would have said there's absolutely no way that that could happen, but we're seeing once again today that the video arena is being constantly challenged with more and more competitors and oftentimes this results in a much better experience for the user because now you have a lot more choice. Now, some of the results might surprise you. You see, Moki 1 produces video 30 frames per second, which helps create a smooth visual experience. The videos that Mocky1 generates can last up to 5.4 seconds and they maintain their temporal coherence, meaning that the motion flows naturally from one frame to the other without abrupt jumps or inconsistencies. Now they also have realistic motion dynamics and physics simulation. Mach E1 simulates realistic physics such as fluid dynamics, which is the movement of liquids, fur and hair simulation and natural human actions. This makes the animations of any characters look more lifelike. For example, if there is a scene that involves water or an animal with fur, Moki1 ensures that these details move in a realistic way, adding a layer of believability that crosses the uncanny valley, a point where AI generated visuals become so realistic that they start to evoke an emotional response from the viewers. Now, Moki 1 focused on the motion quality. When assessing Moki 1's performance, human evaluators were specifically asked to concentrate on the quality of movement rather than the details of of each individual frame. They used criteria such as interestingness of the motion, how realistic it seemed, and how fluidly it was portrayed. And of course, to measure that performance, the ELO scores were computed using a protocol similar to the LMSYS Chatbot Arena. And that kind of scoring is often used in gaming and ranking competitions, which basically means they just put these models side by side. And of course, Moki managed to come out on top in most of these scenarios. Now, I think this model is absolutely incredible. They've managed to do something that most people didn't think was even possible. And I think a lot of that can be put down to the architecture that they used, which is rather fascinating too. So when we actually take a look at the architecture that they used, we can see that Moki1 actually represents a significant advancement in open source video generation. It uses a massive 10 billion parameter diffusion model based on an architecture called the asymmetric diffusion transformer, which is asymdit. And what that tells you in simpler terms is essentially that this model is incredibly powerful because it has so many parameters, which are basically the tiny settings that helps this model understand and generate video content. And Asymdit is a new kind of architecture which is designed to make the entire process more efficient. Now, Marqueeone is entirely built from scratch, meaning that it's a brand new system and not just an upgrade of something old. It's also the largest video generation model that has been openly released, and the design is simple enough that developers can tweak or hack it to fit their needs. Now, of course, running these models can take a lot of computing power, so genmo focused on making Machi one as efficient as possible. Alongside Moki one, genmo is also releasing something called a Video VAE Variational Autoencoder and the VAE is quite important because this compresses the video information down to a much smaller size, which is 128 times smaller to be exact. And it does this by breaking down the video spatially and temporally and reducing the complexity. This allows people to use less computing power to run Maki one, making it much more accessible. Now, if you want to know how Asymdit actually works, this architecture processes both the user prompts and the video tokens in a streamlined way. And it focuses a lot of its processing power on understanding the visual part of the video rather than just the text. It uses something called the multimodal self attention, which means it can look at both the text prompts and the video content at the same time to understand how they should work together. This is similar to how stable diffusion 3 works, but with one key difference. Mock 1 gives a lot more focus to the video part. By having more parameters dedicated to the visuals, the model ends up being better at generating realistic and detailed video content. Now, interestingly, many video generation models use several pre trained language models to understand prompts, which can be quite complex. However, MOQ1 simplifies this by using a single powerful language model called the T5XXL to handle all the prompts. This makes the model much more straightforward and efficient while ensuring it can still understand and generate based on user inputs effectively. Now, Mocky one is designed to handle a very large amount of video Information, up to 40,44,520 video tokens at once to be precise. A token is basically just like a small part of the data that makes up the video. And to make sure it knows where each piece should be, Maki 1 uses a technique called learnable rotary position embeddings which can be extended to work in three dimensions covering both space and time. And this helps the model keep track of everything that's happening in the video so it can generate coherent and well structured scenes. Now this also benefits from some of the latest advancements in AI model design like Swiglu feedforward layers, which help the model learn better and faster. It uses query key normalization to make the training more stable and sandwich normalization to keep internal activations, which are essentially the parts of the model that light up when it's working under control. And these tweaks help ensure that the model runs smoothly and produces high quality output without instability. Now genmo have said that a technical paper will follow up with all the details to encourage progress in video generation, but I think that this is absolutely incredible. Now for those of you who are wondering what is coming Next from the Genmo team, well, it is here. Moki1HD and Machi1HD will support 720p video generation with enhanced fidelity and even smoother motion, addressing edge cases such as warping in complex scenes. Now, whilst Genmos Moki 1 is absolutely stunning, there are a few known limitations. For example, the initial release generates videos at 480p, which is of course not HD. And in some cases, in edge cases with extreme motion, minor warming and distortions can also occur. And if you're wondering about what exactly you want to use this for, Mocky1 is also optimized for photorealistic styles, so it doesn't perform well with animated content. And they're also anticipating that the community will fine tune the model to suit various aesthetic preferences. Which means that it's likely that in the coming weeks and coming months going to be getting specialized versions of this video model that could be even better. And if you head on over to the webpage, one of the top creations was one that I really do like. This was one that actually tests how well this model performs against OpenAI's Sora. And we can see that the stylish woman walking downtown in Tokyo looks rather good. Now there are other examples on this webpage that you can of course check out depending on the different things that entertain you, but I think that this model is one that is really, really fascinating considering the level of quality and the level of control over.