This New AI Generates Videos Better Than Reality - OpenAI is Panicking Right Now!

Kuaishou's new AI model CLING generates 2-minute videos in 1080p from text prompts, with realistic physics and temporal consistency. The narrator argues it may outpace OpenAI's Sora, and notes OpenAI has revived its robotics team.
Questions this video answers
June 7, 2024.
LARRUZ.
Up to 2 minutes.
1080p at 30 frames per second.
16 seconds (1080p), released in April (year implicit from context).
Chapters
If you've been following all the latest AI news and events, then you know everyone's been going nuts waiting for OpenAI to drop their new Sora model. But while we were all patiently waiting, this Chinese company called Kuaishou just went and dropped an absolute game changer. Out of nowhere, they released this wild new AI model called cling. And man, it is blowing everyone's minds with what it can do. So let's get into it. Kling is a video generation model developed by Kuaishou, a company you might know from their popular app Kwai. This new model is like Sora, but some are saying it might even be better in some areas. What's really cool is that CLING is open access, meaning more people can get their hands on it and see what it can do. Let me paint a picture for you. So, a prompt that says, a Chinese man sits at a table and eats noodles with chopsticks. Kling takes this and generates a video that looks almost too realistic. And if you remember that infamous video of Will Smith eating noodles that looked like something straight out of a horror movie, well, Kling's version puts that to shame. It's like night and day. Seriously. Hey, Uncle Phil, come try this. Kling can generate videos up to 2 minutes long with just a single prompt in full 1080p quality at 30 frames per second, which is actually pretty impressive. It accurately simulates real world physical properties, which means the videos it creates don't just look good, they behave like real life videos too. The magic behind CLING lies in its diffusion transformer architecture. This technology helps it translate rich textual prompts into vivid realistic scenes. Plus, Cling uses a proprietary 3D Vae that's variational auto encoder and supports various aspect ratios thanks to variable resolution training. Basically, it can handle different video dimensions and still produce high quality output. One of the standout features of this model is its advanced 3D face and body reconstruction technology. It allows it to create videos where characters show full expression and limb movements, all driven from a single full body photo. So essentially, this tech is what makes the videos look so lifelike and consistent. Now, this whole thing is making it pretty clear that China is seriously stepping up its game when it comes to AI development. KLING is just a glimpse of what's coming out of the country, and it's giving us a hint that they might be ahead of the curve. OpenAI has said they'll release Sora by the end of the year, but with Kling already out, they might have some catching up. To do. The big question though, is whether China will make Kling available worldwide. Right now it's accessible through the Kuai app, but you need a Chinese phone number to use it now. Kling isn't the first AI video generation model from China. Back in April, Kuaishou released VDU AI, which could create 16 second videos in 1080p resolution. Cling is the next evolution of that, offering longer videos with even better quality. If you check out cling's website, you'll see some of their demo videos, and let me tell you, they're pretty impressive. For example, there's a video of an emperor angelfish swimming in a rocky underwater habitat, a man riding a horse in the Gobi desert during sunset, and even a white cat driving a car through a busy urban street. These videos showcase the model's ability to handle complex scenes and movements while maintaining high quality. The technology behind cling involves a 3D spatiotemporal joint attention mechanism, which helps it model complex movements and generate video content with larger motions that conform to the laws of physics. So when you see a man riding a horse in the desert, the horse's movements, the dust trails and even the sunset in the background all look just right. KLING also uses efficient training infrastructure and extreme inference optimization. This allows it to generate videos up to 2 minutes long at a smooth 30fps. One example they showed was a little boy riding a bicycle through different seasons in a garden. The video maintained high consistency and detail throughout the which is no small feat for AI generated content. Another cool thing about this model is its strong concept combination ability, like it can take different ideas and merge them into a single coherent video. Like this prompt white cat driving a car through a bustling city. Footage like that doesn't exist in real life, but CLING can create it from scratch, making it look believable. And it also excels in movie quality image generation. The AI can produce videos that look like they were shot with a professional camera crew. One example they showed was a man and a woman walking under a starry sky with the stars moving in the background. The level of detail and quality in these videos is truly cinematic. What's more, CLING supports various video aspect ratios. Whether you need a square video for Instagram, a portrait for TikTok, or a landscape for YouTube, it can actually handle it all. And this flexibility is super useful for content creators who want to use the same video across different platforms. So let's talk about some more examples from the demos. One video shows a Chinese man eating noodles with chopsticks the details are so precise that if you saw this at a lower resolution, you might not even realize it's AI generated. Another video shows a chef chopping onions in a kitchen. And the way the onions are cut and split by the knife is incredibly realistic. But the video I mentioned before, or a cat driving a car through a busy street. This one is particularly impressive because it combines a lot of different elements. Cars, buildings, pedestrians, and the cat's movements, all in one seamless video. Another demo features a volcano erupting inside a coffee cup, showing off Kling's ability to create fictional scenes that look convincingly real. And one of my favorites is a LEGO character visiting an art gallery. The video captures the unique way LEGO characters move, just like in the LEGO movies. There's even a clever focus shift from one character to another, showing that Kling can handle depth and focus changes just like a real camera. Another standout feature is Kling's ability to simulate real world physics. One demo shows milk being poured into a cup, with the milk flowing steadily and filling the cup. Realistically, this might seem simple, but it's actually quite challenging for AI to get these physical interactions right. Cling's technology is also capable of generating videos with temporal consistency. This means the AI maintains a logical flow and coherence over longer videos. In one example, a video of a train traveling through different landscapes stays consistent for the entire two minutes, which is really impressive. Overall, Kling is showing us that China is rapidly advancing in AI video generation technology. They're not just keeping up with the best models from the us. In some cases, they're surpassing them. This could lead to a competitive race in AI development, with countries striving to outdo each other, which might bring both exciting advancements and potential risks. I can't wait to see how OpenAI responds to this cling AI model from Kuaishou. It definitely seems like they're going to want to get their SORA model out there sooner than originally planned to keep up with the competition. But while we're all eagerly awaiting OpenAI's response and reaction to cling, we're also getting some really interesting news from the company itself. In a surprising move, OpenAI has revived its robotics team after disbanding it three years ago. With AI driven robotics investments surging, the company is now actively hiring research engineers for this newly reassembled team. While specifics are under wraps, job listings indicate the engineers will focus on training multimodal models, developing features for partners, and optimizing core models. Interestingly, OpenAI aims to integrate its technologies tech into other companies robotic systems rather than competing directly. Over the past year, OpenAI's venture fund has invested in several humanoid robotics companies like Figure AI, whose robot leverages OpenAI's models. This collaboration hints at a promising future for AI powered robotics. Robotics has always been key to OpenAI's mission, and this revival suggests a strategic pivot to capitalize on the integration of AI and robotics. Alright, don't forget to hit that subscribe button for more updates. Thanks for tuning in and we'll catch you in the next one.










