A YouTube explainer (Two Minute Papers) by Dr. Zsolnai reviewing OpenAI’s new “consistency models” for image/video/animation generation. The speaker contrasts these with diffusion models, noting consistency models can produce usable images in one-to-two steps (much faster than the ~20+ steps typical of many diffusion approaches), and discusses potential real-time applications and current limitations.
OpenAI just released an incredible new paper that talks about. Look, not diffusion models. That is interesting because absolutely everyone is talking about diffusion models. These are AI techniques that typically start out from noise and over time reorganize this noise into an image. You see, this concept works for even video generation. It is a series of still images after all. But things get crazier. Diffusion also works in 3D, where the noise shows up as whatever this is. This can be denoised too into virtual character models. Or it works for computer animation too, where the noise can show up as twitching insanity. Diffusion models can also perform voice synthesis. And it gets even crazier. Look, this is doom, of course, but not coded in the way you think it is coded. This is neural doom. It works by first starting to play the game, learning how it works, and here, here's the punchline. Then it learns to remake the game itself. So it is learned and run by a neural network. But here comes the problem. And the problem is time. Typically these diffusion models work in steps. And many models, even the new stable diffusion 3.5 requires 20 or more steps to get usable results. That means that it takes a long time to create anything from seconds to minutes very often. So real time applications are still just a dream. And now OpenAI just published a paper on not diffusion models, but what they call consistency models. And behind a big bunch of mathematics lies something really cool. Yep, these models can give you a pretty good image in only two steps, maybe even one step, which is so much faster than previous models. So is it any good? Dear fellow Scanners, this is Two Minute Papers with Dr. Zsolnai. Yes, I would say so. It is good when comparing the quality against previous diffusion models. Here lower is better. So when using one step it's still a bit far, but with two steps it is rapidly closing in. If you look at the visuals, that seems to make sense. So in the future with techniques like this, we might be able to run neural doom and even more advanced games, but in real time just by showing it to a neural network and getting it to learn how it works. So maybe you would even be able to draw something and make it come alive as a video game. Or get cool animations in video games generated on the fly. Wow, what a time to be alive. Now, two limitations. One, creating the new model still relies a bit on diffusion models. And two, let's not write these previous models off that quickly. For instance, flux in text to image has a super quick diffusion model called Schnell which can also give you a reasonable image in two to four steps, which is also quite fast and can be done in less than a second per image. Not real time, but still fast. However, you fellow scholars know that this is just the first version of an entirely new kind of model and just imagine what we will be capable of just two more papers down the line. So what do you think? Let me know in the comments below. And if you feel inspired and you wish to run your own experiments on Nvidia GPUs, check out Lambda. Lambda is a cloud service for large scale AI projects and it is my absolute favorite. Seriously. Try it out now@lambdalabs.com papers or click the link in the description. If you use Lambda, you can run your own experiments on super powerful hardware and you also support two minute papers.
