Luma Labs' Diffusion Revolution: from Dream Machine to Multimodal Worldsim - Amit Jain, Jiaming Song

Hello and welcome back to The Cognitive Revolution. Today I'm speaking with Amit Jahn and Jaemin Song, CEO and chief scientist at Luma Labs, makers of the Dream Machine and the new Ray 2 video generation model. I'm also joined for this episode by my friend Steven Parker, creative director at Waymark and one of the few creators that has logged a proper 10,000 hours with video and image generation models dating back to the original DALL-E over the last few years. Our conversation begins with a discussion of how the Luma team trains models to create fantastical and other fundamentally out-of-distribution visuals for which there is little to no relevant training data available. But considering the force of intellect that both Amit and Jiaming display, their belief that video models are on the critical path to AGI, their ambition to create multimodal AGI at Luma Labs, and the range of novel and occasionally hot takes they share, I think this episode should be of interest to anyone, regardless of whether or not you're particularly interested in video generation models specifically. Keys to Luma's model development success, as you'll hear Amit explain, include a relentless focus on dataset curation, frontier advances in efficient learning algorithms, and a strong drive to understand what their models are actually learning as they go through the training process. These fundamentals create base models that can learn new concepts, including the BoltCam and many other camera motion concepts they've recently introduced in a highly sample-efficient way. Meanwhile, for things that existing models can't learn so quickly, we also discuss Luma's outer loop of product development, which consists of building scaffolding and other behind-the-scenes systems that unlock new model capabilities and also validate customer demand. With that done, they then seek ways to internalize those capabilities in the next generation of the model, And then they repeat this process for each generation as customers continue to apply new and better models to harder and more valuable challenges. For me, the most interesting part of this conversation was the discussion of model interpretability, emphasizing that we should not expect AIs to process, represent, or understand information like we humans do, or even to do so in a way that's generally human-grokkable. Amit likens current interpretability techniques to archaeology, in the sense that they're fundamentally limited to piecing together what models have already learned in the past. More interesting, from his perspective, is the study of training dynamics and engineering of datasets that are needed to teach models what they most need to know. In the last 15 minutes or so, Jiaming offers an intellectual history of diffusion models. This gets pretty technical, and for most people, myself included, it will require some additional study to fully understand. But I would summarize it by saying that the generative AI era really began with the realization that with the right problem formulation, unsupervised learning can work on web-scale datasets. For text, this was simple next-token prediction. And for images, it was gradually adding noise to real images and then training models to remove that noise one step at a time. Since then, there've been a mix of practical tricks, theoretical insights, and model-enabled dataset improvements that have unlocked far more precise steering of outputs and breathtaking efficiency gains. From distillation techniques, which amount to training a model to perform multiple denoising steps in a single pass, to consistency models, which try to ensure that a model will generate the same output regardless of where it begins on its denoising path, to flow matching models, which use theoretical connections to differential equations to take a more direct path through the latent space, to Zhongming's latest inductive moment matching technique, which optimizes the model in distribution space and performs generations in a small number of optimized steps. All of this should at minimum give you a sense of how inference prices have fallen so precipitously even as quality has dramatically improved. While we didn't have time to go as deep into the philosophical underpinnings of Luma's multimodal strategy as I might have wished, I left this conversation with the sense that Luma Labs is definitely a company to watch. It won't be easy for any model development startup to compete with the big tech hyperscalers in the scaling laws era, but Luma's mix of product market fit, vision and ambition, and research prowess gives them as good a chance as any I've seen. And I absolutely look forward to having them back again in the future. As always, if you're finding value in the show, we'd appreciate it if you'd take a moment to share it with friends or write us a review on Apple Podcasts or Spotify. And of course, with the stakes of AI development continuing to rise, I welcome any feedback you might have for how I can do a better job of elevating the discourse and help steering the future away from catastrophic risks and toward the dream of AI abundance. You can reach us via our website, cognitiverevolution.ai, or by DMing me on any social platform. Finally, a quick reminder that I'll be speaking at Imagine AI Live, May 28th through 30th in Las Vegas, the ADAPTA Summit, August 12th and 13th in São Paulo, Brazil, and the Enterprise Tech Leadership Summit, September 23rd through 25th, again in Las Vegas. Tickets are on sale for each of these events now, and if you'll be there, please do reach out and let me know so we can meet up in person. With that, I hope you enjoyed this insightful and thought-provoking conversation about the development and philosophy of frontier video generation models with Amit Jain and Jiaming Song of Luma Labs.

Amit Jain and Jiaming Song, CEO and chief scientist at Luma Labs, makers of the Dream Machine and Ray 2. Welcome to the Cognitive Revolution.

Thanks for having us. It's very exciting to be here.

Yeah, thanks for having us.

I'm excited for the conversation as well, and also excited to have my good friend and longtime teammate, Steven Perker, here as well. Steven occasionally co-hosts when we do an episode on creative models, especially in the image and video domain, because he is the creative director at Waymark. And he has logged more hours than anyone I know with these kinds of products and really has an excellent handle on the exploding array of options in the market today. So I thought to broadly structure this conversation, we might start with discussing like the latest and greatest stuff that you guys have launched in the products, the latest models, the camera motion, all that new cool stuff and how it fits into the broader picture and what you're seeing in terms of usage. And then I want to get into some of the more technical stuff because you've also put out a very mathematical paper recently on an advance in pre-training for diffusion models and a really interesting position paper on how you think we should be thinking about pre-training going forward in general. So I'm excited to get into all that as well. And then maybe at the end, if there's time, we can get a little bit more speculative and talk about like world models and the future of multimodality and what superintelligence looks like and all that great big picture stuff as well. So we've got a lot of ground. to cover, and I'm excited for it.

Steven. Awesome.

Kick us off with maybe a few reflections on your use of Ray 2 recently and what that has you thinking about as we begin today.

Yeah, thank you. Thanks for having me, Amit and Jiaming. It's an honor and a pleasure to speak to both of you. So thank you for taking the time. I have been playing a lot with Luma Labs recently, really enjoying it. One of many video gen tools that I love to use in my arsenal of possibility when I'm working on various projects. And for my own money, tend to find tremendous value in several different models being used at the same time as they all have different strengths, different weaknesses, um, really just a whole blend of capabilities. And Luma is right up there at the tippy top of especially the cinematic end of models that I like to use. So. I just really want to give a shout to you guys there. And my first question there is like, I think more and more of what people want from an image-to-video scenario starts with AI images. That's just a hypothesis on my part, but is that correct?

I think a lot of workflows do start on image side because images are just much easier to iterate through. Iteration cycles are fast and generation times are low. So like you can generate 100 images and find exactly what sort of things that you're thinking about. So yeah, I think a lot of people actually lean in on image workflows as a significant part of their work today.

Okay. So that makes sense with my own workflow and especially what I see out there on social media. And so I think what I'm driving at is like a lot of those images to me seem like they can be strange or new. I'm thinking of like avocado chair type combinations, weird characters, all of these sorts of things. And What I really want to know is like, what has it been like to push these models toward a greater understanding of what I presume is like more novel subject matter?

Yeah, that's really interesting. And currently we are designing a feature and that is one of the most, more important problems because see that there's no data, right? Because by necessity, these are out-of-distribution things and stuff that you would just not find in regular, regular use cases. So I think the singular answer there is like, you need to really work on a very strong base model that has strong capabilities of like understanding what is happening and being able to like then deal with these very unrealistic outer distribution scenarios. So let's say like a pickle sitting on an avocado chair, like, and you start with the pickle standing up and you want it to come and sit down. There's no scenario you've ever seen that, right? But you've seen chairs, you've seen people, and you've seen anthropomorphic things, and you've seen them doing things like sitting down. So this again comes down to the idea that these models are not memorizing behaviors. These are not memorizing how this thing's done, how this is done, how this is done. They're generally distilling out base ideas and capabilities, like the core things. What does it mean to be anthropomorphic? What does it mean to sit? What does it mean to be a chair? That kind of thing, right? And then when you combine them together, the better your model is at these foundational understandings, the better it's going to be able to do when it's presented with these kind of like entirely out-of-distribution, funny, uncharacteristic scenarios.

If I'm understanding that correctly, like improving the base model is just giving you better and better performance with things that are out of distribution. But another thing I seem to notice is that, especially with image-to-video, The kind of inherent characteristic qualities of that initial image also seem to be carried through better, more effectively with longer and longer gens. Do you think that's aligned with improvement of the base model as well? Or are you guys doing more stuff behind the scenes to check against that image more regularly as the gen is occurring? What's going on behind the scenes there?

There's a lot that goes on behind the scenes of like trying to understand what is in the image.

Hmm.

These things are not just models, they're systems. And so we definitely take care of many things in the back to try to break some of these down. But generally, they are crutches for the model to be able to understand. In the next iteration of the model, we make it so that we don't need that system. So, like, initial model, which was called Ray Machine, and now it's like, or now retroactively renamed to Ray-1, it had many of these crutches. To make it feel and work much better, right? Like, you know, to understand motion, to understand different vocabularies, to like translate it into language. It understands all this kind of stuff. Ray 2 doesn't need even like 10% of that. But now, because people are now pushing it to do more things, we have to like build some of these systems again. Like, oh, well, okay, I see. That's what someone is trying to do. How do we understand this part? For instance, like characters, right? All the character understanding in Ray 2 is very much external to the model, right? But then as we design Ray 3, it's all going to be internal to the model. So this is a trend we have seen in language models as well, that when people push the current model to doing things it was not designed to do, labs like us, we build systems around it to at least address. We gather the data and then we just bring it to the model. And this also applies to people, what we call application layer, that kind of stuff. Application layers come up, they build like these specific use cases, but then we see that and we are able to like then just build it out in the next model. And the general rule of thumb in this industry is that, or at least in the technical side of it, is that anything that can be done in the model is going to be better than what is done external to the model. And we can talk a lot about that, but that's generally the situation.

That's great. I'm happy to double-click there as much as you want. I know we're early in, but I do think this is a super fascinating topic that It kind of skews towards secret sauce, company proprietaries, that sort of thing. And so there isn't frankly a lot of conversation about these kinds of systems built behind the models that then you try to reincorporate into the model moving forward. Is there anything else you could say maybe more specifically about beyond data as a loose term, like making sense of these systems and how you teach a model from there to appreciate what was inherent about that system that was helping you?

Yeah.

I think that would be fascinating to know.

That's a big conversation. And, Jia Ming, you should also chime in anytime. But to go a little bit deeper into inside the model versus outside the model, like intra-model versus extra-model. That's a really important thing because whatever you can do in the latent space, all the thinking, whatever you can do about— so for instance, let's talk about video for a second. If you're able to reason about like the actions that are being performed, the characters that are there, what would that character do, the lighting of the character, all these kinds of things inside the model, rather than like a system that actually like generates the right lighting, then tries to piece together the narrative arc, then tries to piece together all these things. Of course, that system can work. That's not the problem. But of course, like people try to do that. Like if you remember, what was the name of the company? It was a scriptwriting company or like a copywriting company, Jasper. Right. And they tried really great ideas, to be honest with you. Right. But the thing is, when you can do it inside the model, inside the latent space, you're just able to work with a lot more information. You're able to do the kind of edits that like outside the model, you just don't have the control mechanisms for the model to be able to do. It's like this thing, but there's a blood-brain barrier. between a generative model and the outside world. And when people say, hey, we want controllability, they want to penetrate the blood-brain barrier better and be able to tell the latent space what they exactly want. And it'll get there. There's no question about it. But inside, there's a level of intelligence that is just really, really great. Outside, there's a level of intelligence that's really, really great, which is you or I who's using it. And then there's a barrier. So things where you can actually communicate from outside in a higher level and then let the brain inside do as much as possible internally, collate information, think about sequencing of events in the video, think about what happens to a character, their arc, their timeline, their causality, all these kinds of things. The more you can do inside the model, the richer the outputs you're going to get. This is what chain of thought also looks like in language models, right? Like instead of forcing the model to like structure its thinking into like, oh, like do this, do this. No, no. Chain of thought is, if you have read the recent Anthropic paper where they claim chain of thought is actually facetious and it's just a way to seed the latent space of the model, right? Like to direct it, make the model be able to direct it and whatever it's outputting is not that realistic of what it's actually thinking about.

Right.

That is actually an indication of that. We have seen that now, like instead of like trying to force the model to output JSON through external coercion, let's just make the model very good at structured output. And then like suddenly you have agents, right? So you're going to see that in multimodal especially because in multimodality, if you're able to simultaneously think in audio, video, language, image altogether, right, and combine reasoning from language, appearance from video, and like all the aesthetics you've seen in images and then audio that you're able to like hear in all the videos, these contexts, if you're able to combine them, you're going to have much better outputs than through external systems. But yes, we do build these systems, then we obviate them in the next model training.

Jiaming, any particular thoughts there?

Uh, yeah, of course. So I tend to agree with Amit here. So, uh, like instead of trying to like program this, into complicated workflows. Just imagine how you would communicate with another human. You don't actually need to program their brains for them to understand that you want to achieve this level of workflow. Of course, there might be some back and forth, but the process itself is pretty much natural. And I think this can only be done if you build these capabilities into the model in the sense that the models need to be more intelligent. I guess right now, even the current generation of models, especially video models, feels less intelligent in the sense that you have to tell them or program them to achieve some particular type of task, like image-to-video, for instance. Of course, there are practical reasons that this is done, but I think eventually it will be intelligent enough to tell, basically take in whatever task you want. In the multimodal sense, you tell them, for example, I want to do image-to-video keyframes, like camera motion and all these different tasks. And the model should be able to do it without, say, even thinking about it as a different type of task.

So could you guys maybe give a couple of examples of, presumably it wouldn't be too sensitive to say, things that were essentially scaffolding in the Ray 1 generation that are now handled internally by the model? And then if you're willing and open to it, things that are currently scaffolding on the Ray 2 that you hope to be inbuilt into the model when we get to Ray 3?

Like, I guess one example would be like maybe camera motion. Like in like Ray 1.6, we had camera motion features shift, but that was like, obviously like Harder to actually control and like lower quality than what we recently released. So in in this iteration, you can see that there is a lot more like different types of like flexible camera controls, such as like ball cams, which without the user having to prompt for exactly the extrinsics and the extrinsics of the camera, it just works. So of course, you can also give it more explicit controls. That's what some users would want. But I guess for other users, maybe what they really want is just like I convert this scene into. Like, like give me this effect into the scene without like me having to specify the very detailed controls. So I think that's one of the like features that we have it in the model. I think what would be like really interesting is to like build into the model, like, like now it's one level ahead of like being able to represent the scene, like the boat cam. Like again, you want slightly more control than just generating a boat cam. But not as much control as you have to specify the exact camera coordinates. So maybe somewhere in between you can have something like, oh, I, given this camera trajectory scene, I want this Bolt cam to be like moving faster, moving slower, having like more angle changes, or like focusing on some other like subject matter. So this kind of like more interactive, like Editability of the scene is something that we are like thinking about, like maybe having in the model in the next generation.

Hey, we'll continue our interview in a moment after a word from our sponsors.

Cool.

That was actually my next question. So maybe just to restate a little bit here for our audience who might not be super familiar with Boltcams, that's bringing us to Apple TV+ Severance Season 2, right? Which made a huge splash this year with their opening shot, which is a Bolt cam robo arm shot. I think it was super impressive. Famously took them months and months to create, and was just a big wow moment for audiences. And now, you know, just a little bit later this year, we have it as one of your key like camera motion concepts already available for people like me to use when generating. It's one of a range of motions that you have added to the model capability. Super useful, and I imagine like. Very highly in demand from your editor type of user approaching the model, as well as novices and everybody else, but got to be high on the professional list. What has it been like to develop that feature specifically with an outlook on the professional user or maybe customer requests coupled with modern trends like Severance Season 2 and the bulk cat?

So basically, like, there are models, right? Like the things you teach it during pre-training. And then there are many capabilities that people want to teach the model. And language models have this really special ability, which is called in-context learning. You get, you show it some examples and like it becomes that. And I think that's the truly emergent intelligent capability. Visual models aren't there yet. They'll get there and soon enough, actually, but they're not there just yet. But that doesn't remove the need for teaching these models specific things you want in the moment. So we have been working on this idea. So we published a post about this, a white paper, whatever you wanna call it. We call them concepts. And we realized that in visual, especially creative use cases, there are many things people wanna teach, many things that they come up with, like a specific motion or particular kind of color grading or a particular human pose.

Right.

Which is just really suited for the story you're trying to tell or the ad you're trying to make, whatever it is. But there's no way to get it out of the model because it's a new thing you just came up with. There's no data for it on the internet. There is no way to generate large samples of it to even be able to fine-tune the model. So we designed this idea of concepts to make it so that models can learn From one to just a few examples, their capabilities don't degrade like they do when you're fine-tuning or creating LoRAs, and they can be composed together. So how closely can it represent a capability that the model had at pre-training, but users actually teach it? So we call them concepts. The tool that we're using to build them, they're called Concept School.

Right.

You go to school, you take the model to school, And you're the teacher and you're going to teach it concepts or lessons, whatever you want to call it. So the camera motions you're talking about, the Bolt Cam and like dolly and reverse dolly, all these kinds of things that we taught, actually we were able to teach it to the model very quickly, or relatively speaking quickly, with very few examples. And you're going to see like the next batch of it come out this week and the next batch come out next week, so on and so forth. And our goal is to basically teach our models everything about Filmmaking this way and eventually also give this ability to people to be able to teach them. Right now, like it's a little bit finicky as new technologies tend to be. So we haven't made it open source or sorry, open access yet, but we will in the future. So these camera things, right? Like, you know, the way we are building them to answer your question on feedback and these kinds of things, there are like some of these which are basic storytelling tools, being able to track a shot, being able to move left, being able to move right. Like, you know. Basic things. And they're done by people 1,000 times a day. Whenever you're like shooting something, it's like, oh, the camera moves left, like, duh, right? Or you're tracking an actor who's like doing these kinds of things. So you want that. And then you want to also balance it with some things which are absurd and which are funny or which people just can't do in real life very easily. Like for instance, Bolt Cam. If you want to do Bolt Cam really well, that smooth tracking, you actually need a robot.

Yeah.

So MKBHD has one. I'm sure James Cameron has a few, but like most of us don't. So can we just have it in the model? So like we try to balance like this great utility with some things which are just absurd and fun, but ultimately like we're building this for professionals. We're building this for people who want to tell stories, who are already telling stories, or who want to become Professionals in that world, and we can talk about like the changes that are happening in the industry very quickly, not the industry, but moviemaking industry very quickly. But yeah, we're designing for people who want to tell stories and these are all storytelling tools.

So 2 things there that just brings to mind for me. One of them selfishly is talking about those repeat actions that the pro user takes. One of the selfishly, one of the pro actions I take all the time is leveraging your audio generation feature, which is great. It's. For people who don't know, after you generate a video, you can just press the audio button and it will give you another prompt opportunity and give you the ability to generate audio for that clip. However, one thing I do all the time is reverse the playback direction of my video when I'm editing. And so just a tiny little plug here. I would love to have the ability to quick flip the playback direction before generating that audio. So that I'm not dealing with reverse audio when I take that clip out. But I'll just leave that as a footnote. I think really what this is getting to is like, you're about storytelling. You have all kinds of users telling all kinds of stories, and that skews towards fantasy, Hollywood cinema, anime. It's everything, right?

Yeah.

How do you imagine that translating to multimodal understanding? Is it just like, It's an attempt to understand everything everywhere all at once, or is there like particularly unique insight that you feel like you gain in the pursuit of art first that helps with the overall mission toward multimodality?

See, what we are trying to do is, uh, so the mission of Luma is to build multimodal intelligence, right? Multimodal general intelligence. If you were to personify, not personify, but if you were to materialize it in front of you, And you were to ask, like, what does that look like? The intelligence that LLMs embody, that looks very much like abstract intelligence that humans have, with language and things like that. When you think about multimodal intelligence, of course it has that abstract part, but what's that actually look like? It starts to look very much like world in a globe. Like you have this world in front of you and it has all the physical properties and all the physical phenomena that happen day to day in our physical world. It also has those intelligent beings inside it that do things that like increase the entropy of the universe, that like interact with each other, that do all these kinds of things. So a term a lot of people use is world simulator, right? But I think simulation is a weaker term here than it should be used. It's basically just like it's a world model. It's like a physical manifestation of the universe that you have outside. Of course, it's a weak facsimile. It's an approximation, but yeah, it's some model. of the world, of the processes that we are having. Now, coming to the creative side of it and why that is important for this mission is like storytelling. Like if you think about it, like language models, right? And we were like, oh, this is only good for JSON. And then we were going to produce just JSONs, right? That's not very good. When you force intelligent systems to play games, to come up with like abstract new things, to come up with like instruction following, like, oh no, I want this, then I want this. When you ask them to dream, That's when like you get intelligent systems versus just like procedural systems that are like following a set of rules that you have created. When you force them to deviate from that, making movies, telling stories is a very critical part of it. It's a part of like human existence too, right? Like how good someone is at storytelling is generally a very good bearer of IQ, right? Like can they actually think beyond just the most like physical thing that is in front of them? Right. What is the consequence of A? What is the consequence of that? What is the consequence of that? What does it lead to things? That's what stories are, right? Event A then happened, then B happens, then C happens, then D happens. Video models are on the critical path to that general intelligence. Once we're able to combine video, audio, language all together, they will become really, really good at storytelling, right? Being a partner for us, like who can actually sit next to us and like help us dream, help us imagine, help us like think through those kinds of things. So. Creative pursuit. It's not a surprise, by the way, that people thought the first things to be automated will be the mechanical things. It's not a surprise that the first things to be automated are, or not automated, but the first things where AI is able to help are actually creative pursuits because that's where we require intelligence. That's where these things come into play. So yeah, I think art has always had a very significant role to play in general thinking, general intelligence. Art is also going to have a very significant role to play in building artificial general intelligence.

Hey, we'll continue our interview in a moment after a word from our sponsors. Can I ask a couple kind of world model questions?

Yeah.

I think this is super interesting. One big thing that jumps out at me though is if I was trying to create a generally useful AGI to go out and do stuff in the world, right? To maybe like, control robots and walk into my house and make me a coffee, to take a famous example, and to vacuum up my living room after my kids have thrown toys all over it, whatever the case may be. I would want this sort of world model. And I'm certainly a big— I wouldn't say believer at this point is the right term, because I think it's pretty well demonstrated that the large foundation models are learning these higher order concepts. And that's, I think, become pretty much indisputable at this point. So I'm well convinced by the evidence in the literature that this is happening. But I wonder about the sort of fictional side of it, the magical side, though. Like, there are no dragons in the real world, but your models have learned to also represent dragons, right? So in a way, you have a real world model plus. It sort of goes beyond what is real and into the imaginative, the fictional, et cetera. And I wonder, obviously that's good for storytelling because we want to tell stories that are not bounded by reality, but does that have downsides for the sort of practical utility of making multimodal intelligence that I just want to come into the world and do useful work for me?

Yeah. So I think there's a short-term answer for the short-term, and there's the answer for the long-term. I guess the answer for the short-term is that if you want to use this type of model for your daily activity task, like right now, then yes, it might be better to angle towards the more physical, applicable vertical, so to speak, for this type of models. However, There are many cases where you ask the model to do some tasks. So for example, say, pick up this clothes with a dragon on it and push it, put it in the washing machine. And if the model doesn't really know about the concept of a dragon, you can't actually pick up the dragon pattern clothes into the washing machine. So even this kind of knowledge that is fictional can still be very useful for real-world physical tasks. Because again, even the concept of a dragon by itself is related to other physical concepts that people see in the world. For example, it looks like a lizard. It has scales that are exhibited in reptiles. It has wings, which you see in bats and birds. So even these concepts are this, if you think about it, are what, like, humans' interpretation to, like, what they see in the real world. So it's like a dragon is generated by humans looking at the real world as a part of their training data, and they generate something that is, like, out of ordinary. But back to the original question, yes, if for the short term, yes, I do think maybe, like, focusing on more, like, physical, like, realistic things can be better for these kinds of tasks. So the model doesn't, like, hallucinate as much. But in the long term, I believe that the model should be able to tell itself apart from what kind of world it is acting in, because there's also other types of worlds it needs to be able to act in. And it's not just in the physical world. For example, in a virtual world where things people are currently doing with agents are also a perfect example where the model doesn't need world knowledge, but it is a world that is different from the physical world that we are acting in. So having a unified AI that can be doing both tasks could be useful. For example, eventually your robot could be like, okay, put this dragon t-shirt into my dishwasher and then go to the computer and type an email responding to my friend or something. This requires the AI to be able to reason between physical and non-physical or imaginary worlds. Or virtual worlds, just like humans can do. So I think eventually this will be a capability that does not have to be specialized into any sort of models.

I don't know if you're doing interpretability work on your models internally, or if you have any partnerships, academic or otherwise, that allow you to do that kind of stuff. Maybe you know the answer because you maybe have done this work, or maybe you could speculate as to whether you would expect to find features in the model that would represent, like, this is a realistic physics simulation versus this is a magical realism type of scenario versus this is a Minecraft environment that we're in right now, or we're playing Pokémon or whatever the case may be. It seems like that probably would happen at sufficient scale, but I don't know if we are there yet or if anybody's really had the opportunity to look.

Upfront, I'm not an expert on the topic of deep neural network interpretability, or I have delved too much into this space. My very shallow understanding about this is that it might be good to seek more alignments or try to get the model to align with what you want and try to reason from there. So for example, one feature that we have discussed heavily in the first virtual machine is this implicit knowledge about 3D. So for example, you can ask it to generate things that reason about depth. And even in fantasy, unrealistic, it has— in art, it has a reasonable interpretation of depth and also other types of physical features, such as clothing simulation, such as waves, and such as Air, these kinds of stuff. So it's in this case, and the question will be asking that you're asking is, is there a kind of neuron or like a substructure I can find within the model that like shows that? And I guess the honest answer will be, it's very difficult to like find that within the model, but it probably might be easier if we prompt the model and ask it to do the task for us. So one example is like, for example, people have shown that these models are very great priors for things like 3D vision tasks, such as depth estimation. So in this case, even almost a lot of the state-of-the-art depth estimation methods are actually based on these pre-trained models, which actually have a very good understanding of the world. So I do believe that some degree of interpretability, or the model has a great internal knowledge of the world, but it's also up to humans how interpretability exists. So currently it seems like the API between like the human neurons and the like model neurons are not that well welded just yet. So probably we'll still have to use like the language that both sides understand to communicate this interpretability thing.

Yeah, there's definitely some fraught challenges there.

Yeah.

Abhay, go ahead.

This is an interesting philosophical question that Nathan and I go back and forth on all the time. I mean, we tend to hear from people a lot that These models have physics knowledge inherent in the training. They have a great, rich 3D understanding of the world that we see it in this or that. I personally am not entirely convinced that they aren't just seeing more and more kind of movement across a 2D space and have like greater and greater pixel understanding. I can understand somebody arguing against that from maybe a systems-based training regime where we're training first on 3D scaffolding or something like that, but—

Yeah.

As I, my own naive understanding is that sort of thing isn't happening in the training. I just think it's a really interesting question. Like, are we actually seeing the physical appreciation for a 3D space or are we just seeing more of an expert interpretation of a 2D pixel space?

So I would say it's, uh, I would draw this like ability to maybe predict or like reason about physical scenes in this representation of the 2D pixel space. To a Turing test. So in the Turing test, your goal is to not tell whether the model knows grammar or not, whether this internally represents the same with the grammar that humans define. It's not really part of the question that we are asking. What we are asking is that whether talking to the model, it feels the same to talking to a human. In this case, it's similar. So on the one hand, being able to render, do frame next scene prediction, even perfectly over the next few seconds or minutes or so, doesn't actually mean that inside it is the same physical knowledge that we currently define as humans. But on the other hand, from a utility standpoint, it might constitute as passing the Turing test or Like visual generation.

Yeah. And I have had a lot of time to think about this problem. And the question is basically philosophical to what Jiaoming is saying, that it borders on the definition of understanding, right? What does it mean to understand? And people make the argument that humans have something very special or deep where we understand at some level, but it's very hard to actually argue about that as well, because how would you know? Right. You can say like, oh yeah, but I know this is 3D. That's why I understand it. Like it clicks in my brain that this is 3D and not 2D. But that's all like the brain is this really interesting organ that is simultaneously thinking and telling you, whatever you is, or itself that it is thinking. And that's so weird, right? Think about it. Like it's self-aware. The self-awareness part is the most interesting part of it. So. Coming back to, let's say, 3D understanding for just a second, what does it mean for humans to have 3D understanding? Actually, humans don't really— I'll take a slightly different stance than most computer vision researchers take, which is that, oh, humans actually perceive 3D and things like that. We don't really perceive 3D.

Really?

Yes, we have 2 eyes, we have stereo vision, but stereo dies out at about 20 centimeters. Outside of that, the disparity is almost nothing.

Right.

And then like case in point, if you like hurt one eye and you have an eye patch, you can still drive. You have some initial trouble trying to grab a piece of glass or like something close up, but you really drive. Of course you have hands and you have proprioception. So the brain gets other signals, right? That teaches it like depth and that teaches it some of these things, but it's hard to really argue that the, like the brain actually maintains some sort of 3D representation in the brain.

Right.

It might not. We just don't know. We have no way of understanding it. So a generative model, why is it any more special that it has an explicit 3D representation inside it, like a mesh or whatever have you, than just understanding these concepts more implicitly or more in terms of 2D space and time? And as long as it— to Wujiaoming's point, as long as it's able to generate something that looks really consistent, Why do you care how it actually did it? Like planes fly, but they don't flap their wings. The phenomena that birds use and the phenomena that planes use is actually very similar, right? Birds generate lift by, by like moving the wings itself. Planes do that by pushing themselves through the air and like doing that. So while the phenomena is the same, the mechanisms are very different, right? Humanoids right now, like we are trying to build them, they move. But their movements seem very different from how we do it. It's going to happen more and more. All machines that do things humans do, like dishwashing, we wash dish very differently than dishwasher. But are the dishes any less washed because the dishwasher actually didn't have hands? I don't think so. So philosophically coming back to it, I don't think machines have any prerogative to think exactly how we do, have representations inside them like how we do. It doesn't make them any less intelligent. It doesn't make them any less capable. In fact, they should do it differently because the substrate they're on is very different. Human brain is a 20-watt piece of organic tissue. On the other hand, here we are running things on gigawatt-scale clusters. Why should they think the same way? So that's my answer to that problem, actually. I think people who are focused on making machines think exactly as humans do, making them exactly as interpretable as humans have are misguided. They are not only reducing the capabilities of machines in that process, but also wasting their own time. They should use that time to scale attention better. They should use that time to design regimes that can do more efficient learning. All these kind of things, right? Yeah, I think it's a waste of time. All right.

I agreed with the first 85% of that, but The last 15%, I want to challenge.

Go for it.

When it comes to interpretability, in terms of why would you care?

And I am—

the 85% does include the idea that I don't expect the AIs to be representing things in the same way that we are or thinking broadly in the same way that we are. And I don't think that should be used to discount them. I have a funny series of tweets where it's like, it's only a concept if it comes from the concept region of the human brain. Otherwise it's just sparkling notions or whatever. So I'm with you in terms of viewing that as a straw man. But I do think when you look at something like Golden Gate Claude, for example, where they were able to say, okay, we were able to go in, isolate what very strongly appears to be the Golden Gate Bridge concept. And now when we artificially turn that up, we get Golden Gate Claude. And then, yes, that's just a curiosity and a cute demo. But I actually thought when you were talking about the concepts feature, that you were building that maybe you were doing it that way, where I could imagine that you might say, if you had taken an— and I'd be interested to hear how you are doing it, if it's— because it seems like it's not this, but what I had imagined you might be doing there is taking a similar approach, trying to identify directions in the latent space and then injecting them or turning them up in order to enable these different concepts. And it seems like you probably could do that and be like, turn on anime or turn on— Toy Story style or whatever. Stephen has a much better vocabulary for the different styles than I do. But I assumed that your concept work would be similar to Golden Gate Claude. And I guess the questions there would be like, doesn't that seem like it would be a very useful thing to study and potentially be able to marshal? And if you're not doing that, maybe can you tell us how you are doing it?

So there's a difference between treating something as a black box and being able to do empirical experiments on it. But we are in the philosophical land again because we are not really— think about physics, right? A lot of people think our knowledge of the world and it's completely interpretable. And because we have an equation, we understand how the systems work. That's not how any of the laws of physics actually are, right? Science is not the act of interviewing God if that existed, right? I'm not saying one way or the other. Science is basically, we have an observation, can we derive a pattern out of it? That's about it. Science is empirical. Theoretical physics is also, again, very much about building a mathematical model, an approximation, a representation. Machine learning is very much the same. The entire universe is very statistical. The best theory that we have of how the universe functions right now, quantum mechanics, is entirely uninterpretable. Today, we don't understand why things are the way they are at all. The measurements get verified up to 6 sigma, 8 sigma, sometimes 10 sigma, as accurate as it gets. There are some measurements that are up to 23 sigma, but we don't understand why the world is that way. Right? That's what you say when you're asking for interpretation. I'm saying ML models, because of their sheer scale, are just not grokable by the human brain. You're not going to come up with this coherent, consistent model of how the ML model functions because it's just not that kind of a system. We can do empirical experiments on it after the fact, like the Golden Gate experiment that you're talking about. Like, oh, we found a cluster of this, we found this. It's like archaeology. We can do archaeology. All right? But here's something really powerful that we can't do in the physical world. In the physical world, we can only do archaeology. Here, we are actually involved in the formation of the planet and all the eons it went through. That's called training. So we know what the model is going to do. So instead of people spending their time on archaeology, I think a much better use of time is understanding data and training processes. You actually learned so much during training. At the risk of Xiaoming giving something away here, like when we train our models, like we are working on actually designing these models with great degree of efficiency. And in that process, we do so much information theoretic work on our models to try to understand what sort of information flow architecture is in there. Like what is happening to high-frequency details? What is happening to low-frequency details? What is it learning in the earlier stages, later stages? When should we actually change the curriculum of like what it is actually learning? This is all basically interpretability, what people talk about. This actually has real consequences. This changes what the model learns. And some of the things we have learned are really interesting, which is the information theoretic pathways are set about in the first 20,000, 30,000 iterations. The model starts out as this scalar field, really. What is a good example? Think about a field of a maze. Or a field of corn that is grown. It's just plants everywhere. They look uniform, all these kinds of things. As you train, pathways keep forming in between them, like someone walking and putting the plants down, right? And then they're like crop circles. And when you look from the top, you're like, oh, there's a pattern. There's a circle and there's things like this. These happen very early. And the distribution of data you have in that time and the kind of learning rate regimes you've used decide a lot of what is going to happen in the later stages. But in the later stages, a lot of different things happen, right? In the later stages, like these pathways are emphasized or deemphasized. Someone goes and undoes the crop circles. Someone goes in and does like, well, we're not going to take that path as much. This is the kind of interpretability allows you to design the models that give you the output you want. So the post-fact interpretability work, I think it's pretty interesting. No, don't get me wrong. Intellectually, this is like empirical science, right? Like it's very good. And if you have an alien model in front of you or something someone else made, it's very useful to understand what's going on. But if you want to actually control the models, you want to control the data and the training process. And I mean the actual hyperparameters of the training process. There's so much that goes into that. So that's where actual interpretability comes into play, in my view. And that's where we do a lot of work.

Yeah, that's super interesting. So, I mean, you could expand on any number of dimensions of that. One that I've been thinking about Quite a bit recently is you might call it batch strategy. It sounds like you are probably getting pretty intentional in those early phases of training about making sure that you are feeding batch by batch the right mix of data. Because I can imagine if you had— I've always looked at these loss curves and you see these occasional spikes and you're like, what's happening there? And My one sort of interpretation is like, maybe that's a bad batch, some sort of cluster of data that the model wasn't prepared for. But then also, if my interpretation is right, then that batch is probably sending a bad signal to the model as well that's not actually constructive for general learning purposes. So are you actually doing this sort of batch-by-batch construction of data to try to get the mix right at that very granular level, or am I over or misinterpreting what you're saying?

I think Xiaoming can answer this a bit better, but I can clarify my own statement. I don't mean like handpicking it that much, but I think we come at it from the other direction, which is a lot of curation and filtering and removing like garbage examples. The technique to training really great models is not to throw billion samples of garbage data at it. And it will figure out, out of the garbage, right? The technique is show it what you want it to learn and show it good examples from there, right? At large scale. So yeah, you do a lot of this preprocessing, filtering, that kind of stuff. But, Zhiming, I don't know if you have a different take on that.

Yeah, I think it's a very interesting question to be seen. Basically, it's like, what is the right dataset? Like, essentially, interestingly enough, the kind of statistical methods that we're using, like, for, like, training these machine learning algorithms are actually quite different from what the real world actually is. So if you run this algorithm, whether it is on, like, language models or diffusion models, you're mostly making, like, IID assumptions. Basically, you have a data point, you assume that these were drawn independently and identically from the world distribution. But the real question is like, is there a world distribution or, and whether like the, like how you even define the world distribution is a real question. So I think regarding the question of like, what is the best dataset? It's actually very hard to like have this right answer. So I think people probably take a second, like, you know, less statistically driven, but more say like objective driven approach. So it's like, okay, how do I control the dataset such that I get a better quality or better outcome. And it has been deeply studied also in the realm of, let's say, language modeling, for example. How much English do you want to have in your entire corpus? To be honest, it is very hard to reason about these in a theoretical level because you wouldn't want to say that, okay, we just based on how much English to be put into the dataset based on number of populations who speak English. So That's probably not the right way of like doing it. And instead, like having a more like empirical focused approach is probably the right way of finding it. But the exact solution is probably very messy and there's not really a lot of like key, like concise principles behind it, despite how easy it is to explain maybe the idea of like machine learning to a like general purpose audience.

Maybe Zhiming. Would it be too much to ask for you to give us a short masterclass in the intellectual history of diffusion models? Because I think you're probably the best person in the world maybe to do that.

Yeah, of course.

So I sketched it out and the mantra that I always say to myself about the original diffusion models is the big unlock was there's all these images out there. It's really easy to just programmatically add noise to them step by step until you get to pure noise. And then the insight is if you just train the model to do the reverse and learn a denoising step, then if you run a bunch of those steps in a row, you can start from pure noise and eventually get to some image. The original versions of that were like totally undirected and just generated an image out of nowhere. Then we got classifier guidance, and that sort of was enough to steer us in the direction of particular images. And then I want you to take over and tell us the major updates in the field that have brought us to where we are in terms of Efficiency and control.

Yeah, of course. So I guess like the first known method of diffusion models was actually reported in this 2015 ICML paper. It's from like Raphael Dickstein. So he described the actual like algorithm for like how to train or how to formulate the forward and reverse process of a diffusion model. However, that never really caught up because back then the experiments were done on like very small, like MNIST datasets. And it has like still the same problems that like plagued diffusion models, like very early on back then. So it was like, to researchers of that field, it seems like, okay, I have like GANs that work in one step and it also works much better on the large scale. Why am I bothering myself on this method that seems like very hard to grasp and like very slow to Like generate samples and so on. So the first kind of like real breakthrough on this field came then in 2020, which was on the paper called like Denoising Diffusion Probabilistic Models by Zhang Fei-He et al. And the basic idea is that, first of all, it validates the, like the validity of this idea and made it as efficient or not efficient, but made it as performant as GANs on certain like tasks. So it was still very inefficient. So in the sense that you need like many of all the steps to like converge to the right image, but it is one of the first algorithms that does not have the problem of like the unstable training that GAN gives. So for like practitioners like us, like the algorithms being trained, like stable to train is very important because it's a huge difference between like you launch the job. You can go to sleep at night and waiting like until the next few hours, the result gets better versus in the GAN case, like things just become unstable, like out of nowhere. And you have to do a lot of like digging in this case. So the training algorithm being stable is a very huge deal here. And this also, this is actually why people got interested in the diffusion models, even though like they were like very slow, like for the, at their time. I was like part of the people who work on these like general directions. I worked on other types of generative models, including GANs before. So like, to me, like diffusion models seems a very like fresh perspective because before the existence of that paper, there was really not any like method that has trained stably and high sample, like high quality, like sample generation. So then I was trying to think more into this case of now we have this model generates high quality samples. Back in that time for CIFAR. This model is very simple to train. What is the problem with the model? So the problem with the model is that it generates very slowly. It takes like 1,000 neural network iterations to get to a high-quality sample. So I was more looking into like solving like that particular angle of the problem. And this is how I led to my work on like denoising diffusion implicit models or something that is able to accelerate the model sampling time to like 20 to 50x. back in the days. Of course, simultaneously, Yan Song was also from our lab at Stanford, was working on the more generalized idea of diffusion models. So his idea was more like, instead of having this fixed discrete number of time steps, we can actually make the time step continuous matter and wrap it more into the math that came from the 1980s from this person called Anderson. So it was all in this stochastic diffusion equation framework and score matching. And Yann was a very early kind of advocate on score matching and denoising score matching. So it naturally fits into his framework. So then he found this very interesting connection between diffusion models and denoising score matching. So that was the, like, kind of the first 2 breakthroughs, one on the theoretical level and the other is on the practical level. And the rest you have described from, like, OpenAI did the work on Training these models on like ImageNet, which is again, a big breakthrough at the time. It shows that it is competitive with GANs in these cases, as well as like using classifier guidance that people are using until now. And then people start to use like classifier-free guidance in the sense that they realize we don't actually need to train an additional classifier. You can add, say, this unconditional signals to the model such that you can replace the goal of a classifier. And this is also from like Jonathan Ho and Tim Salinas. After then that, there's a few papers around 2022, which tries to even scale it up beyond ImageNet. In early 2022, there was this paper called Glide from OpenAI, which basically tries to try diffusion models on text-to-image. Like a few months afterwards, the Imagen paper from Google came out and a few months afterwards, the Stable Diffusion basically got released. So basically this is how we got to Stable Diffusion. But of course, like after Stable Diffusion, there has been an explosion of these techniques and similarly like open source models. And then the next thing people care about is like one, how to get higher quality models, how to make it work on videos and how to get it to be more efficient. I will talk about more on the more efficient side of things. So there has been a lot of efforts like initially to do distillation and there still is, but previously the way people do distillation was kind of hacky. So the idea is that I have this like a thousand, maybe many minutes time, like process. I try to use a model to represent like 2 contiguous steps so that I reduce the number of steps by 2. And then I try to train another model to simulate that 2 time steps so that overall, like have 4x acceleration and repeat this again. So this is called like progressive distillation. It will also come from like Jonathan Ho. But this is like pretty tricky to train because the implementation gets a bit hairy. So early 2023, there's this other paper called Consistency Models, which at the time was aiming to be a replacement to diffusion models in the sense that it is both efficient and you can train it in a single stage. Of course, he also described a method called Consistent Distillation, which is a distillation-based method to this, like for consistency models. So basically you use an existing diffusion model as the base and you try to distill it with like consistency models ideas. What actually became more popular in the field was consistency distillation, because actually when people tried consistency models training on other use cases, it turns out to be like not as easy as it seems to train it very stably. So people came up with new methods to stabilize the training process for the consistency models, which usually is still based on initializing the model from a diffusion model, so to speak. So then comes with this type of like distillation techniques. And there's another set of techniques that are based more on GANs, so to speak. So the trade-off between consistency models and the GAN-based methods is that the consistency distillation techniques are in some sense more stable to train, but the GAN-based methods are less stable to train, but they have maybe higher quality if you run them at fewer steps. So again, this is this interesting trade-off between how stable it is to train the model and how like easy it is to get to like high quality, like samples. Back to the original story between like GAN and diffusion models again. So, you know, that kind of is the current status quo on diffusion models, the history of diffusion models, how people have scaling it up and what are the key problems in how to make diffusion models even faster via this path of distillation. Yeah.

Could we do just a little bit more on classifier-free guidance and maybe also on what the intuition is with the consistency model. Like, when we're doing guidance, what exactly are we doing to guide the model? I get it in the classifier sense, or at least I have an intuition where I would be like, okay, you feed it to the classifier and take its feedback. But when we're classifier-free, I think it's a little less intuitive. And the consistency model thing, I think, is also a little less intuitive than the simple distillation.

Yeah, sure. So before we talk about classifier-free guidance, we can talk a little bit more about classifier-based guidance. So the idea is that diffusion models, it tries to represent a score or the gradient of the log probability of this distribution called p. You try to guide it with some conditional signal. Let's call it y. So basically, Instead of actually trying to sample from p of x, you want to sample from p of x given y. However, this, you can train on this, of course, but another way to like treat p of x given y is to apply the Bayes' rule. So basically the p of x given y is equal to the joint of p of x and y divided by like p of x. So I guess that's basically, you can think about like p of x. As the score of the diffusion model without any condition. And the p of x and y part is the score of the model with condition. Because the other reason is that the p of y, which is the condition part, you don't actually care about that during sampling. So you can just drop it. So basically that is why in classifier-free guidance, we have this unconditional model, which represents the denominator. And the conditional model, which represents the numerator, but all in all, it's basically application of Bayes' rule. A consistency model is basically something like this. Maybe it's easier to explain consistent distillation first. The idea is that I want to have a one-step model to generate the right solution, but I want to find a way to bootstrap it from a regular diffusion model. So the idea is like this. Suppose I have a perfect one-step model and it follows the trajectory of the diffusion model. So basically the idea here is that we want to learn a model to distill the process in which a diffusion model would normally go through with many steps. We're just distilling this function. So a consistent model would do, in principle, in the distillation case, is that you can compute the one-step prediction at different time steps. I guess time step is a concept that is correlated with how much noise you add. So the more noise you add represents a different timestep or maybe a higher timestep in this particular case. So in a consistency model, the idea is to build a connection between 2 quantities. The first quantity is basically at a given particular timestep, what is the prediction that you are going to make in one step? The second quantity is that If you suppose you are using 2 steps and one step is the regular diffusion model step. So you run one regular step of diffusion model to a timestep that is closer to your original state. And then you run this same consistency model to reach the final step. So basically the way how consistency distillation is trained is that it tries to basically minimize this loss function. And because your consistency model at the earlier times, like at the timestep that is closer to the clean signal, has an easier time predicting what a real number is, then it allows the model to build this connection and train this function. But essentially what consistency model tries to do is that it tries to use a model to distill the otherwise hard-to-compute process that is run by these diffusion models.

So there's so many directions we could go here. Let's do your latest contribution, the inductive moment matching, which I basically take as being a best of all of these prior approaches combined into one. There's sort of echoes of the consistency model idea and echoes of distillation with a sort of an interesting— what jumped out to me maybe most about it was the idea that the model is being optimized in distribution space on more of a, I mean, I guess it's always batch level, right? But it's like— At a higher level than just example by example?

Like I mentioned, we want to achieve these in the generative modeling algorithms. One, it is that it has high sample quality. 2 is that it is stable to train. And 3 is that it is relatively efficient when you are trying to draw a sample from it. So most of the— well, most, but all of the existing methods suffer from one of the 2 drawbacks. So for example, generative adversarial networks like GANs are not that stable to train. Same kind of goes to some degree of consistency models as well. Diffusion models are stable to train, has high sample quality, but it can't generate high quality samples in very few steps. So the inference cost is higher. So what we want is actually we want to find an algorithm that satisfies all 3 of them. So basically high quality generation, fast to sample, and also a stable training process. In this case, we try to actually reason about a generalization of what consistency models are kind of doing. So in the sense that instead of trying to match this pointwise sample, so basically I have this function and I want to exactly match the samples. What we can end up with is we only need to match the distribution-wise because we actually don't care about exactly this function has to have matched. For example, if you try to have a bunch of samples and you're trying to push them to another set of samples, there are many different solutions you can go with. But in consistency models, you are forced to follow one type of solutions. And that is actually a little bit more restricting to the model. And maybe your model needs more capacity to achieve what it is being asked for. And this is possibly one of the reasons why these GAN-based methods are having an advantage because the GAN-based methods, are actually comparing samples at a distribution level. So instead we are thinking about, okay, instead of trying to match the samples pointwise, we can just match the samples at a distribution level. What is another algorithm that can match distributions based on samples that is not GANs? So it turns out actually this idea has been discussed in the statistics community at least 15 years ago. And this idea is called maximum mean discrepancy. So the idea is, can be like heard, it is maybe a bit more scary to hear about this concept. But what you can think of is it has a very interesting relationship with GANs. So in GANs, you have the discriminator that is trying to maximize the distance between like the prediction of a real sample and the prediction of a generated sample. But in maximum mean discrepancy, you do the same thing, except for that the discriminator is no longer a like a neural network. It is a function defined on the space called RKHS or reproducible kernel Hilbert space. So this, you can just think about it as a feature representation of the function, a simple function based on complicated infinite-dimensional features. It is a more closed-form function. And what is interesting about MMD or maximum mean discrepancy or the RKHS, this choice in general, is that you don't actually have to optimize the discriminator. So once you define a particular type of space to optimize for, the optimal solution can already be represented. So meaning that in this case, you skip the inner loop of optimizing the discriminator, which makes this whole process more unstable. And you're just ending up with a very stable optimization process that still minimizes the distance between distributions as you're trying to learn with the generator model. Of course, the downside of that is that you chose this space of functions a priori. So you don't have the space to maybe optimize it to the best case scenario. But in our experiments, we didn't find this to be a huge problem. So essentially the easier way to interpret how our approach on inductive moment matching works versus what consistency model is doing is that it is a generalization of consistency models in the sense that it does distribution-level matching, whereas in consistency models, the distribution matching is based on a single point. And of course, you can't easily represent a distribution with a single point. So it becomes a more degenerate case. So this kind of explains why in certain cases, consistency models are unstable to train. And we actually did ablation studies controlling the amount of samples we use to compare distributions. And in one sample, the consistency model is unstable. With 2 samples, it is also a bit unstable, but it becomes unstable later. But with like 4 samples or more, the training process actually becomes more stable. So that's how we get into this like one-stage process that has high-quality generation.

Maybe just last question, because I know we're at time. Just looking forward, you guys are obviously deeply invested in multimodality. I would love to understand like how you think about multimodality broadly. And where you think it's going?

We probably don't have time to explore this answer, to be honest with you. This is like the entire foundation of the company. But I will say this, which is that like currently what people think of multimodality is not really it. These are language models that have been fine-tuned to like work with image, work with audio, work with video. And it shows some capabilities which are actually really beneficial in the context of like whatever the thing the language model is doing. That's very good. But as you're seeing, individual capabilities like understanding of objects, like object recognition or segmentation, or like just being able to understand what's going on, following long threads of like things people doing, causality, all these kinds of things. These understandings in multimodal models is still worse than dedicated computer vision models. And that tells you that this might not be the approach. That we're not quite there. It would be like if we built all these language models, but they were still worse than RNNs at interpretation of language or natural language, right? That's not the case. Language models are fantastic at that. What we're seeing basically is that, yeah, this is a very promising direction. And obviously the bag-of-tokens approach is extremely helpful, but we need to look beyond just language backbones and trying to retrofit everything to it. Luma's approach is very different, and we are coming at it from the direction of like this unified singular latent space where we can think and reason about all these different pieces of information as if it's one. And technically it is one, right? Nature doesn't make a distinction between video and image and audio and things like that. These are just signals. They just happen. They're all part of the same simulation. They're all part of the same environment that we are in. And then an action. Produces these signals in all of these different modalities, exposing different facets of its existence and occurrence. So we need to think about it in that same way. And currently we see that like most of the industry is extremely short-sighted when it comes to like thinking about multimodality or, well, for good reason, by the way, right? There's so much to be done in language and people should continue to do that work, but there's a new approach that is necessary to actually solve multimodality. Right? And that, that's what we are taking. It doesn't answer all your questions, but we don't have time. We'll come again next time.

Okay, cool.

I'm looking forward to part 2 already. So for now, I'm John, John Min Song from Luma Labs. Thank you for being part of the Cognitive Revolution.

It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please don't hesitate to reach out via email at tcr@turpentine.co. Or you can DM me on the social media platform of your choice.