Frontier Visual AI: Research to Runtime | Dreamforce 2026

This is gonna be fun.

Yeah, should be.

All right, I can't wait. So in August of this year, Black Forest launched Flux 3. Now, for those of you that don't know, Flux 3 is one model, it's one set of weights, but it generates images. It can do, I think, 20 seconds of video with synced audio. And it also powers robot actions, all with the same model, not 3 different models, you know, it's just one. So maybe we can start with what Black Forest Labs is focused on building and kind of the problems you're trying to solve. Oh, by the way, I'm Nate. I spend my days trying to tell the difference between, you know, the AI that we talk about in hype and kind of what actually works on the ground. And Zion is, you're a co-founder, you're a researcher at Black Forest, And I guess you've been at the center of this whole thing. So yeah, maybe we can start there. What do you focus on building? What are the problems you're solving?

Yeah, so at Black Forest Labs, we really think that visual intelligence is the path forward. Pixels are an extremely high-bandwidth way to communicate and extract information from the world. And so we really try to bring reality into the models and leverage as much of image, video, audio, and text as we can to help people solve many tasks.

So tell us about the vision. So the vision is a single unified model across, I think, image, video, and it gets into audio, but there's also a physical component. So maybe open that up for us a little bit.

Yeah, I mean, so if you think about Like if I, if I pick up a, a glass of water and there's ice in it and it's clinking around, you can hear that. And you can, um, you know, me putting this on a table, there's a physical relationship between this table and this cup. And so it's a very natural next step to say, okay, if this is already modeled by the world that we're experiencing, then we should be able to get the most out of that.

Yeah.

And, um, you know, if you can have a video of me putting this on a table, then we should also be able to extract the actions related to that. And, uh, yeah, and the audio. So, yeah.

So it's sort of, uh, I think the metaphor I've always been told is that humans may not be able to do calculus on paper, but we do a lot of calculus when we catch a ball.

Yeah.

And like you just reach up your hand and the ball is there and your brain does all of that. And so it feels like that's sort of analogous is the—

Yeah, yeah, exactly. And it's very, Natural and intuitive. And I think that that's, yeah, that's really what we're trying to bring to these models is like give them an intuition of the world and the relationships.

You know, it feels like world models are starting to have a moment in the way LLMs were having a moment a couple of years ago. It feels like they're starting to bridge over from, okay, this is in a slide deck, okay, this is in a demo video, to this is really happening. How has that felt for you? Does that feel like that's sort of the moment we're in? Do we feel early? Like, tell me a little bit about that.

Yeah, I mean, when I, when I, when I first started, it took 30 minutes to generate like a tiny image. And, and like, I was so excited that like if I generated a field of cows that like 30% of them had 4 legs and most of them had, uh, yeah, yeah. So like, that's, that's sort of where it started. And then You know, as, as I joined Stability and, uh, started working on these models, like, uh, Yeah. We, we started to see the path forward. And, um, and then at Black Forest Labs, we were really able to, to really hone in the vision of, okay, the world is visual, the world is audio, the world is physical. Let's try to bring these together because at the end of the day, One model connecting all of them is going to be the most efficient way to apply them to downstream tasks. Right. Yeah, right.

No, that makes a ton of sense. Maybe we can go to an example next, right? Because I think some of this can feel very theoretical. So I understand you're deploying Flux at, I think, the Audi plant— the Audi plant, sorry— for assembly line tasks. Can you Talk a little bit about what that's been like, what the models are doing on the assembly line, how that all works?

Yeah. So, I mean, ultimately what happens when you deploy a model into the real world is you hit all of the beautiful things of reality, which is nothing is static and you can't rewind. So as soon as you take an action, you've already changed, you know, like sort of the butterfly effect. You've already changed the world. And you need to be able to combine action with understanding in this recursive relationship between them. And so when we started deploying the models to robotics, we realized that we really have to tightly couple the actions and the understanding because that's what's going to ultimately be the most productive in a real-world application on an assembly line. And the second point is that you have to do it fast because you don't want to take all day to sort 3 balls into buckets. And so a big effort is optimizing these models for real-time or close to real-time inference. And so That's what I think the 2 most important things that we are working on is just making sure that the actions that we're having the robot take are meaningful and efficient and done so in a reasonable amount of time.

That makes sense. So I know whenever I've done launches in the past, you know, your product makes contact with reality and surprising things happen. How has being on the assembly line sort of shaped your understanding of Sort of where architecture for world models needs to go next, where architecture for Flux maybe needs to go next to better adapt to the realities that you're seeing?

Yeah, it's a good question. Um, I think it sort of goes back to what Flux 3— what we started out when we, when we wanted to make Flux 3. We, we wanted to make this one set of weights that we could then adapt to later tasks, and we, we really did prove that with this, uh, Flux Mimic collaboration. We took the model that we had been training for creative audiovisual applications, and we had a very strong feeling that that would also be useful for robotic actions. And so we sort of proved that out with this FluxMimic architecture. But moving forward, I think one lesson that we'll take away is that once we sort of prove that you can do these robotic tasks with a model like this, Why not start incorporating those robotic tasks earlier in the process so that it's a first-class citizen from the beginning?

Yep.

And as we said, like audio and visual teach you more about the world that we're interacting with, and that makes a better downstream model. If we add in robotics, then it's just like a cherry on top.

That makes a ton of sense. Um, I kind of wanna move into like talking about Self Flow. So for those that you, you do that don't know, you, you, there's a recent research paper you did on Self Flow. Um, I'm gonna challenge you in a second to explain Self Flow like there's no math. Um, but I did some reading on it and I wanna give the audience sort of my version and you can tell me where I'm wrong. So my understanding is image models more or less used to kind of hire a teacher from another school, so to speak. a separate, like a static model that would teach them what things mean. But eventually you find that that kind of approach tops out. The teacher stops knowing enough, the student can't get better no matter how much compute you kind of throw at it. And so you end up with SelfFlow firing the teacher, right? And saying, you know what, model, teach yourself. And it ends up working out better. Is that kind of approximately right? Is that where we are?

Yeah, I think that's a really good way to explain it. And even more like, Simply, SelfFlow is really just the model that we're training teaching itself. And the way we do that is we just have one version of it that's a little bit older, like sort of giving lessons to the newer frontier version of itself. And I think one way to sort of think about it is like if you're playing soccer or something with your friends and you twist an ankle or something, or you fall over and you have like everybody on the field rush over to you trying to tell you like what you need. Do you need Tylenol? Do you need a splint? Do you? And then you're just like, no, I just need to like sit down for a second.

Yeah.

Um, that's really what we were just allowing the model to like tell us what it needs and sort of guide its own learning process. Because as you said, if you have an off-the-shelf teacher, you, you do gain the benefits of that teacher. So if it was really good at retrieval or relating objects to each other, you do gain those benefits, but you also get the drawbacks. So if we're training a very large model, but you have a teacher that's a 10th of the size, intuitively you sort of understand that maybe there's a mismatch there and maybe it's a good starting point and you can—

Yeah.

Use it to sort of bootstrap the model early on in its training process. But I think a lot of what people found using these older methods is that you would have to drop the teacher off at a certain point throughout training, as you said. And with SelfFlow, we're able to continue that teaching process much later into the training phase. And ultimately, we're squeezing more out of the compute that we already have. Yeah.

And there's some interesting implications here. So I think if I read the numbers right, standard training takes something like 7 million steps. Your previous best method took 400,000. But now if you go into Cellflow, 143,000, so like a 50th of the size. What does that open up in terms of who can get into this space, who can do models? Like, what are the larger implications here?

Yeah, so I think I would say there's 2 big ones. At Black Forest Labs, we're always trying to push the frontier of visual intelligence, but we're also trying to contribute to the space and give back to the community. I mean, I myself came from the open source community and many of my coworkers and co-founders also did.

Yeah.

And so by releasing Self Flow, we're enabling, you know, professors and students around the world or even hobbyists at home to train architectures and ideas much more efficiently and sort of bring the barrier of entry down, which I think I personally love. And I think a lot of us really—

Yeah.

We love the space and we want to push it forward. And then the other side of that coin is, as a smaller team, we're able to say, okay, let's take the compute that we used to take for one model and split it amongst our own researchers. And now we can explore many, um, many different ideas in parallel, much more efficiently and, you know, sort of take the best from, from multiple streams of work and combine them into one.

I just find that so exciting because I think one of the things I get passionate about watching this space is the idea that AI is fundamentally democratizing, and that means that small teams can compete.

Yeah.

Um, yeah, no, super exciting. Um, I think it's always a question with world models, sort of how far you want to walk into this claim, but I'm curious. We've talked about robotics, we've talked about the Audi factory example. When you think about where we're going with this space, and I think that the example for a lot of us with like diffusion type approaches is probably Midjourney. That's the most popular one. Um, how do you talk about where we are in terms of moving from we're just generating media here to no, we're actually understanding like, how does the world work? Right? Like I drop it, it lands. I really understand that.

Yeah. Yeah. I think, um, like, as you said, like it used to sort of just be that we're, we're generating images and that's useful for mood boarding and sort of getting a, a general creative direction. But, um, if I could mention like a previous release we did, uh, we sort of pushed the frontier with Flux context. And that was sort of the first time you could, um, really, uh, reliably take, let's say, a brand or a logo or a character and really direct the output and bring in your own brand identity or a character from your, your show or your, um, you know, your comic or whatever. And, um, now with Flux 3, we're bringing that into video as well and audio. And so now, um, like I'll give the example of like, let's say you have an idea and you sketched out a character and you want to go into your garage and now make an anime. You can like physically act out what you want that character to do, then also give Flux 3 the sketch of your character.

Oh, wow.

And then now you have a self-directed anime from your garage. And so I'm really excited to see how, you know, businesses are gonna use this to much more efficiently, you know, pivot on creative ideas, but then also just enable, you know, the, the kid in his garage to make, you know, the next like awesome series from their home.

You know, I, I think one of the things that's most intriguing about Black Forest Labs is, you know, we've covered already in just this, this conversation so far, You're working with creatives, you're working on the assembly lines. Those normally are worlds that don't meet, right? Like you don't think about being on the Audi assembly line and then making a home movie. What's it been like to be in a space where your customers can range that far?

Yeah, I think it's a really good question. I think they're all connected by just reality. And I think that that's like an important thing to go back to. And it sort of simplifies it for us because we're not, you know, we don't have just a robotics team focused on just training the best robotics model. And we don't have, you know, something, I mean, we do have people who can embed themselves like forward deployed engineers, but really the bulk of our work is just making the best single model we can and then adapting that downstream. And so it sort of becomes natural once you realize that all of this is connected. Yeah. So I think—

And I guess that sort of gets at a question that I've kind of had in the back of my head when I look at smaller labs. You guys don't have infinite compute, you don't have an infinite team. What is it that like helps you to focus? How do you leverage the team? What are some things that you've learned along the way about competing effectively at this scale? Um, and really delivering value?

I think, uh, we've always been obsessed with extracting the most out of our compute budget that we can.

Yeah.

That's why, like, these latent diffusion models have been so popular because it drastically reduces the amount of compute that you need to create a model. Um, and we've sort of made that a very fundamental aspect of our company. And so anytime We're approaching a new task or a new feature, we're always coming at it from the point of view of how can we do this in the most efficient way possible? How can we think outside the box? And having a small team, we actually benefit from not having so many hops between teams, or just the bandwidth of communication is so much higher. And so when we're working on the frontier, ideas can flow much easier between us. Yeah. And, you know, you have to just be throwing ideas at the wall and just seeing what sticks. And so that's why, you know, we're very big on efficient training, efficient inference, uh, and efficient communication.

That makes a ton of sense. And I think it kind of gets at one of the things I've seen, which is that sort of But the— what Paul Graham, I think, said it like, the advantage that a small startup has is speed.

Yeah.

And the quicker you can move, the better off you are. Have you guys seen that play out as you've started to kind of connect with these early customers where you're able to sort of speak to their needs effectively, get into the spaces they need you to be in quickly? And so you were, Mentioning that you're like wrestling with resource allocation for deployed engineers, but that speed still comes through.

Yeah, yeah, exactly. So, I mean, when, when we first started out, there was only like 15 of us or something like that.

Yeah.

Like I was answering emails, I was jumping on the calls. And I mean, I, I think about like, I was speaking with this video game studio that wanted to also make a movie inspired by their show or by their video game. And because we were a small team, I was just sort of able to hop on a call with them and be like, okay, yeah, like, here's our technology, here's how you can best apply it. And because we try to make the models with the end user in mind, it becomes a very natural solution. And it's more just about talking with the customer and figuring out what it is that we can provide them to do, like, to get them iterating quickly on a creative idea. And just now that we've iterated on these Flux architectures, that process of applying it to a business use case has only become easier and easier. So it's actually meant that, you know, you could sort of see these LLM agents enable people, like, in small teams to code quickly. Well, the Flux architecture has enabled our small team to be helpful for many customers. And then I could then help those customers help their internal teams work even more efficiently.

And I imagine that must feel empowering for your customers because, you know, the major labs can't sit down and say, we'll change our model for you, right? Like, that's not— That's not sort of what they're doing, but you guys can sit down and say, let's have a conversation. And that's super cool.

Yeah, yeah. And I think that that's, I mean, personally, it's like I spend so much time on research, but I love talking with the customers because there's been so many times where they've said something and I think about a research problem I've been working on. And it just like, it either like completely throws out the idea, which is great because it means I don't waste time working on it. But it has also been times where You know, to them it is so obvious what the user interface should be or how they should interact with it. And so I love like talking with the customers and bringing that back to the research team and really drilling home the fact that we need to be doing like user experience driven research because at the end of the day, we're not doing like theoretical PhD thesis.

Yeah.

We're making a product at the end of the day. And so we really do need to think about how these models are gonna be used and, yeah, and deployed in the real world.

And that's been what's so exciting, kind of digging into this in preparation for this talk, like just getting to see the sheer range of the different customers you work with. And I find that like divergent use cases tend to produce more resilient business models, tend to produce more resilient products. It just makes me really excited as I kind of dig into the Flux use case, kind of where you're going, because I see that full range there. I think it's super interesting.

Yeah. Yeah. I mean, we've, we've seen everything from like Martin Scorsese use it to storyboard, uh, ideas for a future movie.

Oh, that's so fun.

Yeah. And, uh, I mean, obviously the, the robotics demo is very fun as well because it's, it's very tangible. Um, but I, I, I also just, yeah, I think about The people in between sort of a creative and a business solution and sort of, they were always the glue between a product and like a creative process and really enabling that person to communicate a creative idea to the artist. And then also just like flush out an idea much more efficiently. And I think that's what really what we see our previous Flux models be the most impactful is just enabling somebody who considers themselves creative and has taste, but they're not sure exactly how to communicate that. We've seen Flux like really unlock those individuals to ultimately like build the identity and the brand that they want to.

Yeah, yeah. It gets at one of those like big themes I've seen in 2026 around scaling taste and what that looks like. That's, yeah, that's exactly where that's going. Yeah. Maybe we go back in time a little bit. It's very difficult for me to do that because AI is like up to the minute. You get all the news, but let's go back 12 months. 12 months ago, where were you? Where was the product? And then maybe we can take a little jump forward and say, what has happened in the last 12 months? What have you learned? What have you discovered? How have you evolved? We can get a sense of, I guess, the trajectory for Flux in the last 12 months.

Yeah, so I mean, we, we build amazing image models, and the people who have used the older versions of the model are very familiar with the idea of prompting and, and inputting text and getting out an output. And we have seen some amazing examples of what people have been able to do with this. Um, but actually, it was probably about a year ago where one of my co-founders and I sat down and we were like, Okay, we need to kill the text prompt because—

Okay, yeah, we talk about it, but now you did it.

Yeah, uh, because it— I mean, for one, it's kind of like annoying to type out your ideas, and then like some words have more power than others. And, you know, if you're just trying to get something out the door, you don't want to have to memorize the best prompt structure. And so we've done a lot of work to sort of strip back and take a step back and say, okay, what's a very natural way to interact with these models. And so that's why we released Flux Context and Flux 2, which, as I said before, sort of enabled people to upload images and use those in the generation process and actually build up like an internal library of assets where you're, over time, as you interact with Flux, you're sort of building up a library of your own.

Yeah.

Like mood board and sort of pulling from various inspirations. And as we move forward, we just want to continue to push that. And so, you know, now we've added video inputs and video editing. And yeah, just bringing that, bringing the time to first generation as low as possible and reduce the friction as much as possible. So yeah.

Let's dive into that just for a second because I don't think people fully recognize, like we always assume, well, if it's going to be a multimodal, it will be a multimodal output, but you're saying here video can go in.

Yes.

Yeah, that's super cool.

Yeah, as video in, audio in, image in, which means that, like I said before, if you're like, just want to jump in front of a white wall and act something out, you can totally do that. Or if you've already shot something and you're like, oh no, there was a boom mic in the background or something and we completely forgot about it, or that guy's wearing a branded t-shirt, we don't want it in there. You can just upload the video into Flux and ask it to erase it. Or I don't know, you just wanna prank your friend and turn their hair green. Right.

Yeah.

Like, the possibilities are endless. And, um, yeah, just giving people the control. And I think it's a much more natural, um, uh, user experience because we have phones in our pockets. You can quickly snap a picture. you know, you're walking down the street in like Korea or something, you see a very cool like mural or something, you can take a photo and say like, okay, I want that for inspiration later.

Yeah. Yeah.

Yeah.

No, that makes sense. And when you talk about speed and sort of a focus on speed and speed and generation, can you give us an example of like the difference you've been able to make with that focus in the last 12 months?

Yeah, I think, I mean, our infra, team has done an amazing job of optimizing the models. Shout out to Kyle and Tati and everybody working on that team. Many sleepless nights just optimizing the backend.

Yeah.

So that's sort of one aspect of it. But then we also have an amazing distillation team, which essentially just means we train this big model that's super smart, and then we distill that knowledge down into a much more efficient architecture. or a much more efficient way of generating outputs. And I think that we're always pushing on that frontier to sort of go back to the efficiency as a first-class citizen within our company. We're always pushing to make it as seamless of a user experience as possible.

Now, if we look ahead, like we jump ahead a little bit in time, do you think that emphasis on like Making sure that it's a low-friction experience, going beyond the text prompt, making sure that it's as fast and smooth as possible. Are those themes you expect to see continuing over the next 12 months as you work, or do you think there'll be new themes that will come out as you go?

Yeah, no, I think 100%. I think as we, as we're able to push the frontier of like inference speed, that lowers the The sort of time to response from the model. And so you could imagine a world where as you're speaking, the model's updating in real time.

Oh wow.

That was controlling a robot in real time.

Yeah.

This is sort of like pie in the sky stuff. Yeah. Yeah. There, there is a clear path towards that now. And I think now it's just a matter of really, uh, innovating on the same recipe that we have and, and pushing the frontier.

No, that makes sense. Was there, like, if we look at kind of lessons learned, was there an assumption you made about scaling or capabilities over the last year or so as you were building that turned out to be incorrect? And you're like, wow, I'm glad we learned that. We actually had a different perspective earlier and now we've really sort of unlocked something new.

Yeah, it's hard when you, when you go into a new, new era of models, we'll say, like, Yeah. You know, Flux 1 was just an image generation model.

Mm-hmm.

And, um, and now as we add more modalities and more inputs and outputs, you have to make certain assumptions about what you think is the best thing to do. Yeah. So we dedicated a lot of compute to video, and I think that was a good bet because we have an amazing video model. Mm-hmm. We've been praised constantly about how natural the videos look. And so, but I think moving forward, you have to be very careful about how you balance the different modalities because images are important and audio is important. And if you could have the most natural looking video ever, but if the audio sounds like it's a robotic voice, then that's not super useful. Of course, you've already taken the time down by generating a video. But it would be extremely useful if it could be, uh, as equally good in all of the modalities.

Yeah, no, that makes a ton of sense. It's funny, my, my, uh, algorithm must have known that we were doing this conversation because it was feeding me notes from a sound designer for the Lord of the Rings movies. And he was talking about the obsessive detail he put into each of those sound pieces. And it got me thinking, like, if you're watching something and it feels immersive, You never think about it as a user, but from your perspective, designing the model, you have to think about how to allocate and sort of enable a model that would create those kinds of experiences.

Yeah, 100%. Like, I think, I mean, at BFO, we're perfectionists. And so like, we really want to, like, again, we're thinking about where this is going to be used. And so yeah, like 100%, like, like audio needs to sound natural. It needs to, it needs to feel like it's there. And when you looked at all video models that didn't have audio, it was already such a big jump just to add any audio whatsoever. But I think that goes back to sort of the cool demo thing where it's cool to see a video and audio being generated together. But if you want to actually use it in a business or use it in a real use case, that audio needs to sound natural and it needs to sound like it belongs.

It does. That's exactly right. Well, I think that's a I feel like I've learned a ton. I hope you guys have learned a ton. I guess I leave really excited. I think the idea of visual intelligence is really energizing. I think combining that with the world model and seeing where we go with robotics, it feels like there's really exciting stuff in the next 12 months for you guys too. So I'll be excited to follow along. Thanks for joining, Zion. It's been a super fun conversation.

Likewise.