In this interview, Lin Qiao, CEO of Fireworks AI, explains why the next generation of generative AI will rely on hundreds of specialized expert models rather than a single monolithic model. He emphasizes that latency and cost are critical for B2C and developer‑facing applications, and Fireworks addresses these challenges by offering a tiered catalog of models across text, audio, vision, and embedding modalities. Qiao describes the rapid depreciation of both models and hardware, positioning Fireworks as a platform that continuously evaluates cutting‑edge models and hardware (GPUs from NVIDIA/AMD and custom ASICs) and integrates them for developers, freeing them from low‑level complexity. The company’s new F1 system automatically decomposes complex tasks (e.g., medical claim summarization) into sub‑problems and routes each to the most suitable expert model, providing logical reasoning across the model suite. Qiao also discusses Fireworks’ competitive stance against hyperscalers, citing the company’s lean, agile approach as a differentiator. The partnership with MongoDB is highlighted as a natural fit: Fireworks supplies the inference engine while MongoDB provides industry‑leading vector search for Retrieval‑Augmented Generation (RAG). Finally, he outlines privacy‑first deployment options—hosted enterprise APIs and on‑premise installations—to meet data‑sovereignty requirements for regulated sectors such as healthcare, finance, and insurance.
Chapters
The next wave of quality is not going to be single model solve all problems. The next generation of quality, there'll be hundreds of small expert models solving a narrower problem. Then we focus on solving the other two problems, latency and cost. Why is it important? Because GenAI as a tech is gonna power a lot of B2C, to consumer, to developer applications. And those applications has to be hyper interactive. It's a very hard problem because GenAI models are on the highest spectrum of model complexity and size. And cost is challenging because as those applications can quickly scale, it's B2C or to consumer, they don't want to quickly scale into bankruptcy. And that's literally the conversation people are really worried about. They have a great product-market fit, but they cannot scale. As a platform, we offer tiers of models. One is self-serve. You don't talk with any of us. We offer hundreds of models across all modalities, from large language model to audio models to vision model, which can see, to embedding models, which can do semantic search. That's the area we work very closely with MongoDB, to image generation model, from text to image to from text to video. Huge variety of model with different modalities. And with different specialties.
What's the risk that the hyperscalers come and eat your lunch?
There's always this potential risk. I think if this risk is always come true, then there will be no startup in B2B business. But as a matter of fact, I think because we are running very lean, we're very agile, we're very leaning into being the cutting-edge forefront of new development, I think we see things much, much deeper and faster than hyperscalers. And that's where we are confident we can outrun them by providing the best developer platform for GenAI.
This is a world that's evolving so incredibly fast, it's almost hard to keep up. I think it's overwhelming for a lot of people. How do you stay on top of, you know, what's coming through? There's new models still coming through all the time as well. And how do you decide what you're going to add to selection of services that you serve to your customers on that front?
Yeah, so that's a very interesting, very dynamic market. From model side, I would say model's depreciation is much, much faster than any model builders have thought about, have imagined. Almost every week there's a new model breaking some record in certain dimension, but not just models. The hardware is also moving very fast. I would say hardware depreciation is also getting much faster. In the past, like every 3 years there's a new hardware scale and now it's every 1 year. Across multiple vendors. So the hardware space is also very busy. So if you look at the whole entire space for the developers, then you figure out which model are they catching up with the new models, are they catching up with new hardware? It's exhausting. As a platform, we want to be the place where we handle all this complexity. We evaluate the cutting-edge model quality in specific areas. We evaluate all the different, various different hardware offering, and we do all the integration. So our developer just focus on solving their problems by using our tools. So that's kind of the ease of use, making AI more accessible path.
Forgive me if this is a stupid question, but you said we do all the integrations. What do you mean by that?
On the hardware side, we build on top of the various different kind of GPUs delivered by NVIDIA, by AMD. We're also looking actively into custom ASICs where they can significantly accelerate certain kind of model. Depends on the workload pattern, whether they're bottlenecked by memory bandwidth, by flops, by network. Then we can pick and choose how to kind of which hardware is the best suited for a certain kind of workload pattern, for certain kind of models and so on. It's never a single model solving all problem they need. For business task, they need to decompose a complex task as in, for example, how to automate medical claim legal summarization, that problem. The problem has to decompose into actually at least 3 subproblems, if not more. And each subproblem They need to figure out, oh, which model is the best to solve these problems? Do I need to fine-tune or do I need to prompt engine my way? Tomorrow there's a new model coming up. What should I do? So we are trying to obfuscate all these problems by saying, for this problem, we can automatically decompose into small subsets of problems and find the best model to solve that problem. So we're hiding all the complexity. That's kind of the new system we released recently called F1 to build on top of those hundreds of more expert models and have a logical reasoning system to be able to combine the total intelligence offered by all these models without user figure out the low-level details.
Let's talk a little bit about the partnership with MongoDB. How do you partner with them? I believe they were also an investor in your last, you know, your Series B.
That's right.
What's the partnership look like? What are you trying to achieve together and where do they fit into the Fireworks story?
The partnership with MongoDB is we are industry-leading GenAI inference platform for LLMs. MongoDB is the industry-leading vector search engine for RAG and so on. So that's kind of very natural match for developer to get access to the best platform end-to-end.
I think from my limited understanding of this world, that one of the challenges around RAG and one of the things a lot of organizations are doing when they're taking a RAG-based approach is they're working really hard to keep their data on-premises because it is often proprietary data and they're concerned about pushing it out to big large language models in the cloud. With your kind of unified F1, is it? Mm-hmm. How do you make sure that when you're working with companies who take a RAG-based approach that they know, particularly because you're working with so many language models and it's being different workloads are being farmed out in different directions, how do you make sure that that data stays safe and sovereign and proprietary or whatever, however customers want to consider their control over that?
Yeah, so privacy is absolutely top of our concern. So on one hand, we have hosted enterprise API for ease of use, then our customer don't need to manage anything. We manage everything for them. On the other hand, we can also deploy into customer's premise. That's where kind of data plane lives in their premise and no data leaves their premise. And to many of our enterprise customer, that's the only way we can deploy. And we have successfully deployed into many different segments, including healthcare, finance, financial services, and insurance those kind of different market segments.
