Lin Qiao (Fireworks AI): Build a Moat by Owning Your AI Stack

Hey everyone, it's great to meet you here. I'm Lin Qiao, co-founder of Fireworks. Today I would like to talk about something special. As every one of us are building agent of our own, and you should own your intelligence, own your intelligence in building your own agent. And we all agree actually 2026 is the year of agent. On Fireworks, we have seen so many different companies, so many variety of agents being developed, empower the transformation all the way from coding agent. As we know, software, uh, development has been significantly disrupted and even changing how we think about, um, SaaS as a whole segment. There are so many agents that process tons and tons of documents to make our, uh, workflows and, and process much more efficient. And we have been seeing companies disrupting sales and marketing, giving an ICP profile. Their agents can go out and source all different companies and find the right stakeholders and target, send a precise message targeting at them. And we have seen agents being developed giving, uh, your talent, pro— uh, talent profile you want to hire and automatically find those ideal people that you can interview. And even they automate the interview process. And we have seen so many agents trying to build customized, um, customer service for their specific company. And this list keeps going on and on across retail, finance, legal, healthcare, and various different parts of our life. And here I want to tell you some specific stories of customers building on top of us. With a raise of hand, how many of you are using Cursor? I guess there's a lot of competition in the coding agent, coding copilot space. Uh, but yes, Cursor built on top of us. Uh, they started with us when they were very small, and their flagship, um, fast code generation product was the first one onboard on Fireworks, and now they're building all sorts of software development agents that is also being on top of Fireworks. And they use Fireworks to scale from small to very big now. And I guess many of you use Uber or Lyft. So Uber is the biggest delivery and mobility company in the world. They have been starting to develop agents from way back. Um, and they have built something like, um, customized, personalized customer support, uh, driver onboarding, zero-day support, uh, and internal data analytics and so on. Shopify is the fourth biggest e-commerce platform across the world, and Shopify enable merchants to have their own platform have their own personalized experience of, uh, of people buying from individual merchants. They build this agent, agent platform called Sidekick where, um, people can, the merchants can customize and build specific appearance of, uh, and experience, uh, for, for their specific customer to, to come to their site and, uh, and have, uh, unique experience. Um, but also generate code to, to define the theme, the theme of, of of the experience they want to have and Shopify build on top of fireworks. So this is something interesting that's happening fundamentally different from before. Search has been disrupted. A lot of our day-to-day life, heavily digital life, heavily indexed on search and search in e-commerce field before it's, um, it's very precise, it's keyword-based, and you ask very specific question. For example, I want to find a red Nike running shoe with this size. But now you can change that into natural language. What's the best running shoes for the high arches that is good for trail running, right? So, so then it will be able to figure out what is the best product and do the match for you. And in order to drive this, and we are seeing so many companies, their portal of access to their merchants are through a traditional search engine, a Lucene, usually Lucene-based search engine, into an NLP or LLM-based search engine. And those are massive transformation that's happening across the board. And second, we have seen a lot of coding agents, it generate a lot of code based on what you type, based on kind of, uh, learning from the project you are, you're, you're working on right now to a prompt-based coding agent that a, not just a product manager can code, an accountant can code, a lawyer can code, and so on. So that's kind of one of the most exciting shift I've seen from my GNA team that is using vibe coding to make themselves more productive. And this is real. And we have many companies using Firewalls to build those kind of capabilities. And as we work day to day, there are a lot of content not— that's not in digital text. It is in tons and tons and piles and piles of documents, whether it's physical paper or it's kind of in images, PDFs, and so on. Extracting intelligence out of those content is a critical part of workflow automation. And typically historically we have deploying people to read those content and convert those, extract information out of those content before it can be processed by other softwares. This is very slow. For example, there are legal firm or insurance claim that is going like lawyers has to sift through a lot of documents of the insurance itself and the patient records and then write memos and summarize what to do. It's a lengthy but also extremely expensive process. And today that can be fully automated through a new modality of, of GenAI called vision language model, that a language model that can see and understand content from non-text base. And many companies use this capability to build or renovate their new product offerings to be much more efficient and automated on top of Fireworks. So in our history right now, we, we go, we, we go through long education or specialization into the areas we focus on to deliver domain-specific value and impact. For example, we have to train for a long time to be a doctor. We have trained to be a long time for to be a lawyer. We have to train for a long time to be, uh, a software developer and so on and so forth. However, this is also being disrupted. There are so many areas of a domain specialty that can be much easier to get extracted and make people more, much more efficient. There are many those kind of agents focus on marketing research, focus on health condition diagnosis automatically. And the paralegal and case management can be semi-automated or fully automated. Personalized customer service is no longer a dream, and there are even scientific research that can be significantly accelerated. The magic here is all of that can be enabled through a simple chat interface, and there are many company building all various kind of, uh, those kind of agents built on top of Fireworks. So there's one common pattern how they're building those agents that's very unique today. One principle they'll be using is they do not view model just as API, and they build on top of that model API as a utility, and they think about model as part of their product, and they're doing product model co-design. There's a fundamental reason why they're doing that this way. The fundamental reason is if you look at the foundation model, the foundation models are powered, the intelligence came from public internet and labeling companies label data. If you look at the world's totality of world's data, that is only a very small fraction of what's available. More than 90% of world's data is private data locked inside application, locked inside enterprises. So by definition, your application and the foundation models, the data distribution are misaligned. It's very important to have the process to have this model continuously learn from your private data. And pick up your domain-specific intelligence. And in the same process, it can run much faster and much more cost efficiently for you to scale quickly to a lot more users of your product. So model will become your IP. And we know building an agent is very complicated process. It always start from you're testing your product ideas, testing your product-market fit to once you hit product-market fit, you want to start to customize and build a model as your IP to you want to further scale your product to millions of consumers or billions of consumers. It is a complicated process and we are here to help all of you to build and scale your agents. First build, There's tons and tons of models available these days, and the model depreciation cannot be faster. Every 2 to 3 weeks there's a new model pop up, whether it's closed or open, to top a leaderboard. And we have so many customers coming to us and saying, what do we do? Do we keep following them or do we just pause? So it has been exhausting but also exciting at the same time that our innovation cadence and velocity is so high right now. Fireworks is always on day zero launch of the best state-of-the-art open models. And in this case, uh, we even launch faster than, uh, than Google for their own open models. Along the way in the past 2 years, we have enabled different modalities. All the way from large language model to image models to embedding model, vision language model, as I just talk about, audio models, and we enable video generation models. It's not just a 200s of models, we're scaling it to cover 1,000, more than 1,000 models. So you have unlimited option to select and pick the best one for you. It is amazing year in the past one year. It's amazing year for open models. It's amazing year for open models for you to own your own intelligence and treat the models and customize the model to become your IP. This is just a quick snapshot of what has been launched within a span of two months. Variety of top-notch open models from OpenAI, from various different other model providers, they all are topping leaderboard of some aspects. And as you can see, the, the columns are representing the model characteristics. They're all very different, they're all very complicated, and enabling those models to test, to build into a production stack is not a a small feat. And that's where with Fireworks we enable those for you so you don't have to worry about any of those, and you can access those state-of-the-art models through OpenAI-compatible API or through Anthropic-compatible API and easily plug those models into your day-to-day workflow. Welcome to, come to Fireworks, and we offer experimentation platform, and that's where you can have a taste of everything we offer. You can select various different models, various different modalities. You can do, um, CI/CD based and test your customized model quality evolution along with your application and continuously tune it. So we have open source project called Evo Protocol. To help you connect the dots across the board end to end. Okay, I will skip those details. Come build it with us today. Our customer told us our experimentation platform give them 100 times faster iteration to do product market fit experimentation. We onboarded more than 10,000 companies on our public platform, and that's 20x growth within a year. After you start building and you like your product, and you will start to customize your product and the model to own your IP. So here, Data Flying Wheel is a key differentiation, and the Data Flying Wheel is going to address a lot of concerns because In current, as you can see, application development cannot be easier, cannot be faster, cannot be more automated. Application itself is not the mode because it's very easy to get copied. And the data extracted from application describing, understanding your user intent, your customer engagement, why, and the preferences are the real mode. You should, uh, you should build on top of. And here, well, I'm gonna share with you how you create this data flying well to power your product. And the mechanism is quite simple. The concept is quite simple. The concept is you use your data to tune the model and let the model learn. From your data. Let the model learn from your specific vocabulary. Let model learn from your specific business logic so the model cater towards your use case. And for last year, for 2 years ago actually, we started launching supervised fine-tuning, and today it's very mature and being used by many customers already. On top of that, last year we have launched a new technology called reinforcement fine-tuning. This is similar to how we humans learn knowledge over the years, is through positive reward and negative rewards. And I will touch on a little bit details later. Okay, want to show you— oh, okay, here we go. So the idea of reinforcement learning is very, very simple. So imagine we go back to 200 years, that we're farmers, and we want to grow the best tomato. And the process is you plant the seed, grow it, and you harvest, taste whatever you like, and keep the seed of the tomato you like, and then keep, keep the, repeat the process. And the end result is you will grow the sweetest, biggest, nicest looking tomato, however you want to select. So this whole process is actually reinforcement learning. As analogy, tomato here is the model. Tomato field is the model for you to plant, um, tomato. And the model output is the tomato itself. And the evaluation to judge is the testing process. So now you, you have this idea, you can map it to, uh, many of your day-to-day agent development. So a lot of you must be building a single, called single-term agent that you just ask the agent the question and they give you back the answer. And actually pretty widely used. For example, the agent can be used, can be built to fix a code. Code means many things. Code means design, uh, code means workflow, business workflow. Code means SQL generation, means many things. Um, and, uh, there's also classification, uh, that can be done as a single-term agent, uh, or writing, or specific kind of writing, um, improvement as, as a single-term agent. There's also multi-term agent, as in, as in you are— you interact with the agent, uh, multiple times to get the best answer out of it. And those multi-term agents either calls into other tools to retrieve or, uh, ask for help and get the best answer. And those tools has been MCTified, or those agents will interact with bespoke logic, your business logic, and give you the best answer. So either way, the current reinforcement learning flow will work well, very well. So here we offer multiple capabilities. One is it's a very complex workflow to manage to, for you to do reinforcement learning. So we manage and host the whole entire, uh, system. So you do not have to worry about that and you just focus on, uh, driving your product development. We offer you full transparency of observability. You understand exactly how the training goes, whether it's going up and right, if not, then what action to take, uh, and how it's gonna impact the metric, your product metrics and so on. And once you train, you like the model and it shows a lot of promise with your offline data, then one-click deploy into inference so you can do A/B testing immediately. Here I want to show you a quick demo of the system we built. And that's the end result. You can see it's progressing really well. And we offer a lot of different ways to look at the metrics. And you can go back to any particular run and test it, test it through a chat API or integrate with your product directly. Okay, can you help me advance the slide? Cannot click forward. Cool. Here I want to show you some great results using this approach. For example, we have many customers, they do code rewrite. And, uh, as you can see, the middle layer, uh, the middle one is the open source model that we get when we started. Uh, y-axis is quality, the higher the better. And the light blue one is the closed model provider. As you can see, when open model started, it's worse quality as expected. After we use reinforcement learning, the quality is more than 10% better. We— that's a massive progress. The benefit is not just better quality, and you can start tuning a much smaller model compared with Claude's model and reach 8 times faster speed and, uh, and much cheaper cost. Usually it's 5 times to 8 times cheaper cost. So you can scale to much bigger number of cohort, um, without breaking your bank. Here is another example where, uh, we, we also started with, uh, this is kind of agentic, typical agentic use case. We're calling into a lot of tools, um, and the tool calling accuracy is extremely important for the quality of the agents. And we tune the open, open model in the middle into also another, um, much better quality compared with a closed model. So more interestingly, the tuning approach is that it can compound with each other, and you can combine supervised fine-tuning with reinforcement fine-tuning. These are completely two different algorithms, and once you combine, magic happens, and, and the result will be much, much better than, um, in terms of improvement. And typically in our system, we already put the combination together so you don't have to, um, it's already enabled for you, uh, to, for you to improve the quality quickly. And this is another example of, uh, tool use function calling. And, uh, in this example you can see the improvement, uh, from the middle one to the end result is massive. Now I would like to talk the last topic of scaling. Once you hit product-market fit, you have customized your model and you need to scale to a huge number of cohort. And this is completely possible in, in this AI era. However, when you think about scaling, you have to worry about so many things. Which hardware do I use? Where do I get the capacity? You, you know, like 2026 and 2027 is gonna be a severe capacity crunch time for the whole entire industry. Not just GPU is scarce, the parts, the memory, the transceivers, all the different parts, supply chain dependency is scarce. And where do I even find capacity? And it's a very complex system stack for you to tune and do inference. And any parts of this system, uh, can have problems. And how do I handle so many failures and issues myself? and how do I optimize for production as I have global deployment? I, I need, I need very low latency across the board, across the globe in different regions. What do I do? So here we want to take care of it. We already run on top of 10 different cloud providers across more than 20 regions globally, and we provide a virtual cloud. By the way, we do not, this is for our internal service. We do not, you, you cannot access our GPU fleet through this layer, but our service is deployed across all the different regions and cloud providers in a virtual way. So, so that way you can, we have a global routing and scheduler for, for load balancing, for disaster recovery, and we take care of those problems and you don't have to. Spend life, spend cycles, spend talent to, to manage that. And the secret sauce of this virtual cloud is how we do inference. Our inference is called Three-Dimensional Optimizer. It optimize for quality, model quality. It optimize for speed of inference. It optimize for cost efficiency. The bad news is such optimization is very complicated. There are different, so many different options, and those options stack onto each other and lead you into more than 10,000 choices for a given model, for a given workload. And find this one choice out of these 100,000 is a very complex task. The good news is we take care of it. And we do this three-dimensional optimization across better speed, better quality, and best, better cost efficiency for you. So again, you do not need to worry about that building on top of Fireworks. So today Fireworks serve more than 13 trillion tokens a day, and this is bigger than Google Gemini's API traffic. And we process more than 180,000 requests per second. And put it in perspective, this is the ballpark of Google search volume. We provide extremely fast speed for you to drive real-time agentic experience. And this is just one of the charts for artificial analysis that our speed is crossing the ASIC, ASIC speed. ASIC is hardware accelerators. Our enterprise customer has built on top of us and scale really fast too. And, uh, we have a food chain company that scale from 1 shop to 100 shop to 1,000 shops within 3 months. And we have digital native companies scaling to 10x more professional customers of themself. Within a few months. And we have a software productivity company scaling from 10 million users to 100 million users also within a span of few weeks. So here's Fireworks. We help you to build, customize, and scale your agent and let you focus on building your best product. By the way, we have a, we have a program for, for startups. Um, and we give you a lot of support if you're a startup co-founder. We give a lot of technical support and also a lot of credits for you to start building. Thank you so much. If you're interested, feel free to, uh, find me after the talk.