LLM Selection Has Become Exhausting (with Lin Qiao)

So when people are trying to figure out these, you know, these relatively small, say, LLMs for a particular task, does model selection really matter? I mean, do your— are your clients like picking, oh, okay, I'm going to use this Llama model of this size, or Qwen, or do they make those decisions, or is this something that's kind of like handled automatically by the autonomous intelligence system?

So model selection matters, but it's also exhausting. So funny thing, there are two interesting phenomena that's so unique to AI. One is the model depreciation is very fast. As you can observe, every couple of weeks, there's a new model launch. When it's closed open, someone topped the leaderboard and then a couple of weeks later someone else topped the leaderboard. And we often, they all are strong in different ways. So, and we start to see the researcher focus start to diverge, right? It is clear some labs are really good at chat-based models. Some labs are really good at coding to use agentic models. Some labs are really good at multimodality models. Some are really good at long context. They'll all start to diverge into focus on specializing in different areas. Speaking about specialization, so different use cases will suit different kinds of model specialty. Very well, but it's really hard for people to figure out which one is the best for my use case. And my use case will also evolve over time. And the benchmark result, public benchmark result is fully saturated and we need to figure out how to pick and choose continuously, which is exhausting. So we do help our customer figure that out. But at the same time, there's another level of complexity. Its hardware depreciation is also very fast. This is something new. Before this wave, usually every 3 years there's a new hardware SKU. And now last year myself, NVIDIA launches 3 SKUs, 3 new SKUs, and 2026 there will be a lot more new SKUs from all different vendors, whether it's GPU or custom ASIC. So then how to manage hardware becomes a very hard problem. And these two combined is causing so much headache to the application developer who want to stay on top of all the different kind of wave. It's just too much deep knowledge and expertise they need to gain in order to kind of pick the best for them. So we are the platform that we abstract out We eventually want to abstract out hardware. We want to abstract hardware selection, and we want to kind of provide the best model for various different use cases. And so basically there are various different kind of mapping from specific use case, specific workload patterns to the model, to the hardware, to the inference setup, to the fleet design. So all of that requires a big team in-house with deep experts, which is really hard to find. And we want to kind of make it super easy for our customers.

Right. So instead of needing to find the deep experts, they can just come to Fireworks and work with you guys, work with your solutions.

Absolutely. And interestingly, there are— so last year by itself, I haven't counted, Every month there are a few new models get deployed and launched, and it has been very exciting. But also I will say early on we bet on open models. That was a big debate in the company that we debate on many what-ifs, right? But because our background in PyTorch, which is an open source project, PyTorch is now, we built PyTorch from ground up. It is now the dominant AI framework. We firmly believe in the power of open science, and it will— in the long run, we believe that is going to really shape the industry in an unprecedented way. So that's why we bet on open models from very early on, and it's great to see that open model performance is converging with closed model. I think 2026 is the year that that convergence will become more prominent. And super excited about that.